An exploratory study into qualities of outstanding teaching as perceived by undergraduate students revisited.

Baglione, Stephen L. ; Avakian, Aram ; Danikas, Stephanie 等

INTRODUCTION

The perception of teaching excellence has been studied by researchers for decades, yet what is an "outstanding" professor remains elusive. Teaching excellence is becoming more elusive because of larger class sizes, greater pressure to publish, and expansion of globalization and technology (Smart, Kelly, & Conant, 2003). One problem is that faculty and students differ on what constitutes teaching excellence (Grunenwald & Ackerman, 1986; Kelly, Conant, & Smart, 1991). Students perceived faculty communication and subject knowledge as paramount (Grunenwald & Ackerman, 1986). A review of master teachers in marketing found them engaging and caring in the classroom because of strong communication skills (i.e., enthusiasm, humour, and enjoy teaching), real-world experience, and availability (Conant, Smart & Kelly, 1988). They have an interactive classroom style coupled with fast and constructive feedback. Their syllabi are detailed and comprehensive. Smart, Kelley, and Conant (2003) found master teachers associate their teaching with strong communication skills, a real-world perspective, carrying/empathy, an involvement orientation, and organization preparation (p.77). The emergence of technology is also important, which is different from the 1988 study. A similar study with students found teaching excellence associated with communication skills, caring or empathy, real-world perspective, knowledge, organization and preparation, availability, and fair evaluations (Kelly, Conant, & Smart, 1991).

REVIEW OF LITERATURE

Dimensions of Teaching Excellence

In contrast to previous descriptive studies, Faranda and Clarke (2004) used in-depth interviews with upper-level business students to assess student perceptions of teaching effectiveness. Five "themes" or dimensions were identified: 1) rapport; 2) delivery; 3) fairness; 4) knowledge and credibility; and 5) organization and preparation, with rapport and delivery cited most. Students desire to build relationships, especially ones of mutual trust and harmony, with their instructors. They develop great respect for the professor who listens well but also enjoy occasions when the instructor 'opens up' and tells students about him or herself, sharing personal and professional life experiences (p. 275). Rapport has four subcategories: 1) approachability; 2) accessibility; 3) personality; and 4) empathy. Delivery deals with how the instructor presents and conveys subject matter. Energy, passion, and enthusiasm for the subject matter are all rudimentary for successful delivery (p. 277). The subcategories for delivery are communication, personal style, and pedagogy. Fairness, the ability to demonstrate just, equitable, and impartial treatment in the classroom, for example through grading, workload, and treatment of students during class (p. 277) is the third theme. The subcategories are performance evaluation, and assignments. Good professors, according to students, are knowledgeable and credible, with subcategories of expertise, experience, and intelligence. Organization and preparation is the final dimension, and focuses on relaying knowledge, creating expectations, and motivating. The subcategories are clarity, thoroughness, and instructional materials.

These dimensions are consistent with and expand upon Brown's (1975) study, which identified three preferences by students in faculty: personal warmth, intellectual skill, and academic rigor. Faranda and Clarke (2004) broaden the teaching factors identified by Grunewald and Ackerman (1986) by emphasizing a strong desire for approachable and organized professors. These dimensions are congruent with the seven found by Kelly et al. (1991) and four service quality factors (i.e., responsiveness, reliability, empathy, and tangibles) of Allen and Davis (1991). This study confirms Desai et al.'s (2001) findings on the importance of faculty-student interaction and general classroom management.

Specifically, student responses on the instructor's level of enthusiasm and the maintenance of fair and impartial standards are consistent with prior literature (Desai et al., 2001; Kelly et al., 1991). Faranda and Clarke's categories mesh well with the call to incorporate active learning in the classroom (Race, 1993) and the belief that active learning "promotes the development of students' cognitive and communication skills (Exley & Dennick, 2004, p. 2). These same qualities are found in Revell and Wainwright (2009) identification of what makes lectures unmissable: a high degree of student participation and interaction; a clear structure which enabled students to identify key points and make integrative links with other areas of the course; and the passion and enthusiasm of the lecturer, and the degree to which she/he can bring a subject to life (p. 214). Korthagen (2004) argues that non-traditional competencies are essential for good teaching: empathy, compassion, understanding and tolerance, love, and flexibility. Conant et al., (1988) list all five dimensions from Faranda and Clark: knowledge, delivery (communication, applied projects, real-world perspective, and involvement orientation), rapport (caring/empathy), organization/preparation, and fairness (challenging and fair evaluations). The dimensions coincide well with Bruneau and Campbell (2002) review of Generations X and Y students where interactive, stimulating, practical, and personal education works best.

PROPOSED HYPOTHESIS

Understanding what constitutes teaching excellence allows professors to tailor courses and create an environment conducive to learning, thus, the researchers focus on the beneficiaries of outstanding teaching: students. The researchers expand upon these dimensions to include differences by gender and class rank (i.e., freshman/sophomore and junior/senior). (Major was identified by Faranda and Clarke as a possible moderating factor, but we did not sample enough from non-business majors in our original study.) The contribution of this study is replicating Faranda and Clarke's work (2004) quantitatively, and forcing students to trade-off among dimensions of teaching excellence. Based on prior research, the hypotheses are as follows:

Hypothesis 1: The five dimensions of teaching excellence will be of similar importance between student respondents in 2005 and 2010.

Hypothesis 2: The five dimensions of teaching excellence will be of varying importance and knowledge/credibility will be the most important dimensions.

Hypothesis 3: The five dimensions of teaching excellence will differ by gender.

Hypothesis 3a: The five dimensions of teaching excellence will be of varying importance and knowledge/credibility and fairness will be the most important dimension for women.

Hypothesis 3b: The five dimensions of teaching excellence will be of varying importance and knowledge/credibility and delivery will be the most important dimension for men.

Hypothesis 4: The five dimensions of teaching excellence will be of varying importance by class rank (i.e., freshman/sophomore and junior/senior).

Hypothesis 4a: The five dimensions of teaching excellence will be of varying importance and knowledge/credibility and rapport will be the most important dimension for underclassman.

Hypothesis 4b: The five dimensions of teaching excellence will be of varying importance and knowledge/credibility and delivery will be the most important dimension for upperclassman.

METHODS

Survey Questionnaire

Through multiple iterations, a rough draft was developed. It was modified repeatedly during the pre-test, a protocol analysis with 13 undergraduate students. The survey was administered in 2005 to four undergraduate business classes and an honor's program general education class. In 2010, the survey was administered to five business classes.

Data Analysis

Data was entered by one person and then reviewed by another to ensure no data entry errors. Frequencies were then examined as a second check to ensure no data points were outside the range of feasible answers. Respondents used a constant-sum scale (100 points) to evaluate the five prevalent dimensions found in the literature on teaching excellence, with higher values indicating greater importance. Respondents were instructed that responses were ratio scale. Brief explanations were provided for each dimension. The data were analyzed in SPSS version 19. A one-sample t-test is used to estimate whether the means are different from an average of 20, indicating equal importance among the five dimensions (i.e., 100 points divided equally among each of the five dimensions). When respondents allocate more or less than 100 points, a constant number of points will be added or subtracted, respectively, to each attribute to make them sum to 100. Differences among the five dimensions on demographic characteristics and year administered are tested through an independent-samples t-test. If responses between 2005 and 2010 are similar, responses will be combined.

RESULTS

One-hundred and twenty-one surveys were collected in 2005; three were unusable because of incomplete data or failure to take the exercise seriously (n=118). Six respondents allocated less than 100 points among the five dimensions. These were adjusted upward to 100 by adding a constant to each. Our sample is predominantly male (63%) upper-level (60%) business student (63%) (Table 1). Almost one-third are sophomores, juniors, or seniors, with a small percentage of freshmen.

One-hundred and 10 surveys were collected in 2010; one was unusable because of incomplete data or failure to take the exercise seriously (n=109). Our sample is predominantly male (55%) upper-level (68%) business students (96%) (Table 2). More than half are seniors, a quarter sophomores, and few freshmen. One respondent allocated more than 100 points among the five dimensions. This was adjusted downward to 100 by subtracting a constant from all dimensions.

An independent-samples t-test comparing respondents in 2005 and 2010 reveals no statistical difference between means as presented in Table 3. Hypothesis one is supported (i.e., dimensions are longitudinally stable). Subsequent analysis will use combined data from the two years. (Note: Knowledge and credibility will be abbreviated during analysis to knowledge, and organization and preparation will be shortened to organization.)

Among the five teaching dimensions, knowledge is the most important (M=22.64), followed by delivery (M=22.32), rapport (M=20.10), organization (M=17.59), and fairness (M=17.35) (Table 4). Using a one-sample t-test to determine whether the dimensions are equally important (meaning each receives 20 points of the maximum 100), the authors found all but rapport statistically significant from 20. Delivery, knowledge, organization, and fairness are highly significant; the first two are above the mean, more important than others, and latter two below, less important. Knowledge is the most important, which supports hypothesis two.

Respondents are categorized on key demographic variables to determine whether differences exist. We examined gender and class rank using an independent-samples t-test. A difference was found for gender (Table 5), which partially supports hypothesis three. Fairness was the only gender difference, with men valuing it more than women. Hypothesis 3a was partially supported. Delivery and knowledge were most important for women instead of knowledge and fairness. Fairness was actually the lowest for women. Hypothesis 3b was supported: knowledge and delivery were most important for men.

Comparing lower- (i.e., freshmen and sophomores) and upper-level (i.e., juniors and seniors) students also did not produce any statistically significant differences (Table 6); thus, hypothesis four is not supported. The most important dimensions for underclassman were delivery and knowledge. Hypothesis 4a is partially supported, because rapport was actually less than 20 (M=19.43). For upper-classmen, knowledge and delivery are most important, supporting hypothesis 4b.

CONCLUSION

This research was designed to provide insight into undergraduate students' perceptions of teaching excellence and to determine whether these perceptions differ longitudinally. The variables associated with teaching excellence are longitudinally stable across the five-year period (2005-2010). According to students, subject knowledge and credibility, as demonstrated through expertise, experience, and intelligence, is the most important dimension in teaching excellence. But how knowledge is delivered is almost equally important. Students also believe teaching excellence means that the faculty member is approachable, accessible, and empathetic. Conversely, Faranda and Clarke (2004) found rapport and delivery were the most listed categories. Knowledge and credibility, while most important in our study, ranked near the bottom in Faranda and Clarke. Our analysis forces comparisons among dimensions to determine importance not merely that it was listed by respondents. To a lesser extent, organization and preparation and fairness are components of teaching excellence. At the collegiate level, more learning occurs outside the classroom with most classes only meeting about 40 hours a term. Students expect to learn some material on their own and may believe knowledgeable and approachable faculty will answer questions or deviate from scheduled class exercises when students need more assistance in learning. Within institutions of higher education, checks and balances exist to enhance fairness. Students may discuss problems with faculty advisors, department chairs, and, ultimately, file grade appeals with deans. Also, maybe fairness is not an issue because professor are usually fair.

The range among these dimensions is close. It suggests that no one trait is responsible for making a professor's teaching 'excellent,' but it is a combination that allows students to favor one professor over another. This is similar to what Brown (1975) concluded almost 40 years ago. This conclusion holds regardless of gender or class rank (freshmen/sophomore or junior/senior), although women and men both listed fairness low, with men valuing it more than women.

Therefore, the research suggests that knowledge and credibility must be evident to display teaching excellence. Students are not as concerned about how that knowledge was acquired, just that it is present. Merely possessing knowledge does not translate into teaching excellence. Imparting that knowledge through communication is an important nexus between the sender (faculty) and receiver (student). Even the presentation by the most knowledgeable faculty will generate questions. Students expect faculty to be approachable and empathize with them.

IMPLICATIONS

Faculty should self-assess and be peer reviewed on all five dimensions. They should focus on improving in all dimensions. Knowledge can be conveyed by discussing credentials at the start the semester, developing exercises that illustrate concepts, displaying relevance for the material, and relating to student experiences. Faculty should be accessible before and after class and express interest in helping students to develop rapport (Faranda & Clarke, 2004). Offering personal experiences and being caring and empathetic also helps. Grading policies should be explicitly listed on syllabi, discussed during class, and adhered to throughout the semester. This also aids in displaying organization. Structuring class and discussing objectives when class starts and summarizing learning after are also necessary. PowerPoint slides can aid in organization (Revell & Wainwright, 2009); however, merely reading them is viewed by students as monotonous (Bruneau & Campbell, 2002). Classes must be interactive, which if done properly, addresses all dimensions.

RECOMMENDATIONS FOR FUTURE RESEARCH

The dimensions may have interactive effects. This research suggests that there is no significant difference between the perceptions of male and female students on important teaching dimensions, but further research could examine perceptions when the professor's and student's gender are the same or different.

Class size may also be a moderating factor, with modifications necessary if a class exceeds a certain size (Smart, Kelly, & Conant, 2003). Course duration and delivery method may impact results. Would students in an eight-week course respond similarly to those in a 16-week format? Is the establishment of rapport crucial in a shorter class where time is limited? Do students online as opposed to face-to-face classes differ in their perceptions of teaching excellence (Bangert, 2005)? Do traditional- and non-traditional age students value the same teaching criteria? Non-traditional age student bring a wealth of experience to the classroom that may enhance the importance of knowledge. Does student major and class type (i.e., quantitative or qualitative) impact results? Are students majoring in quantitative subjects (e.g., Mathematics and Statistics) focused on different dimensions? Similarly, there may be an interaction between class rank (e.g., freshmen or seniors) and major. Each class surveyed included a mix of student achievement as measured by GPA. Would students with high GPA, presumably of higher intellect and motivation, value different attributes than students low in both? Future research can examine whether faculty and students would evaluate the dimensions equally (Grunenwald & Ackerman, 1986). This study does not deal with the root cause of perception, be it the media, socioeconomic background, generational, or personality. Finally, the study could be expanded internationally. Liu and Meng (2009) found "evidence that characteristics of a good Chinese teacher are similar to those identified in Western countries" (p. 326).

LIMITATIONS OF THIS STUDY

This study was only done at one non-secular institution with traditional-age students. The physical environment and class atmosphere have been cited as important in good teaching, but were not explored in our study (Parpala & Lindblom-Ylanne, 2007). Additionally, other research in this field of teaching excellence indicates that there are several specific ways in which professors conduct classes which "are more effective than others in motivating business students to learn and to retain information" from classes (Bruneau & Campbell, 2002, p. 11; Bruneau & Campbell, 2004).

REFERENCES

Allen, J., & Davis, D. (Spring 1991). Searching for Excellence in Marketing Education: The Relationship Between Service Quality and Three Outcome Variables. Journal of Marketing Education, 13, 47-55.

Bangert, A.W. (2005). Identifying Factors Underlying the Quality of Online Teaching Effectiveness: An Exploratory Study, Journal of Computing in Higher Education, 17(2), 79-99.

Brown, R. (1975). College Students' Perceived, Effective Teaching Styles, College Student Journal, 11(4): 302-305.

Bruneau, C.L., & Campbell, M.E. (2002). Breaking with Tradition: Motivating Generations X and Y Business Students, Academy of Educational Leadership Journal, 6(3), 11-20.

Bruneau, C.L., & Campbell, M.E. (2004). Insight to Teaching Generations X and Y Business Students, Academy of Educational Leadership Journal, 8(1), 11-22.

Conant, J.S., Smart, D.T., & Kelley, C.A. (Fall 1988). Master Teaching: Pursuing Excellence in Marketing Education, Journal of Marketing Education, 10, 3-13.

Exley, K., & Dennick, R. (2004). Giving a Lecture: From Presenting to Teaching, London: RoutledgeFalmer.

Faranda, W.T., & Clarke III, I. (2004). Student Observations of Outstanding Teaching: Implications for Marketing Educators, Journal of Marketing Education, 26(3), 271-281.

Grunenwald, J.P., & Ackerman, L. (Summer 1986). A Modified Delphi Approach for the Development of Student Evaluations of Faculty Teaching, Journal of Marketing Education, 8, 32-38.

Kelley, C.A., Conant, J.S., & Smart, D.T. (1991). Master Teaching Revisited: Pursuing Excellence from the Students' Perspective, Journal of Marketing Education, 13(2), 1-10.

Korthagen, F.A.J. (2003). In Search of the Essence of a Good Teacher: Towards a More Holistic Approach in Teacher Education, Teaching and Teacher Education, 20(1), 77-97.

Liu, S., & Meng, L. (2009). Perceptions of Teachers, Students, and Parents of the Characteristics of Good Teaching: A Cross-cultural Comparison of China and the United States, Educational Assessment, Evaluation and Accountability, 21,313-328.

Parpala, A., & Lindblom-Ylanne, S. (2006). University Teachers' Conceptions of Good Teaching in the Units of High-Quality Education, Studies in Educational Evaluation, 33(4), 355-370.

Race, P. (1993), Never Mind the Teaching Feel the Learning SEDA, Paper 80 (Birmingham: SEDA).

Revell, A., & Wainwright, E. (2009). What Makes Lectures 'Unmissable'? Insights into Teaching Excellence and Active Learning, Journal of Geography in Higher Education, 33(2), 209-223.

Smart, D.T., Kelley, C.A., & Conant, J.S. (2003). Mastering the Art of Teaching: Pursuing Excellence in a New Millennium, Journal of Marketing Education, 25(1): 71-78.

Smart, D.T., Kelley, C.A., & Conant, J.S. (1991). Master Teaching Revisited: Pursuing Excellence from the Students' Perspective, Journal of Marketing Education, 13(2), 1-10.

Stephen L. Baglione

Aram Avakian

Stephanie Danikas

Saint Leo University

Stephen L. Baglione is Professor of Marketing and Quantitative Methods at Saint Leo University in Florida. He received his doctorate in Marketing from the University of South Carolina. Dr. Baglione has authored or coauthored almost 80 refereed journal and proceedings articles. He was a Fulbright Scholar at the University of Ljubljana in Slovenia. He has won the University Teacher of the Year. He is a multiple winner of the Saint Leo University Research of the Year (five times) and School of Business Researcher of the Year (seven times) and Teacher of the Year (five times). He has also won the SLU Student Government Union Teacher of the Year and Faculty Advisor of the Year.

Aram Avakian and Stephanie Danikas are alumni of Saint Leo University.

Table 1
Respondent Demographics 2005

Major                         Class

Business       63%    Freshmen     7%
Non-business   32%    Sophomore   34%
Undecided       3%    Junior      30%
Both            2%    Senior      30%

Table 2
Respondent Demographics 2010

Major                          Class

Business       96%    Freshmen     4%
Non-business    3%    Sophomore   28%
Undecided       0%    Junior      14%
Both            1%    Senior      54%

Table 3
Independent-Samples T-Test Compared Responses in 2005 with
those in 2010.

STATEMENT                    t (df)   p       mean
                                              (2005)
                                              (2010)

Rapport: approachability/    0.77     .442    20.54
accessibility/               (225)            19.62
personality/empathy

Delivery: communication/     -0.97    .336    21.64
personal style/relaying      (225)            23.05
information

Knowledge and                -0.31    .761    22.44
credibility: expertise/      (225)            22.86
experience/intelligence

Organization and             0.03     .978    17.6
preparation: clarity/        (225)            17.58
thoroughness/
instructional material

Fairness: performance        0.88     .378    17.78
evaluation/assignments       (225)            16.89

Table 4
One-Sample T-Test Estimated Against A Value of 20

STATEMENT                        t (df)    P     mean

Rapport: approachability/         0.17    .866   20.10
accessibility/personality/       (226)
empathy

Delivery: communication/          3.18    .002   22.32
personal style/relaying          (226)
information

Knowledge and credibility:        3.82    .000   22.64
expertise/experience/            (226)
intelligence

Organization and preparation:    -5.66    .000   17.59
clarity/thoroughness/            (226)
instructional material

Fairness: performance            -5.25    .000   17.35
evaluation/assignments           (226)

Table 5
Independent-Samples T-Test Compared Responses by Gender

STATEMENT                        t (df)    p      mean
                                                 (men)
                                                 (women)

Rapport: approachability/        -0.40    .691    20.02
accessibility/personality/       (223)            20.50
empathy

Delivery: communication/         -1.10    .274    21.74
personal style/relaying          (223)            23.37
information

Knowledge and credibility:        0.01    .989    22.54
expertise/experience/            (223)            22.52
intelligence

Organization and preparation:    -0.65    .514    17.34
clarity/thoroughness/            (223)            17.90
instructional material

Fairness: performance             2.62    .009    18.37
evaluation/assignments           (223)            15.71

Table 6
Independent-Samples T-Test Compared Responses by Class Rank

STATEMENT                        t (df)   p      mean
                                                 (under)
                                                 (upper)

Rapport: approachability/        -0.86    .391   19.43
accessibility/personality/       (225)           20.49
empathy

Delivery: communication/         0.57     .569   22.87
personal style/relaying          (225)           22.00
information

Knowledge and credibility:       -0.56    .647   22.22
expertise/experience/            (225)           22.88
intelligence

Organization and preparation:    1.63     .104   18.50
clarity/thoroughness/            (225)           17.06
instructional material

Fairness: performance            -0.56    .579   16.98
evaluation/assignments           (225)           17.57