Analysis of active learning methods--clicker technology versus class discussion.

Findlay-Thompson, Sandi ; Mombourquette, Peter ; Thompson, Bruce 等

INTRODUCTION

Active learning approaches are greatly praised in higher education with researchers such as Guthrie and Carlin (2004) claiming that traditional lecture courses may be increasingly out of touch with how students engage their world. The use of technology in university classrooms to support teaching has advanced rapidly over the past two decades and includes devices such as SMART boards, electronic collaboration platforms, PowerPoint and other presentation software tools and online evaluation tools.

Educators are continually challenged to find new strategies for engaging students in the classroom in order to increase the effectiveness of the learning process. Although traditional classroom techniques often include class discussion which is considered an appropriate and active learning method, there is a need to more fully understand how the current generation of students view these active learning tools .

One relatively new technology is a personal response system or as it is otherwise known, clicker technology. Current clicker systems generally consist of student transmitters, an instructor receiver, and a computer. The student transmitters are small handheld units approximately the size of a cell phone and allow for students to send question responses via a keypad on the transmitter to the instructor receiver. Real-time processing allows instructors to view how the questions were answered. This instantaneous feedback regarding student comprehension of material presented is one of the greatest benefits of clicker technology (Nelson & Hauck, 2008; Stuart, Brown & Draper, 2004; Yourstone, Kraye, & Albaum, 2008).

Clickers are widely used across university curricula and as a result several studies (DeBourgh, 2008; DeGagne, 2011; Keough, 2012; Lantz, 2010; Martyn, 2007; Sprague & Dahl, 2010) have been undertaken to determine whether or not the use of clickers is effective in creating an active learning environment. The literature review that follows provides thoughtful insight on the effectiveness of clicker technology as well as student perceptions of clicker use in the learning environment.

The purpose of this study was twofold. The first was to analyze whether using clicker technology would increase learning outcomes more than the class discussion approach. The second was to review student perceptions from the fall 2013 classes related to clicker technology use based on responses from a post-semester survey that employed both Likert-type scale questions and open-ended questions.

LITERATURE REVIEW

The majority of early clicker research has been conducted within the natural sciences and the medical and health professions. The benefits and acceptance of the use of clickers in these fields has been well documented and is giving rise to an increased interest by other academic disciplines. One of these fields is management where there is little documentation on the use of clicker technology (Keough, 2012).

The gap in the literature in the field of management led Keough (2012) to conduct his own research in this discipline. He had noted that little documentation was available for the use of clicker technology in university level management classes. Before he adopted the use of clicker technology in the classroom, he felt it was important to understand student perceptions of clicker technology as student mindsets and thought processes differed from discipline to discipline. He undertook a review of 66 clicker technology-based studies focused on student perceptions/outcomes and then conducted his own study in the discipline of management and replicated the existing research studies using students in an upper-level management class.

The results of Keough's (2012) study supported his hypothesis that higher levels of academic performance on exams were exhibited by students using clickers in a management setting than students not using clickers. He also looked at other criterion such as student satisfaction, perceived increases in performance, levels of participation, perceived feedback, and perceived ease of use and his results indicated that each criterion mean was significantly greater than his midpoint of the Likert-type scale he had used for his survey.

In their study, Sprague and Dahl (2010) used an undergraduate marketing course to identify student performance outcomes using clicker technology. Study results validated the broad applicability of this technology but equally as important the study found evidence that clicker technology benefited students who are frequently disadvantaged in the classroom. Diversity in student populations and larger class sizes has created continuing challenges for professors trying to engage students in active classroom discussion. Sprague and Dahl suggested that students facing cultural barriers had a better learning experience when using clicker technology.

Celsi and Wolfinbarger (2002) have extensively reported on discontinuous innovation in teaching. Discontinuous innovation is a paradigm shift either in science or technology that in the teaching profession fundamentally changes the way in which students and instructors interact. According to Celsi and Wolfinbager (2002) clicker technology is an example of discontinuous innovation with the instructor being viewed by the students as a facilitator which in turn caused a behavioral shift in the student from passive to active and participatory learner. They concluded that discontinuous innovation created stronger relationships with students and enabled achievement of learning goals.

In the discipline of nursing, several studies have been written that examined the use of clickers in classroom environments. For example, Patterson, Kilpatrick and Woebkenberg (2010) found that there were no significant differences between groups on any measure of performance and that the clickers did not improve learning outcomes as measured by objective testing. However, they did find that student perceptions of clicker technology were positive citing the immediacy of feedback, the ability to respond anonymously, and the ability to interact in an engaging environment while learning all factors that indicated an increased degree of classroom engagement. Contrary to the findings by Patterson, Kilpatrick and Woebkenberg (2010) a study undertaken by Shaffer and Collura (2009) during an introductory psychology lecture on perceptual constancy found that students using clickers performed significantly better on exam questions concerning the lecture compared to other students who did not use clickers.

Jones, Henderson, and Sealover (2009) undertook a literature review of clickers in the classroom. The result of their research was to offer a pedagogical basis and best practices for clicker use in nursing education programs. One important factor that emerged from their research was that there are differences between teachers and what the teacher does with clickers in the class will ultimately determine whether educational outcomes are achieved. This concurs with the research results of several studies where it was determined that although the learning curve for clicker use did not seem steep for students, there was a considerable learning curve for instructors on how to properly use the system (Hatch, Murray and Moore, 2005; Lincoln, 2008; Sprague and Dahl, 2010).

The ability of the instructor to properly design the clicker system was reviewed by DeBourgh (2008). He concluded that the greatest challenge of clicker use in the classroom was "the challenge of designing effective presentation of complex content in a manner that limits intimidation, generates student interest, and provides opportunities for active student engagement in learning" (p. 76). Further, he suggested that "tactical use of focused questioning is a powerful method to provide feedback to students" (p. 78) and as noted above one of the greatest benefits of clicker technology is the instantaneous feedback regarding student comprehension of material presented (Nelson & Hauck, 2008; Stuart, Brown & Draper, 2004; Yourstone, Kraye, & Albaum, 2008).

As noted, the learning curve for instructors in the use of clicker technology can be very steep. Lantz (2010) found that while some educators had adopted the use of clicker technology as an active learning tool, others were uncertain that "clickers can affect learning outcomes, as though clickers are merely an amusing novelty" (p. 556). He uncovered several reasons for this uncertainty. First, the use of clicker technology involved significant extra preparation time for educators. Second, educators were often vague on how clickers worked and whether or not they would be able to learn how to use them effectively. Third, if educators did invest the extra time and preparation to be trained on how to use clickers, do they really aid in learning?

Lantz concluded that student perceptions of clickers were almost always positive and that significant differences had been found in final grades. However, he did suggest there is a deliberate need to understand that other variations can also be responsible for the grade results. Examples of variations include the time of day that a class is taken, whether the grade comparisons are between sections of the same course or between different courses, or whether the same educator teaches all of the classes being compared.

West (2005) stated that past studies on learning outcomes suggested that better learning outcomes result from changes in pedagogical focus--from passive to active learning--and not from use of a specific technology or technique. While the more recent studies on clicker technology seem to suggest that the clicker technology by its very nature moves the pedagogical focus into the active learning realm, his conclusions regarding a need for instructors to adopt active learning approaches supports one of the seven principles for good practice in undergraduate education posited by Chickering and Gamson (1987).

In fact, in almost all of the research reviewed for this literature review, the seven principles for good practice in undergraduate education posited by Chickering and Gamson in 1987 were cited. The conclusions were that these seven principles were still relevant in today's classroom and that the use of clicker technology was able to enhance and more than adequately meet these "test of time" principles (DeBourgh, 2008; Jones, Henderson & Sealover, 2009; Martyn, 2007). In a meta-analysis undertaken by DeGagne (2011), she reviewed the results of more than 1000 participants across 15 studies published between 2003 and 2009. Her conclusions were that the use of clicker technology in the classroom not only enhanced learner engagement and participation but also had "...characteristics that included the promotion of knowledge retention and critical thinking skills through the process of effective teaching and integration of technology into the classroom" (p. 39).

Martyn (2007) noted that most of the research on the benefits of using clickers in the classroom was compared only to traditional lecture methods. She questioned whether or not it was the clickers or the active learning pedagogies that were the cause of higher grades? Martyn wanted to determine if the use of clickers would increase learning outcomes more than another active learning approach namely class discussion. Her study found that there was no statistically significant difference between the use of clickers and the use of class discussion. The mean for the group using clickers was 85.80 and for the group using class discussion the mean was 87.19. However, both groups of students were asked to rate the active learning methodology and although no statistically significant differences occurred, the mean scores were consistently higher for students who had used clickers.

HYPOTHESIS

H: Students using clicker technology in an undergraduate management course would have higher numerical grade outcomes than their counterparts who are not using clicker technology.

RESEARCH METHODOLOGY

Sample and Data Collection

The primary data collection was done in two parts. The first part involved inviting all of the 60 students who had used clicker technology in two identical courses in Operations Management in the fall of 2013 to complete a survey at the end of the semester on their perceptions of clicker technology. Of the 60 students invited, 36 completed the survey. The second part of the data collection involved comparing the grades of the 60 students who had used clicker technology in the fall of 2013 with those of 76 students who did not use clicker technology but took the identical course in winter 2014. These two cluster groups, the first from the fall of 2013 and the second from the winter of 2014, represented all of the students registered in the four sections being analyzed.

Survey Questionnaire

The survey used in this study was prepared by Pearson Education, the publisher of the course textbook used by all students in the four sections of the Operations Management course being analyzed. Pearson Education was also the provider of the clickers used by the 60 students in the two fall 2013 sections. The survey employed both Likert-type scale questions and openended questions. The purpose of the survey was to review student perceptions related to clicker technology. Pearson Education provided the survey results to the researchers of this study who then used SPSS and a p-value of .05 to conduct the statistical test in this study.

Procedure

Numerical grades were collected from four sections of an undergraduate 3000 level course in operations management. In the fall of 2013, the first two sections used clicker technology as the means of active learning in a course titled Operations Management. Both sections were taught by the same professor. In the winter of 2014, the other two sections of Operations Management used class discussion as the means of active learning rather than the use of clicker technology. All four sections--the two in the fall and the two in the winter followed the identical course outlines, assignments, and exams. The data allowed for a comparison of grade outcomes of students who used clicker technology as the active learning tool with those that did not use clicker technology and instead used class discussion as the active learning tool. Additionally, at the end of the fall 2013 semester, a survey prepared by Pearson Education, the publisher of the course textbook and clicker provider, was administered to the students in the first two sections that had used clicker technology to review student perceptions related to clicker technology. The survey employed both Likert-type scale questions and open-ended questions. Pearson Education provided the survey results to the researchers of this study who then used SPSS and a p-value of .05 to conduct the statistical test in this study.

RESULTS AND DISCUSSION

Students using clicker technology were asked to complete a survey post-term December 2013. Of the 60 students in the two fall 2013 classes, 36 completed the survey. The first part of the survey used a Likert-type scale to determine student perceptions regarding other criterion in the use of clicker technology. Six of the questions and applicable responses are presented in Appendix 1.

A t-test was conducted to analyze the comparison of grade outcomes between 60 students using clicker technology and 76 students not using clicker technology. The findings were that no statistical significant differences existed between the grade outcomes of students using clicker technology and students not using clicker technology (df = 134; p = 0.069).

As suggested in much of the research available, there are many confounding variables that typically can be controlled by ANOVA but in this study the sample size is somewhat limited and the grade outcomes were non-significant and it was reasonable to assume that nothing would be further gained by trying to control for various factors. However, to ensure the assumptions were correct, t-tests and or f-ratios were conducted on three other variables, namely 1) the time of day that students took the course which was either at 9:00 am or at 12 noon, 2) whether the course was held in the fall of 2013 or in the winter of 2014; and 3) whether the course was taught on-campus or via distance learning. The results are shown in Table 1. All of the t-tests and f-ratio results were that there were no statistically significant differences between the grade outcomes and the variable being examined.

The responses from the survey regarding other criterion all concur with the existing research studies. It did not matter whether the quantitative results showed that grade outcomes were higher or had no statistically significant difference between users and non-users of clicker technology. The majority of students' perceptions were that other factors such as student engagement, satisfaction, immediacy of feedback and classroom atmosphere were all significantly greater than the midpoint of the Likert-type scale questions used for this survey.

Because two of the test samples in this study used clicker technology and two used class discussion, these findings agree with those of Martyn (2007) who found that there was no statistically significant difference between the use of clickers and the use of class discussion. This is an interesting finding as it supports West (2005) who posited that better learning outcomes result from changes in pedagogical focus--from passive to active learning--and not the use of a specific technology or technique.

There is considerable debate on whether or not objective quantitative measures of performance are enhanced by the use of clicker technology. However, in the study by Martyn (2007), both groups of students were asked to rate the active learning methodology and although no statistically significant differences occurred, the mean scores were consistently higher for students who had used clickers. This finding appears to be the commonality among all of the articles reviewed for this literature review. In each instance, student perceptions of the use of clicker technology were positive.

In this study, 68.8 percent of students that had used clicker technology reported that their preferred methodology for learning was performing an activity. As clicker technology specifically requires students to perform an activity, it is not surprising then that in every research study reviewed students' perceptions of the use of clicker technology were positive and they felt that the use of clicker technology enhanced the learning environment.

LIMITATIONS OF THIS STUDY

This research involved only a small sample size (n=136) over a two semester time period. In addition to the confounding variables tested above, other factors such as class size, diversity in student population, professor experience using clicker technology, GPA, and cultural barriers to name a few could be criterion that impact grade outcomes.

CONCLUSIONS

It is not possible to state that one active learning methodology is better than another. However, there is evidence to support the argument that students prefer active learning methodologies. Future research on the use of clicker technology in the classroom could be further explored by controlling for specific variables such as teacher experience in the use of clicker technology and diversity in student population and GPA to name a few.

Appendix 1

Participants' Responses to the Items in the Survey Questionnaire

Question: How would you describe the
usefulness of the immediate feedback
charts? (n=36/36)

Answer Options:                            Response Percent
Very Helpful                               52.8%
Helpful                                    38.9%
No comment                                 8.3%
Not very helpful                           0.0%
Not at all helpful                         0.0%

Question: Did you feel you were more
engaged in the learning process because
of the use of clicker technology?
(n=36/36)

Answer Options:                            Response Percent
Yes                                        91.7%
No                                         2.8%
Not Sure                                   5.6%

Question: How would you describe your
technology experience with using
clicker technology? (n=36/36)

Answer Options:                            Response Percent
Very Easy                                  55.6%
Easy                                       30.6%
Acceptable                                 11.1%
Difficult                                  2.8%
Very Difficult                             0.0%

Question: Rank the following activities
in order of which you believe you learn
the most from: (n=32/34)

Answer Options:                            Response Percent
Reading Text, Graphs                       9.4%
Listening to Someone Talk                  21.9%
Performing an Activity                     68.8%

Question: Did you find the clicker
technology allowed for a more social
and relaxed classroom atmosphere or a
more competitive one? (n=35/36)

Answer Options:                            Response Percent
More Social and Relaxed                    85.7%
More competitive                           14.3%

Question: Do you think the clicker
technology is an effective way to award
class participation points? (n=34/36)

Answer Options                             Response Percent
Yes                                        97.1%
No                                         2.9%

REFERENCES

Celsi, R., & Wolfinbarger, M. (2002). Discontinuous classroom innovation: Waves of change for marketing education. Journal of Marketing Education, 24, 64-72.

Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. University of Florida, Center for Instructional Technology and Training, retrieved, August 8, 2014 from http://citt.ufl.edu/tools/ chickering-and-gamson7-rules-for-undergraduate-education/.

DeBourgh, G.A. (2008). Use of classroom "clickers" to promote acquisition of advanced reasoning skills. Nurse Education in Practice (8), 76-87.

DeGagne, J.C. (2011). The impact of clickers in nursing education: A review of literature. Nurse Education Today, 31, 34-40.

Guthrie, R. W., & Carlin, A. (2004) "Waking the Dead: Using Interactive Technology to Engage Passive Listeners in the Classroom," Proceedings of the Tenth Americas Conference on Information systems, New York, August 2004.

Hatch, J., Murray, J., & Moore, R. (2005). Manna from heaven or "clickers" from hell: Experiences with an electronic response system. Journal of College Science Teaching, 34(7), 36-39.

Jones, S., Henderson, D., & Sealover, P. (2009). "Clickers" in the classroom. Teaching and Learning in Nursing, 4, 2-5.

Keough, S. M. (2012). Clickers in the classroom: A review and a replication. Journal of Management Education, 36(6), 822-847.

Lantz, M.E. (2010). The use of 'clickers' in the classroom: Teaching innovation or merely an amusing novelty? Computers in Human Behaviour, 26, 556-561

Lincoln, D. (2008). Teaching with clickers in the large-size principles of marketing class. Marketing Education Review, 19(3), 25-40.

Martyn, M. (2007). Clickers in the classroom: An active learning approach. Retrieved on August 11, 2014 at http://educause.edu/ero/article/clickers-classroom-active-learning-approach.

Nelson, M., & Hauck, R. (2008). Clicking to learn: A case study of embedding ratio-frequency based clickers in an introductory management information systems course. Journal of Information Systems Education, 19(1), 55-64.

Patterson, B., Kilpatrick, J., & Woebkenberg, E. (2010). Evidence for teaching practice: The impact of clickers in a large classroom environment. Nurse Education Today. 30, 603607.

Sprague, E.W., & Dahl, D.W. (2010). An evaluation of the personal response system clicker technology in introductory marketing courses. Journal of Marketing Education, 32(1), 93-103.

Stuart, S., Brown, M., & Draper, S. (2004). Using an electronic voting system in logic lectures: One practitioner's application. Journal of Computer Assisted Learning, 20, 95-102.

West, J, "Learning Outcomes Related to the Use of Personal Response Systems in Large Science Courses," Academic Commons, December 9, 2005.

Yourstone, S., Kraye, H., & Albaum, G. (2008). Classroom questioning with immediate electronic response: Do clickers improve learning? Decision Sciences Journal of Innovative Education, 6(1), 75-88.

Sandi Findlay-Thompson

Peter Mombourquette

Bruce Thompson

Karen Blotnicky

Mount Saint Vincent University

Sandi Findlay-Thompson is an Associate Professor at Mount Saint Vincent University in the Department of Business and Tourism. She joined academia after spending more than 20 years in management at two of Canada's leading banking institutions. Her research interests include women and entrepreneurship, management competencies, active learning methodologies for students, and learning in the workplace. Her research appears in journals such as International Journal of Gender and Entrepreneurship and Journal of Human Resources in Hospitality and Tourism.

Peter Mombourquette is an Associate Professor and Chair of the Department of Business and Tourism at Mount Saint Vincent University. His research interests include entrepreneurship, small business, teaching and learning, student success and engagement, and information technology. Dr. Mombourquette is currently finalizing his 6th book for publication and over the course of the last 13 years has published and given presentations on various topics related to his research areas of interest.

Bruce Thompson is a part-time instructor at Mount Saint Vincent University and employed full-time in the ICT industry. He has more than 20 years of senior management experience and is developing his interest in academia in the area of management competencies in the information, communications, and technology sector as well as the fields of entrepreneurship and student success and engagement.

Karen Blotnicky is an Associate Professor at Mount Saint Vincent University. Her research interests include information technology adoption among small and medium-sized enterprises, the impact of gambling-related advertising on gambling habits of adolescents, marketing orientation, managerial competencies, gender role portrayal in Canadian tourism advertising and entrepreneurial education. Dr. Blotnicky is extensively published, has received numerous grants, authored textbooks and operated an independent marketing consulting firm.

Table 1

Test Results For Clicker Technology Use And Potentia
Confounding Variables

Predictors             Statistic   df      Significance

Clicker Use v.         t=-1.836    134     .069
nonclicker use

Time of Course-        F=2.473     2/133   .088
morning v. afternoon

Term of Course-        t = 1.782   134     .077
fall v. winter

On Campus v.           t= 1.818    134     .071
Distance

Predictors                        Average Course (Std Dev)

Clicker Use v.          Clicker 76.3 (13.4)    Non-clicker 80.2 (10.6)
nonclicker use

Time of Course-         Morning 77.4 (9.2)      Afternoon 81.7 (11.2)
morning v. afternoon

Term of Course-          Fall 80.2 (10.7)        Winter 76.4 (13.4)
fall v. winter

On Campus v.           On Campus 79.0 (11.0)    Distance 74.4 (16.0)
Distance