On the benefit of changing multiple-choice answers: Student perception and performance

The author wishes to thank Edmund Boyle and Joseph Matoney for providing data for the study, to Steven Elder and Robert Edgar for research assistance and to Mark Higgins for comments on earlier drafts.

This study extends earlier research on the benefits of changing multiplechoice test answers to upper-level college courses and also assesses whether these students accurately perceive the outcome of their own answer-changing behavior. The results indicate that when students change answers on multiple-choice questions, they tend to gain two to three more points than they lose over the course of the semester. This result held true for both males and females and for all of the five upperlevel accounting courses analyzed. However, even though 73 percent of the 279 students increased their total points for the semester, most held negative perceptions toward changing answers and predominately underestimated the benefit of their own answer-changing behavior.

Introduction

Throughout formal education one of the most widely used devices to evaluate progress and performance has been the multiple-choice test, or some variation of this objective format. Regardless of the tendency of educators to rely on this type of examination question, few educators or educational counselors include specific multiple-choice exam-taking strategies as part of their curricula. Nevertheless, students assimilate individual test-taking strategies and one tactic of that strategy is dealing with the temptation to change answers after an original answer selection has been made. Students are often confronted with the dilemma of whether or not to change their original selection in light of what appears to be the "new" correct answer. Additionally, prior studies have shown that even though many students change answers, the widespread "conventional wisdom" has been that the first multiple-choice answer selected is usually believed to be the best answer (Foote and Belinky, 1972; Geiger, 1991; Pressley and Ghatala, and 1988; Zakay and Glicksohn, 1992).

Although the findings regarding the benefit of answer-changing are generally clear, most of these studies have been performed on underclass college students, usually in psychology, education or educational psychology courses. This study extends research on answer-changing to upper-level college courses in accounting in order to more broadly assess the benefit of answer changing behavior. Upper level students, due to their advanced stage of mastery, may exhibit different behaviors, perceptions or outcomes when confronted with a multiplechoice examination format than their less educated peers. Also, students of business courses represent an under studied group that may potentially differ regarding answer-switching behavior. The study also analyzes students' individual perceptions toward changing their original answers and compares those perceptions to actual performance. Finally, the gender of the student and its possible effect on answerchanging behavior and perception is analyzed.

Prior Research

The first empirical study by Mathews (1929) examined college students in introductory educational psychology courses and found that over 53 percent of the answers changed on multiple-choice questions were from a wrong answer to the right answer (WR), approximately 22 percent were from the right answer to a wrong answer (RW) and the remaining changes were from wrong answers to other wrong answers (WW). The basic finding of the Mathews (1929) study, that for every point lost roughly two to three points are gained, have been upheld by later researchers (Bath,1967; Foote and Belinky,1972; Mueller and Wasser, 1977; Videler and Hansen, 1980; Geiger, 1991).

Additionally, researchers have examined several individual characteristics and potential causes of answer-switching behavior. Reile and Briggs (1952) found that females changed answers more often, but overall gained fewer points than male students. Bath (1967), however, found that females gained more points than males, and several studies have found no gender relation to switching behavior or net gain (Copeland, 1972; Geiger, 1991; Mueller and Shwedel,1975; Reiling and Taylor,1972). Finally, Videler and Hansen (1980) found that changes were more likely to be made on difficult rather than easy items, although Jacobs (1972) concluded students changed more answers on low to moderate difficulty items.

In a related area of research, Pressley and Ghatala (1988), Pressley, Ghatala, Woloshyn and Pirie (1990) and Zakay and Glicksohn (1992) examined student confidence on multiple-choice questions. They found that students often believed that there was a good chance that their initial answer was correct when, in fact, it was incorrect.

The sparse investigation of student perception regarding answer-switching has usually been based on general opinion (Mathews, 1929; Mueller and Schwedel, 1975) or has aggregated students of like opinion together for analysis (Jacobs, 1972). However, one study by Geiger (1991) did assess individual student perceptions and performance regarding answerswitching. These studies have generally found that students believe they lower their exam scores more often than they raise them when original answers are changed.

Methods

Sample

The sample used in this study consisted of 279 upper-level accounting students from two large northeastern universities. The students were enrolled in the upper-level courses of Intermediate Accounting I (INTERIa and INTERIb), Intermediate Accounting II (INTERII), the first senior-level auditing course (AUDITUG), or the first graduate auditing course for individuals without prior coursework in auditing (AUDITG). The courses used in this study were taught by three different instructors as part of their normal teaching loads.

Because each instructor had complete control over their course and prepared their own examinations, consisting of varying number and percentages of multiple-choice questions, data on each group of students will be presented separately. However, tests of group and overall gender differences will be presented and discussed where appropriate.

Items

Each instructor gave three examinations throughout the semester for each of the five courses. Examinations in all courses consisted of a multiple-choice section and a problems/essay section. The resultant total number of multiple-choice questions included in this study was 28, 259. Table 1 presents information on the subjects and multiple-choice questions given for the semester in each course.

Procedure

All examinations were hand-graded and retained by the instructors. Similar to prior research, the examinations were then scrutinized under high illumination for erasure marks. Since the actual examinations were used, evidence of answer-changing was readily apparent. In the few cases of uncertainty, however, the questionable items were not considered changes. All identified changes were classified as wrong-to-right (WR), right-towrong (RW), or wrong-to-wrong (WW).

On one of the examinations for each course, students were asked whether they believed they gained, lost or didn't alter their own overall test scores from changing answers to multiplechoice test questions. These perception questions were structured as multiple-choice items at the end of the respective test's multiple-choice section and all students freely responded.

Results

Overall Changes

The total number of answer-changes for the combined data set was 1,582 or 5.6 percent of all multiple-choice questions. Overall, the number of answers changed from wrong to right (WR) was 889, or 56.2 percent of the changed answers. The remaining changes were fairly evenly split between RW (n=330; 20.9%) and WW (n=363; 22.9%) changes. This combined result is supported by analysis of each individual course. Individual course percentages of WR changes ranged from 47.6 percent (AUDITG) to 59.6 percent (INTERII).

These consistent findings for all five upperlevel accounting courses in the study are congruent with the results of earlier researches using different levels of students and types of courses. These overall findings, amassed with prior research, validate the contention that upon additional thought, changing original answer selections generally appears to be in the best interest of the student.

Table 2 presents a summary of the mean and percentage answer-changing behavior for students in the five courses. In each course the mean WR changes are two to three times as large as the mean RW changes, and are also represented in the mean net gains (WR minus RW) for each course. These upper-level accounting students demonstrated a mean gain/loss ratio ranging from 2.19/1 (INTERIa) to 3.86/1 (INTERII) on their answer-changing performance. This overall net gain for each course is also consistent with the range net gains found in earlier research as summarized by Benjamin, Cavell and Shallenberger (1984) and Mueller and Wasser (1977). Additionally, the overall percentage of net gain in multiple-choice points across all students was 1.68, ranging from 3.13 percent (INTERIa) to .88 percent (AUDITG).

The percentage of total multiple-choice answers changed ranged from 8.36 percent (INTERIAa) to 4.12 percent (AUDITG). This level of change activity is slightly higher than the mean of 3.2 percent reported by Mueller and Wasser (1977) and the 4.98 reported by Benjamin, et al. (1984), but is still indicative of a relatively small portion of answer changes to total answers.

Individual Student Changes

While the aggregate analysis supports answer-changing behavior, it is also important to analyze the results from an individual student's perspective. Therefore, answer-changing data was also analyzed on an individual basis to identify how many students actually gained or lost points for the semester due to their own answer-changing behavior. This empirical result is then compared to the student's perception of their answer-changing behavior in order to assess the accuracy of the individual student's belief. Additionally, the effect of gender on answer-changing behavior, its perception, and the outcome of that behavior is addressed.

While Table 2 indicates that relatively few answers were changed for the semester, a large portion of students engaged in changing answers. Of the 279 students from the five courses, 265 (95.0%) changed at least one answer for the semester. The analysis of the individual courses is also similar to the overall result with the lowest number of students changing answers in the INTERIa course (78.9%) and the highest in the INTERII course (98.7%). This high level of individuals engaging in change behavior has also been consistently reported in earlier studies (Mueller and Shwedel 1975).

Separate t-test were performed for each of the five courses to determine if there was an effect due to gender on each category of change, the total number of changes and the net gain. Table 3 summarizes the significant results. In all cases of significant differences (p

Net gain, loss and no change in points for the semester was also calculated for each student. Table 4 presents the number of students, by course and by gender, that actually gained, lost, and had no final effect on their total exam points due to changing their original answer selections.

For the combined data, 72.8 percent of the students (71.2% of the males and 74.6% of the females) gained points over the course of the semester due to their individual behavior. In contrast only 7.9 percent (7.2% of the males and 8.7% of the females) had a net loss of points over the semester. An analysis of the five individual courses reveals the same pattern. A Chi-square test of dependence indicates that net change in points for the semester is not dependent on the gender of the student (p> .25) for the combined data or for any of the courses individuality. Additionally, separate t-tests for the magnitude of gains and losses, by gender, indicate that the actual size of the gain or loss is not gender dependent. These results are contrary to those of Bath (1967) and Reile and Braiggs (1952), who found gender differences, but are consistent with the majority of studies that provide evidence that the gender of the student plays no significant role in the final outcome of net point gain.

A Chi-square test was also run to see if the gain, loss or no change results summarized in Table 4 were dependent on type of accounting course. The results indicate a significant (p

Accordingly, a second test was run just on the students that gained or lost points for the semester in order to determine if there were any differences in actual changes in overall exam scores across courses. The results from the reduced data set indicate no dependence on the type of course (p >.25). These last results, combined with the earlier tests, indicate that students of all five upper-level courses benefited from their answer changing behavior and that the benefit was fairly consistent across courses and gender.

Student Perception

For the combined sample, only 34 students indicated that they believed they gained points, while 192 students indicated they believed they lost points, and 53 students believed that their overall answer-changing behavior had no net effect on their scores. Table 5 shows the answerchanging perceptions of the students by course and by gender.

Although the vast majority of students indicated their perception was that they lost points, a Chi-square test indicates that the perception of answer changing was not dependent on the gender of the student (p >. 25). A separate analysis of each of the five courses also indicated no differences in perception due to gender. However, perceptions did differ across courses (p

In order to determine if the individual students maintained appropriately negative perceptions about their own answer-switching behavior, their individual perceptions have been matched with actual net point gain/loss for each student. Table 6 depicts the results in total and by gender.

As can be readily seen from Table 6, only 56 students (20%) correctly perceived the outcome of their behavior, while 11 students (3.9%) overestimated, and 212 (76.0%) underestimated the benefit of their own answer-changing behavior. Chi-square tests of dependence for the combined sample, and for males and females taken separately indicate that there is no statistical dependence between the student's perceptions and their actual performance for the semester (p

Although INTERII students perceived their answer-changing behavior more favorably than their peers, students of all courses generally benefited and were equally poor in the accuracy of their perceptions. Additionally, the results for males and females separately also do not support such a low appraisal of actual performance. Students appear to have greatly distorted negative attitudes. The results of this study indicate that, in fact, these upper-level accounting students far outperformed their perceptions.

One final set of analyses was performed. The students were partitioned based on their perceptions of the outcome from answerswitching. Comparisons were then performed for the number and type of changes made, the actual net gain in points, and the total accumulation of test points for the semester. The separate analysis compared students that thought they gained points to all other students for each of the five courses. The only significant difference found was the accumulation of total test points for the semester (p

This finding lends some evidence that higher performing students (in terms of exams) were more apt to accurately perceive a benefit from changing answers. Interestingly though, these better students did not actually change more answers or gain more points than their peers. Students holding all a priori perceptions equally engaged in and benefited from changing original answer selections. However, the better students appear to more accurately recognize the advantage of rethinking and changing original answer selections.

Summary and Discussion

The findings of this study reinforce the benefit of judiciously changing original multiple-choice answers upon additional thought. Congruent with earlier research, this study found that on average for every point lost due to switching, roughly two to three points were gained. Ninety-five percent of the students changed at least one answer for the semester, and 72.8 percent of all students increased their scores over the semester due to their switching behavior. Additionally, no differences in overall changing behavior, mean net gain or magnitude of gains or losses were found to be due to the type of upper-level accounting course examined.

Contrary to prior research, this study did find a gender effect on behavior. Female students in four of the five classes exhibited higher change activity than male students. This increased activity, however, did not manifest itself in significantly increased net gains for the semester. The literature on answer-changing behavior has reported fairly mixed results for slight behavioral differences due to gender. Future research should continue to investigate this potential determinant in order to ascertain whether research results on gender, including those of this study, are sample driven or influenced by other factors.

It was found that most students have incorrectly negative perceptions of their actual answer-changing behavior, and that 76.0 percent underestimated the benefit of their own answer-changing behavior. It also was found that better students, in terms of exam performance, more accurately recognized the benefit of changing answers, although they did not change significantly more answers nor gain more points than their peers.

The results of this study also seem to support the findings of Pressley and Ghatala (1988), Pressley, et al. (1990), and Zakay and Glicksohn (1992) that students are, in general, overconfident about their initial responses to multiple-choice questions. This overconfidence, then, appears to transfer to a student's perception of benefit when changing their original multiple-choice answers.

It is also important to note that similar results have been obtained in very disparate contexts - introductory courses in education or psychology, and upper level courses in accounting. This cross context effect serves to strengthen the generalizability of the resultant conclusions of both the earlier research and the present study.

Millman, Bishop and Ebel (1965) suggest that the tendency to evaluate and judiciously change one's response is a basic aspect of testwiseness. Sampson (1985) further suggests that the tendency to scrutinize answer selections is not only beneficial, but can be enhanced by proper instruction and cognitive arousal on the part of the test-taker. Accordingly, this study, when amassed with the prior literature, suggests that students should most certainly be conscious of this pragmatic aspect of being testwise. As counselors and educators of students in disparate disciplines, the broad generalizability of these results is empowering. We should encourage students to scrutinize their original answer selections for alternative answers that subsequently appear more plausible, and not be stifled by incorrect perceptions about changing answers.

References

Bath, A. (1967). Answer-Changing Behavior on Objective Examinations. The Journal of Education Research, 61, 105-107.

Copeland, D.A. (1972). Should Chemistry Students Change Answers on Multiple-Choice Tests? Journal of Chemical Education, 49, 258.

Foote, R. and Belinky C. (1972). It Pays to Switch? Consequences of Changing Answers on Multiple-Choice Examinations. Psychological Reports, II, 667-673.

Geiger, M.A. (1991). Changing multiple-choice answers: Do students accurately perceive their performance?, Journal of Experimental Education, 591 119-127.

Jacobs, S.S. (1972). Answer Changing on Objective Tests: Some Implications For Test Validity, Educational and Psychological Measurement, 32, 1039-1044.

Mathews, C. Co. (1929). Erroneous First Impressions on Objective Tests, Journal of Educational Psychology, 20, 280-286.

Millman, J., Bishop, C. H. and Ebel, R. (1965). An Analysis of Testwiseness. Educational and Psychological Measurement, 25, 707-726.

Mueller, D.J. and Shwedel, A. (1975). Some Correlates of Net Gain Resultant From Answer Changing On Objective Achievement Test Items. Journal of Educational Measurement, 12, 251-254.

Mueller, J. and Wasser, V. (1977). Implications of Changing Answers on Objective Test Items. Journal of Educational Measurement, Vol. 14, 9-13.

Pressley, M., and Ghatala, E.S., (1988). Delusions and About Performances on Multiple-Choice Comprehension Test Items. Reading Research Quarterly, 23, 454-464.

Pressley, M., and Ghatala, E.S., Woloshyn, V. and Pirie, J. (1990). Sometimes Adults Miss the Main Ideas and Do Not Realize It: Confidence in Responses to ShortAnswer and Multiple-Choice Comprehension Questions." Reading Research Quarterly, 25, 232-249.

Reile, P.J. and Briggs, L.J. (1952). Should Students Change Their Initial Answers on Objective-Type Tests?: More Evidence Regarding An Old Problem. Journal of Educational Psychology, Vol. 43,110115.

Reiling, E. and Taylor, R. (1972). A New Approach to the Problem of Changing Initial Responses To Multiple Choice Questions. Journal of Educational Measurement, Vol. 9, 67-70.

Sampson, (1985). Effects of Training in Test-taking

Skills on Achievement Test Performance: A Quantitative Synthesis. The Journal of Educational Research, Vol. 78, 261-266.

Videler, D. and Hansen, R. (1980). Answer Changing on Multiple-Choice Tests. Journal of Experimental Education, Vol. 49, 18-20.

Zakay, D. and Glicksohn, J. (1992). Overconfidence in a multiple-choice test and its relationship to achievement. The Psychological Record, 42, 519-524.

Copyright Project Innovation Fall 1996
Provided by ProQuest Information and Learning Company. All rights Reserved