Student quantitative literacy: importance, measurement, and correlation with economic literacy.

Schuhmann, Peter W. ; McGoldrick, KimMarie ; Burrus, Robert T. 等

I. Introduction

Economic concepts are often taught using quantitative methods, including the use of graphical and numerical examples and applications. For students lacking a basic level of quantitative literacy, interpretation of economic concepts can be lost in the translation. While many studies of economic literacy have appeared in the economic education literature, quantitative literacy has yet to receive the attention it deserves. This lack of attention to students' quantitative literacy might contribute to lower degrees of economic literacy.

The purpose of this study is to examine the relationship between quantitative literacy (as measured by math aptitude), economic literacy (as measured by knowledge of economic content) and economic learning (indicated by improvements in economic knowledge over the semester). Using a sample of students taking undergraduate principles of economics courses we attempt to quantify the degree of quantitative and economic literacy and test for learning (or improvement in economic understanding). Our sample of students is typical of what you might expect at the principles level in that they are very likely to be sophomores and very unlikely to have declared an economics major. The principles classes in which they were enrolled are also typical of those at most institutions in that they are the introductory courses in the major and do not have any prerequisite. Thus, our results are applicable to the large audience of instructors of the principles of economics courses.

We address two major research questions using pre and post course survey data. The use of a survey allows us to test a student's quantitative skills (through the use of multiple choice questions) rather than rely on typically used proxies such as courses taken and grades achieved. We first examine the degree to which students who have greater math aptitude also have a better grasp of basic economics concepts prior to taking a principles of economics course. Second, we explore the degree to which this math aptitude is correlated with higher economic learning. More specifically, we measure quantitative aptitude changes over the semester to determine the extent that students learn these skills in their economics class and how this learning correlates with economic learning. An extension of the latter question determines specific areas where students are deficient in their quantitative literacy and examines how these deficiencies might impact economic knowledge. Our results suggest that students who are deficient in skills such as being able to solve a system of equations and compute a percentage, and being able to interpret increases and decreases on a graph will have less economic knowledge at the end of the semester than their more quantitatively literate peers. This work implies that instructors in the principles courses should dedicate resources to assessing the degree to which their students possess quantitative skills and nurture the development of such skills during the semester.

II. Review of Literature

Economic literacy has been investigated through a number of venues. Tests of economic literacy such as the Test of Economic Literacy (TEL) and Tests of Understanding in College Economics (TUCE) have been developed in an effort to assess literacy levels for various populations such as high school students (Soper and Walstad, 1988; Walstad and Soper, 1988 and 1991) and segments of the adult population (Gleason and Van Scvoc, 1995; Wood and Doyle, 2002). Similar tests have been used to make literacy comparisons across countries (Kim 1994; Beck and Krumm 1994; Whitehead and Halil, 1991). The general push for greater economic literacy has been regularly analyzed (Stigler, 1983; Blank, 2002; Krueger, 2002; Lucas, 2002), and programs for promoting greater literacy among high school teachers have been developed (Parkison and Sorgman, 1998). Finally, the definition of economic literacy has even garnered debate (Nelson and Sheffrin, 1991; Ferber, 1999).

Quantitative literacy, on the other hand, has yet to receive similar attention in the literature, especially with regard to its connection with economic education. The importance of such skills for the general population may be demonstrated in their link to labor market outcomes. For example, literacy (including numerical) has a direct and positive effect on labor market earnings (Green and Riddell, 2001) and "quantitative literacy skills [are] a major factor raising the likelihood of full-time employment." (Rivera-Batiz, 1992: 313) For academic economists, on the other hand, the importance of quantitative literacy is best demonstrated by the work of Bennet and Johnson (1979) who find an increasing degree of quantification (equations, tables, graphs) in economics journals (up to 1977). An extension of this trend to current journal content is not unreasonable. Finally, there have also been debates regarding the level of mathematical sophistication existing today in economics as a profession and the (negative) impact this has on the research conducted in the profession. (Blinder, 1999: 4-5)

If quantitative literacy is indeed important for both the general population and future economists, what implications might this have for the classroom? Siegfried et al. (1991) suggest that "mathematical aptitude and skills are useful to an undergraduate economics major ... mathematics can clarify relationships and improve student understanding." (p. 203) As well, mathematical aptitude is important for reasoning ability and knowledgeable citizenship.

The study of quantification and the undergraduate economic classroom spans a wide range of topics including characteristics that influence performance in economic statistics (Cohn, 1972), math aptitude and enrollment in economics courses (Cohn et al, 2000), math aptitude and student apprehension about principles courses (Benedict, 2002), and the impact of quantitative skills on performance at various levels of economics courses. This impact on performance is investigated through an analysis of graph competency, required and elective mathematical background for intermediate economics classes and math aptitudes and course history prior to taking principles.

At nearly every level of economic instruction, graphical analyses are key tools used to portray economic concepts and theories. The degree to which instructors assume that students are equipped to utilize these tools is critical for economic learning. Principles students typically score poorly with respect to their ability to reproduce graphs, but those students who can graph more accurately perform better on post course tests of economic learning (Cohn and Cohn, 1994). Evidence as to whether graphical presentations improve economic comprehension is mixed. For example, principles students randomly assigned to graph intensive lectures (as opposed to those with no graphs) had either no increase or a decrease in performance as measured by the difference between pre and post test scores on economic content covered in the lecture materials (Cohn et al, 2001). On the other hand, Kourilsky and Wittrock (1987) find that a teaching strategy which leads students from verbal to graphical explanations of course content (as opposed to the reverse or verbal only) results in the greatest gains in economic knowledge. A logical extension of the impact of graph competencies is the identification of characteristics that might predict graph competency. For example, students with higher GPAs and a greater preference towards math have been shown to be more likely to answer graphical questions correctly (Hill and Stegner, 2003).

The ongoing academic debate regarding whether to require calculus for intermediate economics classes is one area in which performance in economics classes is directly linked to mathematical aptitude. Brasfield et al (1992) and Durden and Ellis (1995) both find that students who have had a first course in calculus perform better in college economics courses. As well, Butler et al (1994, 1998) find "that a second semester of calculus markedly improves performance of students in intermediate micro, but not in intermediate macro." (1994 p, 209) Von Allmen (1996) expands on this analysis by estimating the relationship between performance in a required calculus class and performance in the intermediate micro class. Consistent with the results of Butler et al, "a one-grade difference in calculus more than doubles the chances of getting an A in microeconomics, and roughly halves the chances of doing poorly." (Von Allmen, p. 21)

The impact of math aptitude on performance in economics principles courses is significantly more ambiguous. Gery (1970) found that the number of math courses taken positively affects pre-course performance as measured by the TUCE. In a further study of this phenomenon, Gery (1972) finds no significant impact of the level of math taken but rather that critical thinking skills and verbal aptitude (SAT) had the greatest positive impact on pre-course TUCE. Alternatively, quantitative aptitude (SAT) has the greatest impact on relative improvements in TUCE scores over the semester. (p. 152) These studies are reinforced by Durden and Ellis (1995). They find that math and verbal SAT scores "are among the most important determinants of student performance in college economics courses." (p. 345) Likewise, Brasfield et al (1992) find that students who completed a two course Business Math sequence "before taking either introductory economics class did significantly better in university level economics [as measured by the final grade in the course] than the students who had not completed the math sequence." (p. 243) Similarly, Anderson et al (1994) find that high school math is the most important indicator of success in a principles course. In contrast, Milkman et al (1995) find that although completion of a two course Business Math sequence has little or no impact on economic learning (as measured by the TUCE), "students who complete only the pre-calculus class prior to enrolling in microeconomics learn more economics than students who have completed the entire business math sequence." (p. 227)

In one of the few tests that include a measurement of math competency as opposed to proxies such as courses completed, grades in courses, or aptitude scores, Espey (1997) finds that the successful completion of a math competency exam (no matter how many times it was taken or how much tutoring was received) improves performance in principles. More recently, Ballard and Johnson (2004) employ a simple multiple choice test to measure the degree to which students understand basic algebraic calculations such as solving a system of equations, finding the area of a triangle, and determining the slope of a line. Competency in these basic skills is found to be an important determinant of course grade, in addition to traditional measures of math competency such as math scores on the ACT and previous math courses taken. Similarly, only one paper has actually investigated the degree to which mathematical skills are obtained in the principles class. Cohn et al (2000) find that students learn more math during an introductory course in economics than in an introductory course in psychology.

The results of this literature review reveal that many of the questions relating quantitative aptitude and economic learning are yet to be resolved. There is no clear consensus as to the impact of quantitative aptitude on performance in the principles courses, although evidence is mounting that suggests a positive impact. One result that does appear to be consistent across studies is that the performance in quantitative classes is more likely to have a significant impact than a simple indicator of whether the course has been taken.

This paper contributes to this body of work in a number of ways. First, we seek to test whether students who enter a principles course with a better quantitative aptitude perform better (ceteris paribus). This differs from much of the previous research in that we actually test a student's quantitative skills rather than relying on proxies such as courses taken and grades achieved. In addition, this work goes beyond that of Ballard and Johnson (2004) by including interpretive questions in addition to simple calculations. Second, the degree to which these measured quantitative skills contribute to economic learning is investigated by employing a pre and post course test that measures economic knowledge. Finally, we measure quantitative aptitude changes over the semester to determine the extent that students learn these skills in their economics class and how this learning correlates with economic learning.

III. Data and Survey Design

During the spring semester of 2002 a survey was designed to investigate the relationship between quantitative and economic literacy. The survey was pre-tested in a principles of economics class, reviewed by faculty familiar with survey use, and revised accordingly. (1) The final version of the survey solicited information on student demographics such as gender, age, race, class standing, and weekly hours spent working or volunteering, and asked a series of questions designed to evaluate the students' quantitative ability as well as knowledge of economics. Students were asked their current GPA, math SAT score, math and economics course background (in both high school and college), and weekly hours spent studying. In order to gauge specific quantitative skills and knowledge of economic content, the survey also asked a series of eight multiple choice math questions and ten multiple choice economics questions (five each of microeconomic and macroeconomic concepts). All variable definitions are shown in Table 1.

The survey was administered in the fall of 2002 to Principles of Economics students at the University of Richmond (UR), Mary Washington College (MWC), the University of North Carolina at Wilmington (UNCW), and the University of Nebraska at Lincoln (UN). (2) The survey was administered on the first and final days of class; 633 students completed pre-semester surveys (UR = 88, MWC = 234, UNCW = 160, UN = 151) and 534 completed post-semester surveys (UR = 87, MWC = 200, UNCW = 146, UN = 101). Students were asked to provide a code by which pre and post surveys could be matched, allowing for an investigation of the learning (both quantitative and economic) that occurs over the semester; pre and post surveys were successfully matched in 344 cases (UR = 46, MWC = 101, UNCW = 117, UN = 80). Remaining cases were not matched because students failed to report identifying codes necessary for matching or because the instructor failed to properly administer alternative versions of the survey. (3)

Table 1 also provides a summary of sample characteristics. Approximately fifty percent of the students in the sample are sophomores, as would be expected given that the survey was administered in principles of economics courses. The sample is equally comprised of males and females. Fifty-five percent of the students are business majors and just over three percent are economics majors (the remaining students identified themselves as 'other' majors). (4) Ninety-one percent of the sample is white, and only four percent is foreign. For the students who reported their G.P.A., the average G.P.A. was approximately 3.0. (5) Nearly all students reported taking high school geometry (ninety percent) and algebra 1 (ninety-two percent). A large majority of students also completed high school courses in algebra II (eighty-three percent) and pre-calculus (seventy-one percent), although significantly fewer had completed a high school calculus course (thirty-seven percent). Far fewer students in our sample had taken these math courses at the college level (except calculus). Forty percent of students had completed a college level course in calculus, yet only one quarter had taken algebra (twenty-five percent) or basic probability and statistics (twenty-four percent).

With regard to economics courses, less than forty percent had taken a high school economics course and over half (fifty-nine percent) were enrolled in their first college level economics class. (6) For those who had taken economics at the college level, approximately thirty-five percent had completed macroeconomic principles, sixty-four percent had completed microeconomic principles, and four percent had taken a survey course in economics. (7) When measured as a percentage of the full sample, approximately twenty-five percent of the students surveyed had taken microeconomics prior to enrolling in the current course (macro), fourteen percent had previously taken macroeconomics prior to enrolling in the current course (micro), and two percent had taken a survey economics course. (8)

IV. Mathematics Questions and Answers

Some of the survey questions designed to measure quantitative skills were selected from a college (UNCW) math placement exam and some were generated by the authors. (9) Questions designed to measure economic knowledge were selected from the Test of Understanding in College Economics (TUCE, 2000). All questions used in the survey are included in Appendix 1. The quantitative questions were selected to test student abilities on two levels: computational questions such as performing simple mathematical computations and reading graphs and interpretive questions such as the ability to interpret mathematical and graphical information. More specifically, the computation questions included solving a system of equations (question 1), computing simple percentages (question 2), and determining the slope and intercept of a straight line (questions 6 and 8). (10) Interpretive questions included calculating a percentage change in a given scenario (question 5), reading increases and decreases on a graph (question 3), interpreting a percentage change (question 4), and distinguishing the differences in two graphs (question 7). The percent of correct responses to each math question for both the pre-course and post-course surveys are reported in Table 2. (Math survey questions are included in the Appendix. The complete survey is available upon request.)

Pre-course survey results suggest that students had good mathematical computational abilities, but were weaker in interpretive capabilities. Most students were able to calculate the solution to a simple system of equations (question 1, 81 percent) and a simple percentage (question 2, 94 percent). In addition, a majority of students were able to look at a graph and determine the slope of a straight line (question 8, 57 percent); however, only 36 percent were able to determine the equation (slope and intercept) for a graphed straight line (question 6). Question 3 indicates that many students can make simple inferences from graphical presentations (64 percent). Only 28 percent of students choose the entire correct answer for question 7 ('both b and c'), many of the students (49 percent) choose a "correct" answer of either 'b' or 'c'. Thus, although students do have some understanding of what shifting a line on a graph means, that understanding is incomplete. Finally, students were unable to interpret the meaning of a percentage change; only 24 and 40 percent answered questions 4 and 5 correctly, respectively.

Incorrect answers can give insight about student quantitative deficiencies. Over 40 percent of students answered 'a' for question 5 on the pre survey; they believed that an increase in production from 60 to 100 implied a percentage change of 40%. Since students could not calculate a percentage change, it was not surprising that students did not understand that a positive but decreasing growth rate of GDP means that GDP is increasing (as only 46 percent of respondents correctly answered question 4). Students also made systematic mistakes when they answered question 3; almost 30 percent of respondents answered 'a'. This answer indicates that students could not discern that steady domestic oil consumption with increasing imports of oil implies that domestic production of oil has decreased. Either students did not read questions carefully, or they had a difficult time interpreting graphs to form logical connections.

Post-course tests yield similar results. Even though the proportion of students giving a correct response compared to the pre-course survey increased for 7 of the 8 questions, the only significant differences occurred for questions 2 and 6. In addition, the proportion of students correctly calculating a simple percentage (question 2) actually decreased significantly while the proportion of students who were able to determine the equation of a straight line (question 6) increased significantly.

Overall, students correctly answered an average of 4.24 out of 8 math questions on the pre-course survey and 4.46 on the post-course survey. This increase is significant at the 12% level (see Table 3). These results suggest that principles of economics courses may generate significant improvements in quantitative skills.

V. Economics Questions and Answers

A student's knowledge of economics is measured based on a subset of the questions developed for the TUCE. These ten survey questions included five microeconomics content and five macroeconomic content questions. (11) The percentage of students answering the economic knowledge questions correctly is provided in Table 2. It is to these percentages that we now turn.

Results from the survey administered on the first day of class indicate that students did have some prior economic knowledge. The percentage of students correctly answering the microeconomics questions was, on average, 32 percent. This percentage is statistically different from 0.25 (at the 1 percent level) which suggests that students did not simply randomly guess the answers to the microeconomic questions. (12) The percentage of students correctly answering the macroeconomics questions was, on average, 26 percent. This percentage was not statistically different from 0.25 which suggests that students performed no better than they would by randomly guessing on the macro questions. These pre-course results suggest that students enter principles courses with some economic knowledge of micro but not macro economic content.

Post-course survey results provide evidence as to the degree to which students learned economic content through taking an economics course. The average percentage of students answering the questions correctly was 43 for the microeconomics questions, and 31 percent for the macroeconomics questions. These percentages are statistically different from 0.25 (at the 1 percent level). Hence we observe an increase in the percentage of students correctly answering questions. Only in the cases of the microeconomics questions, however, were there statistically significant improvements in the percent of students answering the questions correctly.

Yet another way to measure the student's economic knowledge is to consider these economic knowledge questions as a whole irrespective of the content (micro or macro) of each question. This measurement reveals that the average percent of students answering each question correctly on the pre-course survey was approximately 30 percent. On the post-course survey, 37 percent of students, on average, answered the economics questions correctly. These percentages are statistically different from 0.25 (at the 1 percent level). The increase in percentage of students correctly answering questions across the pre and post surveys is significant at the 5 percent level. This result confirms previous research indicating improvement in economic knowledge as a result of completing a course in economics.

Overall, students answered 2.9 economics questions correctly on the pre-course survey compared to 3.7 correct answers on the post-course survey (see Table 3). Furthermore, the number of microeconomic questions answered correctly increased from 1.6 to 2.1 and the number of macroeconomic questions answered correctly increased from 1.3 to 1.6. Each of these differences between the pre-course and post-course surveys is significant, suggesting economic knowledge improvements as a result of taking an economics course.

VI. Modeling and Hypotheses

Quantitative skills and economic knowledge outcomes are measured as the number of correct answers to math and economics questions prior to and after taking a principles of economics course. In order to examine the relationship between student characteristics and economic knowledge acquisition, scores on the economics portion of the survey serve as the dependent variable. Note that the pre and post survey results can be examined individually or together by calculating the difference in correct answers between the pre- and post-course surveys. In the former case, the variable we wish to explain is constrained to be zero or a positive integer, hence a count data model will be appropriate for estimation. In the latter, the variable of interest (change in score) can be positive or negative; hence more traditional regression methods will suffice for estimation.

Poisson regression models provide a standard framework for the analysis of count data when a majority of the data falls in the lower end of the distribution (ie., 0,1,2, ...). The Poisson distribution determines the probability of a count.

(1) P([y.sub.i]) = Prob[y.sub.i] = j] = exp(-[[lambda].sub.i]) - [[lambda].sup.j.sub.i]/j!, j = 0, 1, 2, ...

Where the standard formulation for [[lambda].sub.i] is:

(2) [[lambda].sub.i] = exp ([beta]' [x.sub.i])

In order to examine the relationship between student characteristics and pre- and post-course scores on the economics questions in our survey, we estimate the following equations using a Poisson specification for both the pre-course survey results and the post-course results (variable definitions are provided in Table 1):

Model 1: [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]

and

Model 2: [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII.]

In both models, the dependent variable is the total number of economics questions answered correctly. In order to fully analyze the impact of quantitative skills on economic knowledge we include two measures of quantitative aptitude: TMATHPRE or TMATHPOST (total number of math questions answered correctly on either the pre-course or post-course survey) in model 1, and dummy variables (Q1-QS) for each individual question indicating whether the student answered that specific quantitative question correctly in model 2. (13) The relationship between these same variables and improvement in economic learning is modeled using economic learning (as measured by the change in the number of economic questions answered correctly, post-course survey minus pre-course survey) as a function of quantitative skill acquisition (the difference between post- and pre-course scores on the quantitative questions, TMATHPOST--TMATHPRE). The signs and significance of the coefficients in these models will provide insight into which variables have a significant influence on economic aptitude for principles of economics students. (14)

Surprisingly, other variables such as those representing successful completion of previous math courses were found to be insignificant in our study. Models that included the number of math classes taken in high school (excluding high school calculus) and college and whether or not students had taken high school calculus were also estimated. Consistent with previous research, we anticipated positive coefficient estimates for these variables; however, these variables were not significant when TMATHPRE and TMATHPOST were included in the pre and post course models, respectively. In the pre course model, the coefficient estimates were not significant even without the TMATHPRE variable. In the post course model, only the high school calculus variable was significant without the inclusion of the TMATHPOST variable. In addition, these variables did not impact economic learning. (15)

Consistent with previous literature, we anticipate negative signs on the coefficient estimates for BUSINESS and OTHER; students who are business majors and non-business majors will not exhibit as much economics knowledge or economic learning compared to economics majors (base case). As well, students who have previously taken an economics course in high school (HSECON) or in college will have a greater base of economic knowledge and may exhibit a higher degree of learning over the semester, as they have already acquired the skill set upon which more advanced topics can be learned. However, we note that students with no prior economics courses may show greater total learning of economics content, as they are starting with relatively low pre-course knowledge, so all knowledge gained in the first course constitutes learning. Thus, we make no a priori predictions for the signs of coefficient estimates on MAC HAD MIC, MIC HAD MAC, MIC HAD SURVEY and MAC HAD MIC AND SURVEY compared to students taking their first economics course in microeconomics (base case).

Because of the analytical similarities between math skills and economic skills, we also suspect that students who did better on the math questions (TMATHPRE or TMATHPOST in model 1, or Q1-Q8 in model 2) will perform better on the economics questions and also learn more. (16) While we have no a priori expectations as to the signs on the dummy variables for the three universities (UR is the base case), we do note that one of the four schools in our sample is a small private institutions (UR) one is a small public institution (MWC), while the other two are larger public universities. Given the insignificant differences between the survey versions (described above), we do not anticipate any impact of survey design (RSURVEY) on the results. (17)

VII. Results

Economic knowledge models 1 and 2 (incorporating total and individual measures of quantitative aptitude, respectively) were estimated using a Poisson specification for both pre-course and post-course survey results (Tables 4 and 5). (18) Model 1 (incorporating the total number of math questions answered correctly) was also estimated via OLS using economic learning (the difference between pre and post-course measures of economic knowledge) as the dependent variable and mathematics learning as an independent variable (the difference between pre and post- course measures of mathematics literacy). These results are shown in Table 6.

The results of the pre-course survey models (Table 4) indicate that economics majors did worse on the economics questions than business majors and other majors, though the coefficient for other majors was not statistically significant. Students who were enrolled in macroeconomics but have never had another economics class (MACRO) and students enrolled in macroeconomics who have had microeconomics previously (MAC HAD MIC) did statistically better than students currently enrolled in microeconomics who also had not taken another economic course. The coefficients on all three university dummy variables were negative and significant, indicating that University of Richmond students did better on the pre-course survey than students at other the institutions. Finally, as expected, students who had a greater number of correct answers on the quantitative skills test (TMATHPRE) performed statistically better on the economics questions but by a negligible amount. With regard to model 2, we see that only question five (interpreting a percentage change) was a statistically significant determinant of total number of economics questions correct. (19) This overall lack of significance for the individual math questions is an interesting result, as it shows that before entering the economics course, specific sets of math skills do not necessarily correspond with more knowledge of economics.

The results of the post-course survey models (Table 5) indicate that economics majors are no longer statistically different than business majors and other students; over the course of the semester, they have caught up to business and other majors. We also see that there is now a smaller difference between students at the different universities, yet the coefficient on the UNCW dummy variable is still significant. Once again, macroeconomics students (MACRO) fared more favorably on the economics questions than microeconomics students, although in the post-course survey results there is no longer a statistical difference between students in macroeconomics who have had microeconomics (MAC HAD MIC) and students in microeconomics with no other economic background (base case). Math skills were once again positively related to economic literacy. Indeed, the relationship for total math questions correct (TMATHPOST, model 1) was stronger in the post-course survey. This indicates that students who did better on the math questions continued to do better on the economics questions. In other words, the significance of TMATHPOST implies that learning during the economics class is aided by math skills. Model 2 provides further evidence of the importance of specific quantitative skills. Although question 5 is no longer significant, questions 1, 2 and 3 are now positive and significant. Thus, having skills such as solving a system of equations, computing a percentage, and reading increases and decreases on a graph contributes to economic knowledge as measured at the end of the course. It is important to note that questions 1 and 2 are computational questions, while question 3 is an interpretive question. Hence, in viewing this result (and comparing it to the pre-course results for model 2), we can state that students with these specific skills at the end of the course also had higher knowledge of economic concepts.

With regard to the difference between the pre-course and post-course survey results (Table 6), we find that economics majors have significantly greater increases in economic knowledge compared to business majors and students with majors other than business. Quantitative skill acquisition (TMATHDIFF) significantly increases economic learning. Students in macroeconomics who have had microeconomics (MAC HAD MIC) do not improve as much as students in microeconomics (who haven't had macroeconomics). This would seem to indicate that it is the first microeconomics class that makes the most significant contribution to economic learning. Finally, MWC students improve significantly more than UR students.

Surprisingly, other variables representing successful completion of previous math courses were found to be insignificant in our study. Models that included the number of math classes taken in high school (excluding high school calculus) and college and whether or not students had taken nigh school calculus were also estimated. Consistent with previous research, we anticipated positive coefficient estimates for these variables; however, these variables were not significant when TMATHPRE and TMATHPOST were included in the pre and post course models, respectively. In the pre course model, the coefficient estimates were not significant even without the TMATHPRE variable. In the post course model, only the high school calculus variable was significant without the inclusion of the TMATHPOST variable. In addition, these variables did not impact economic learning.

VIII. Conclusions

Using a survey administered on the first and last days of principles of economics courses at four U.S. universities, the current study set out to determine the important influences on economic literacy and economic learning, measured, respectively, as the number of correct responses on a 10 question economics questionnaire and the difference in the number of correct economics responses on the first day and the last day of class.

Generally, students do not fare well on simple quantitative questions and hence do not possess an adequate working knowledge of the "language" we often speak during our economics courses. Our analysis shows, however, that quantitative literacy is a very important determinant of economic literacy in both the pre and post course surveys. More specifically, we have shown that having skills such as being able to solve a system of equations and compute a percentage, and being able to interpret increases and decreases on a graph will lead to higher economic knowledge at the end of the semester.

We also find that high school economics courses and high school and college math courses do not significantly impact economic literacy or learning. Interestingly, although students who had already taken microeconomics did significantly better on both the pre and post-course economics questions compared to students entering their first microeconomics course (as might be expected), students in the first microeconomics course learned more economics than did students who were in macroeconomics and had already taken microeconomics.

While math literacy improves during an economics principles course, students still performed poorly on quantitative questions at the end of the semester. This is unfortunate; as our paper also shows that quantitative improvement leads to more economic learning. Hence it would seem that in order for students to learn more during economics courses, they need a stronger set of math skills. Moreover, those who teach economics principles should not assume that students have already mastered basic math skills. Indeed, time spent addressing these shortcomings as needed during the course of the semester could prove quite effective in improving economic learning. Providing economics students with a basic math primer or allowing for more in-depth review of basic math concepts and skills, while taking away from the amount of time directly spent on economics content could serve to enhance the amount of economics learning that takes place.

Appendix

Mathematics Questions

1. Solve this system of equations:

3 x + y = 15 -x + 2y = 2

a. x = 5, y = 5

b. x = 4, y = 3

c. x = 3, y = 4

d. x = 0, y = 0

e. none of the above

2. What is 30% of 200?

a. 30 b. 60 c. 90 d. 33 e. none of the above.

3. Consider Figure 1 where US consumption of oil and US imports of oil are plotted for 1991-2000. Which of the following is true?

a. U.S. domestic oil consumption has been steady while imports have been rising therefore US domestic oil production has been rising.

b. U.S. domestic oil consumption has been steady while imports have been falling therefore US domestic oil production has been falling.

c. U.S. domestic oil consumption has been steady while imports have been rising therefore US domestic oil production has been falling.

d. U.S. domestic oil consumption has been steady while imports have been falling therefore US domestic oil production has been rising.

[FIGURE 1 OMITTED]

4. Refer to the following table:

Economic Growth in Poland, 1990-1994 (percent)

 1990 1991 1992 1993 1994

GDP -11.7% -7.8% -1.5% 4.0% 3.5%
growth
rate

Which of the following are TRUE?

a. Polish GDP was larger in 1992 than in 1991.

b. Polish GDP was larger in 1994 than in 1993.

c. Polish GDP was larger in 1991 than in 1992.

d. Polish GDP was larger in 1993 than in 1994.

e. b & c are true

5. If U.S. production of corn was 60 million bushels in 1998 and 100 million bushels in 1999, what was the percentage change in corn production from 1998 to 1999?

a. 40% b. 60% c. 66.67% d. 100% e. 200%

6. Which of the following is the correct formula for the following relationship between the price of a good (P) and the quantity that is demanded (Q) shown in Figure 2 below?

a. Q = 10 + 5P

b. Q = 5 + 1P

c. Q = 10 - 1p

d. Q = 10 - 5P

e. None of the above

[FIGURE 2 OMITTED]

7. Given the relationship in Figure 3 between x and y, which of the following are true?

a. The solid line has larger x values for given values of y than the dashed line.

b. The solid line has smaller x values for given values of y than the dashed line.

c. The dashed line has smaller y values for given values of x than the solid line.

d. The dashed line has larger y values for given values of x than the solid line.

e. Both b and c

f. Both c and d

8. Referring to figure 3 below, the slope of the line shown is:

a. 1 b. -1 c. 2 d.-2 e. 1/2

[FIGURE 3 OMITTED]

TABLE 1
Variable description and mean (proportion)

Variable Name Description Mean

UR 1 = student attends University of Richmond; 0.134
 0 = other
MWC 1 = student attends Mary Washington College; 0.294
 0 = other
UNCW 1 = student attends UNC-Wilmington; 0.340
 0 = other
UN 1 = student attends University of Nebraska; 0.233
 0 = other
MALE 1 = male; 0 = female 0.497
ECON 1 = student is econ major; 0 = other 0.032
BUSINESS 1 = student is business major; non-econ 0.544
 major; 0 = other
OTHER 1 = student is non-business major; non-econ 0.424
 major; 0 = other
WHITE 1 = student is Caucasian; 0 = other 0.907
FOREIGN 1 = student is foreign; 0 = other 0.044
GPA Student (self-reported) GPA 3.053
GEOMETRY 1 = student took geometry in high school; 0.898
 0 = other
ALGEBRA1 1 = student took algebra 1 in high school; 0.924
 0 = other
ALGEBRA2 1 = student took more than 1 algebra course 0.826
 in high school; 0 = other
HSPCALC 1 = student took pre-calculus in high 0.712
 school; 0 = other
HSCALC 1 = student took calculus in high school; 0.372
 0 = other
HSMATH Number of courses above basic math taken in 3.733
 high school
CBASICMATH 1 = student took basic math in college; 0.119
 0 = other
CALGEGRA 1 = student took algebra in college; 0.253
 0 = other
CCALC 1 = student took calculus in college; 0.404
 0 = other
CPROB&STAT 1 = student took probability and statistics 0.238
 in college; 0 = other
CUPPERMATH 1 = student took an upper level math course; 0.087
 0 = other
CMATH Number of courses above basic math taken in 1.102
 college
HSECON Number of economics classes taken in high 0.381
 school
ECON1 1 = this class is the first economics class 0.590
 taken in college; 0 = other
MACRO 1 = student is in macroeconomics and has not 0.314
 completed another economics course;
 0 = other
MICRO 1 = student is in microeconomics and has not 0.276
 completed another economics course;
 0 = other
MAC HAD MIC 1 = student is in macroeconomics and has 0.250
 completed microeconomics; 0 = other
MIC HAD MAC 1 = student is in microeconomics and has 0.142
 completed macroeconomics; 0 = other
MIC HAD SURVEY 1 = student is in microeconomics and has 0.006
 completed a survey of economics course;
 0 = other
MAC HAD MIC AND 1 = student is in macroeconomics and has 0.012
 SURVEY completed microeconomics and an economics
 survey course; 0 = other
RSURVEY 1 = random design survey (economic knowledge 0.444
 questions randomly chosen for inclusion)
TMATHPRE Total number of math questions correct 4.24
 pre-course
TMATHPOST Total number of math questions correct 4.46
 post-course
TMATHDIFF TMPOST--TMPRE 0.21

TABLE 2
Percent of students giving correct answers on math and economics
questions.

 Percent of Students with
 Correct Answer

Math Questions (n = 344) Pre Post Z-stat

 1: solving a system of equations 0.814 0.834 0.701
 2: computing a simple percentage 0.936 0.892 -2.042 **
 3: reading increases and decreases on a 0.640 0.666 0.721
 graph
 4: interpreting meaning of a percentage 0.244 0.273 0.871
 change
 5: calculating percentage change, word 0.404 0.419 0.387
 problem
 6: determining slope and intercept 0.363 0.448 2.252 **
 7: distinguishing differences across two 0.276 0.308 0.922
 graphs
 8: determining slope 0.567 0.616 1.319
 Avg. Percent (questions 1-8) 0.531 0.557 0.689

Economics Questions (n = 344)
 Micro 0.317 0.430 3.042 *
 Macro 0.264 0.313 1.430
 Avg. Percent (all econ questions) 0.291 0.372 2.252 **

*, **, *** indicates significance at the 1, 5, and 10% levels,
respectively.

TABLE 3
Average Number of Correct Math, Micro, Macro, and Economics Questions

 Pre Sdev Post

Math questions correctly answered 4.244 1.741 4.456
 (Out of 8 total)
Economic questions
 Micro questions correctly answered 1.587 1.187 2.148
 (Out of 5 total)
 Macro questions correctly answered 1.320 1.076 1.567
 (Out of 5 total)
 Economic questions correctly 2.907 1.705 3.715
 answered (Out of 10 total)

 Sdev tstat

Math questions correctly answered 1.924 1.578
 (Out of 8 total)
Economic questions
 Micro questions correctly answered 1.256 6.260 *
 (Out of 5 total)
 Macro questions correctly answered 1.138 3.043 *
 (Out of 5 total)
 Economic questions correctly 1.898 6.109 *
 answered (Out of 10 total)

* indicates significance at the 1% level.

TABLE 4
Pre Survey Results--Poisson Regression Estimates

 Model 1

Variable coefficient tstat

Constant 0.643 2.704 *
BUSINESS 0.378 1.884 ***
OTHER 0.276 1.366
HSECON 0.078 1.343
MACRO 0.329 2.936 *
MAC HAD MIC 0.360 3.863 *
MIC HAD MAC 0.044 0.331
MIC HAD SUR 0.057 0.137
MAC HAD MIC AND SURVEY 0.183 0.563
TMATHPRE 0.037 1.906 ***
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8
MWC -0.500 -4.485 *
UNCW -0.377 -3.110 *
UN -0.325 -2.517 **
RSURVEY 0.087 1.158
Pseudo [R.sup.2] 0.154

 Model 2

Variable coefficient tstat

Constant 0.742 2.89 *
BUSINESS 0.409 2.025 **
OTHER 0.322 1.579
HSECON 0.072 1.211
MACRO 0.291 2.586 *
MAC HAD MIC 0.333 3.518 *
MIC HAD MAC 0.027 0.200
MIC HAD SUR 0.191 0.450
MAC HAD MIC AND SURVEY 0.240 0.729
TMATHPRE
Q1 -0.131 -1.566
Q2 -0.024 -0.182
Q3 0.027 0.376
Q4 0.082 1.045
Q5 0.117 1.662 ***
Q6 -0.064 -0.899
Q7 0.071 0.952
Q8 0.111 1.554
MWC -0.478 -4.186 *
UNCW -0.352 -2.871 *
UN -0.297 -2.284 **
RSURVEY 0.089 1.186
Pseudo [R.sup.2] 0.171

*, **, *** indicates significance at the 1, 5, and
10% levels, respectively.

TABLE 5
Post Survey Results--Poisson Regression Estimates

 Model 1

Variable coefficient tstat

Constant 0.999 5.236 *
BUSINESS -0.231 -1.598
OTHER -0.152 -1.046
HSECON 0.050 0.971
MACRO 0.172 1.777 ***
MAC HAD MIC 0.035 0.422
MIC HAD MAC 0.012 0.103
MIC HAD SUR 0.453 1.473
MAC HAD MIC
AND SURVEY -0.357 -1.025
TMATHPOST 0.093 5.776 *
Q1POST
Q2POST
Q3POST
Q4POST
Q5POST
Q6POST
Q7POST
Q8POST
MWC -0.065 -0.642
UNCW -0.187 -1.658 ***
UN -0.028 -0.247
RSURVEY 0.156 2.231 **
Pseudo [R.sup.2] 0.158

 Model 2

Variable coefficient tstat

Constant 0.800 3.658 *
BUSINESS -0.221 -1.516
OTHER -0.132 -0.905
HSECON 0.041 0.783
MACRO 0.198 2.016 **
MAC HAD MIC 0.063 0.746
MIC HAD MAC 0.042 0.363
MIC HAD SUR 0.515 1.636
MAC HAD MIC
AND SURVEY -0.339 -0.965
TMATHPOST
Q1POST 0.178 1.978 **
Q2POST 0.231 2.038 **
Q3POST 0.141 2.094 **
Q4POST 0.009 0.134
Q5POST 0.105 1.633
Q6POST 0.035 0.553
Q7POST 0.099 1.531
Q8POST 0.071 1.031
MWC -0.077 -0.756
UNCW -0.187 -1.640 ***
UN -0.028 -0.244
RSURVEY 0.158 2.235
Pseudo [R.sup.2] 0.164

*, **, *** indicates significance at the 1, 5,
and 10% levels, respectively.

TABLE 6
OLS Results for the difference in correct answers
on the pre and post surveys

Variable coefficient tstat

Constant 2.168 2.855 *
BUSINESS -2.128 -3.130 *
OTHER -1.499 -2.203 **
HSECON 0.079 0.371
MACRO -0.155 -0.390
MAC HAD MIC -0.837 -2.450 **
MIC HAD MAC 0.033 0.074
MIC HAD SUR 1.914 1.256
MAC HAD MIC AND SURVEY -1.403 -1.212
TMATHDIFF 0.254 3.851 *
MWC 0.994 2.357 **
UNCW 0.252 0.543
UN 0.697 1.451
RSURV 0.152 0.531
[R.sup.2] 0.136

*, **, *** indicates significance at the 1, 5,
and 10% levels, respectively.

The authors wish to thank Robin Bartlett, Bill Goffe, Steven Greenlaw, Gail Hoyt, Anne Owen, and Michael Salemi for valuable comments on an earlier draft and faculty at each of the institutions included in this project for their willingness to participate in the survey. Additionally, Jerry Petr provided invaluable motivation for and feedback on this project. We also wish to thank the anonymous referee for helpful comments provided in the review of this paper.

Notes

(1.) Revisions included adding additional demographic specifications and math course background questions.

(2.) Approximately fifty-eight percent of the surveys were administered in principles of macroeconomics classes and forty-two percent were administered in principles of microeconomics.

(3.) Because a large number of students completed the pre-course survey but failed to complete the post-course survey, the potential for sample selection bias exists. Based on the pre-course survey results we found no statistically significant difference between students who completed both surveys and those who only completed the pre-course survey in terms of their characteristics or math and economics skills.

(4.) There were 10 students who were double majors in combinations of economics, business, and something other than business and economics. If economics was one of the majors, the student was counted as an economics major. If the student was double majoring in business and something other than business or economics, the student was counted as a business major.

(5.) This average only pertains to students who reported their GPA. In this sample, the non-reporters gave the answer "this is my first class/semester at this institution--I do not have a GPA."

(6.) At the University of Nebraska, macroeconomics is generally the first course taken, while microeconomics is usually taken first at the oher institutions.

(7.) These numbers do not sum to 100 percent because 4 students had taken both microeconomics and a survey class.

(8.) Of the 6 students who had taken the survey course, 2 were in enrolled in microeconomics and 4 students were in macroeconomics having completed both microeconomics and a survey course. There were no students in the sample who had taken only the survey course and then enrolled in macroeconomics.

(9.) The author-generated math questions were written to address math skills that might be required in a principles of economics course. These include solving a system of equations, calculating and understanding percent changes, and understanding slope and intercept from a graph. These questions were reviewed by economics faculty and pre-tested in principles courses.

(10.) These questions are similar to algebraic questions included in Ballard and Johnson (2004).

(11.) In order to avoid inherent biases in limiting 30 potential TUCE economic knowledge questions to ten, two versions of the survey were developed. The first included ten questions selected by the authors (N = 191, non-randomized surveys), while the second version of the survey included ten TUCE questions that were selected randomly (N = 153, randomized surveys). Students were given the same version of the survey at both the beginning and end of the semester. No significant differences were found between the two versions of the survey, hence we surpress this detail for the remainder of the paper.

(12.) Each question had 4 possible answers.

(13.) Remaining variable definitions are provided in Table 1.

(14.) Variance inflation factors for our data are lower than 10 for all the variables. Thus, there is no indication of multicollinearity (Chatterjee and Price, 1991).

(15.) Similarly, demographic variables such as gender and GPA were found to be insignificant in all models.

(16.) We also estimated models that included the number of math classes taken in high school (excluding high school calculus) and college and with high school calculus as a dummy variable. These variables were generally not significant.

(17.) A significant coefficient for this variable will indicate that there is a statistical difference between the results of the random and non-random versions of the survey (pre-course, post-course or difference).

(18.) The coefficients in the Poisson models are interpreted as logarithmic elasticities. That is, they show the percentage change in the dependent variable for a one-unit change in the independent variable.

(19.) We suspect that the similarity between certain combinations of questions may dilute their individual significance in these regressions. A model was also estimated which included the total number of computational questions correct (out of questions 1, 2, 6 and 8) and total number of interpretive questions correct (out of questions 3, 4, 5, and 7) as independent variables. Only the latter was significant in the pre-course regression (effectively capturing the significance of question 5 in model 2), and neither was significant in the post-course regression. Because these results do not add any information beyond that already captured by models 1 and 2, we do not include them here. A factor analysis, which would effectively combine similar questions into categories of skills, may more accurately reveal the differences between questions, but is beyond the scope of this paper.

References

Anderson, G., D. Benjamin, and M. Fuss. 1994. The Determinants of Success in University Introductory Economics Courses. Journal of Economic Education, 25:99-119.

Ballard, Charles and Marianne E Johnson. 2004. Basic Math Skills and Performance in an Introductory Economics Class. Journal of Economic Education, 25: 3-23.

Beck, Klaus and Volker Krumm. 1994. Economic Literacy in German-Speaking Countries and the United States: Methods and First Results of a Comparative Study. In An International Perspective on Economic Education. Walstad, William B., ed., Dordrecht and Boston: Kluwer Academic: 183-201.

Bennett, James and Manuel Johnson. 1979. Mathematics and Quantification in Economic Literature: Paradigm or Paradox? Journal of Economic Education, 11: 40-41.

Blinder, Alan S. 1999. Economics Becomes a Science- Or Does It? Useful Knowledge The American Philosophical Society Millennium Program Alexander G. Beam, Editor. Philadelphia, PA : American Philosophical Society.

Blank, Rebecca M. 2002. Promoting Economic Literacy: Panel Discussion. American Economic Review, 92 (2): 476-77.

Brasfield, David, James McCoy, and Martin Milkman. 1992. The Effect of University Math on Student Performance in Principles of Economics. Journal of Research and Development in Education, 25 (4): 240-247.

Butler, J.S., T. Aldrich Finegan, and John J. Siegfried. 1994. "Does More Calculus Improve Students Learning in Intermediate Micro and Macro Economic Theory?" American Economic Review, 84 (2): 206-210.

Butler, J S, Aldrich T. Finegan, and John J. Siegfried. 1998. Does More Calculus Improve Student Learning in Intermediate Micro- and Macroeconomic Theory? Journal of Applied Econometrics, 13 (2): 185-202.

Chatterjee, S. and B. Price. 1991. Regression Analysis by Example. John Wiley and Sons.

Cohn, E. 1972. Student's Characteristics and Performance in Economic Statistics. Journal of Economic Education, 3:106-111.

Cohn, Elchanan and Sharon Cohn. 1994. Graphs and Learning in Principles of Economics. American Economic Review, 84 (2): 197-200.

Cohn, Elchanan, Sharon Cohn, R. E. Hult Jr., J. Bradley, Jr, and D.C. Balch. 2000. Improved Knowledge Of Mathematics And Enrollment in a Principles Of Economics Course: Is There A Link? International Journal of Mathematical Education in Science and Technology, 31(2): 195-203.

Cohn, Elchanan, Sharon Cohn, Donald C. Balch, and James Bradley, Jr. 2001. Do Graphs Promote Learning in Principles of Economics? Journal of Economic Education, 32 (4). p 299-310.

Durden, Garey C. and Larry V. Ellis. 1995. The Effects of Attendance on Student Learning in Principles of Economics. American Economic Association Papers and Proceedings, 85: 343-346.

Espey, Molly. 1997. Testing Math Competency in Introductory Economics. Review of Agricultural Economics, 19(2): 484-491.

Ferber, Marianne A. 1999. Guidelines for Pre-college Economics Education: A Critique. Feminist Economics, 5 (3): 135-42.

Gery, Frank W. 1970. Mathematics and the Understanding of Economic Concepts. Journal of Economic Education, 2: 100-104.

Gery, Frank W. 1972. Does Mathematics Matter? in Research papers in Economic Education. Arthur L. Welch (ed.) NY: Joint Council on Economic Education: 143-157.

Gleason, Joyce and Lee J. Van Scyoc. 1995. A Report on the Economic Literacy of Adults. Journal of Economic Education, 26(3): 203-210.

Green, David A and Craig W. Riddell. 2001. Literacy, Numeracy and Labour Market Outcomes in Canada. University of British Columbia, Department of Economics Discussion Paper: 01/05.

Hill, Cynthia and Tessa Stegner. 2003. Which Students Benefit from Graphs in a Principles of Economics Class? American Economist, 47 (2): 69-77.

Kim, Kyung-keun. 1994. Economic Literacy in the Republic of Korea and the United States. In An International Perspective on Economic Education. Walstad, William B., ed., Dordrecht and Boston: Kluwer Academic: 203-18.

Kourilsky, M. and M.C. Wittrock. 1987. Verbal and Graphical Strategies In The Teaching Of Economics. Teaching and Teacher Education, 3(1): 1-12.

Krueger, Alan B. 2002. Promoting Economic Literacy: Panel Discussion. American Economic Review, 92 (2): 475-76.

Lucas, Robert E, Jr. 2002. Promoting Economic Literacy: Panel Discussion. American Economic Review, 92 (2): 473-75.

Milkman, Martin, James McCoy, David Brasfield, and M. Mitchell. 1995. Some Additional Evidence on the Effect of University Math On Student Performance In Principles of Economics. Journal of Research and Development in Education, 28(4): 220-229.

Nelson, Julie A and Steven M. Sheffrin. 1991. Economic Literacy or Economic Ideology? Journal of Economic Perspectives, 5 (3): 157-65.

Parkison, Kathy and Margo Sorgman. 1998. Enhancing Economic Literacy of Classroom Teachers. International Advances in Economic Research, 4 (4): 418-27.

Rivera-Batiz, Francisco L. 1992. Quantitative Literacy and the Likelihood of Employment among Young Adults in the United States. Journal of Human Resources, 27 (2). p 313-28.

Siegfried, John, Robin L. Barlett, W. Lee Hansen, Allen C. Kelley, Donald N. McCloskey, and Thomas H. Tietenberg. 1991. The Status and Prospects of the Economics Major. Journal of Economic Education, 22 (Summer): 197-24.

Soper, John C. and William B. Walstad. 1988. "What is High School Economics? Posttest Knowledge, Attitudes, and Course Content." Journal of Economic Education, vol. 19(1): 37-52

Stigler, George J. 1983. The Case, If Any, for Economic Literacy. Journal of Economic Education, 14 (3): 60-66.

Von Allmen, O. 1996. The Effect of Quantitative Prerequisites on Performance in Intermediate Microeconomics. Journal of Education for Business, 72(1), p. 18-22.

Walstad, William B and John C. Soper. 1988. A Report Card on the Economic Literacy of U.S. High School Students. American Economic Review, 78 (2): 251-56.

Walstad, William B and John C. Soper. 1991. Economic Literacy in Senior High Schools. In Effective Economic Education in the Schools. Walstad, William B. Soper, John C., eds., NEA Professional Library. Reference and Resource Series Washington, D.C.: National Education Association; New York: Joint Council on Economic Education: 99-116.

Whitehead, David J and Tony Halil. 1991. Economic Literacy in the United Kingdom and the United States: A Comparative Study. Journal of Economic Education, 22 (2): 101-10.

Wood, William C and Joanne M. Doyle. 2002. Economic Literacy among Corporate Employees. Journal of Economic Education, 33 (3): 195-205.

Peter W. Schuhmann *, KimMarie McGoldrick **, and Robert T. Burrus ***

* Department of Economics and Finance, University of North Carolina at Wilmington, 601 South College Rd., Wilmington, NC 28403-3297, Phone: (910) 962-3417, Fax: (910) 962-7464, Email: schuhmannp@uncw.edu.

** Department of Economics, University of Richmond, Richmond, VA 23173, Phone: (804) 289-8575; Fax: (804) 289-8878, Email: kmcgoldr@richmond.edu.

*** Department of Economics and Finance, University of North Carolina at Wilmington, 601 South College Rd., Wilmington, NC 28403-3297, Phone: (910) 962-3276, Fax: (910) 962-7464, Email: burrusr@uncw.edu.