District level mandates and high school students' understanding of economics.

Grimes, Paul W. ; Millea, Meghan J. ; Thomas, M. Kathleen 等

INTRODUCTION

Mississippi will soon be joining the growing number of states that require students to complete a formal course in economics prior to graduation from high school (Grimes and Millea, 2003). Under the Mississippi Department of Education's (MDE's) new minimum curriculum standards, the freshman class of 2008 must complete a one-semester course in economics to satisfy graduation requirements. Along with civics, history, and geography, economics is one of the four "strands" of Mississippi's social studies curriculum. The economics curriculum standards of the MDE are modeled on the National Council on Economic Education's (NCEE's) Voluntary National Content Standards in Economics (1997). School districts are held accountable for meeting these standards whether economics is offered as a stand-alone course or infused into other courses. Currently, the state's graduation policies require completion of either a course in economics or a course in geography. Historically, and for a variety of reasons, some of Mississippi's 153 school districts already have local graduation policies that require high school students to complete an economics course. With the upcoming change in the state's minimum standards, it is expected that more school districts will move to mandate the high school economics course prior to 2012 when all graduating students must have completed the course.

The current situation in Mississippi provides an unusual environmental context to investigate the effects of a mandated high school course in economics on student learning. In this paper we address the following question: Do high school students in school districts with a local mandate for a stand-alone economics course demonstrate an equivalent understanding of economics relative to their cohorts in districts where an economics course is not mandated but rather is offered as an elective or is infused into other courses? In addition to addressing the current situation in Mississippi, our empirical results will offer insight into the impact of state-wide course mandates in general.

THE LITERATURE

Only a few previous researchers have examined the effects of economics course mandates. However, all of these studies consistently compare across different states--those with a mandate and those without a mandate. For example, Rhine (1989), Marlin (1991), and Soper and Lynn (1994) all used the National Assessment of Economic Education database to examine teacher attitudes and student learning across mandate and non-mandate states, and Belfield and Levin (2004) employed a nation-wide database of more than 600,000 students to examine the effect of state-level mandates on the general scholastic aptitude of students. Interestingly, each of these studies found that economic course mandates at the state level could have undesirable negative effects; Marlin found that teacher attitudes towards economics were lower in mandated states, which could lead to poorer student performance, and Belfield and Levin found that an economic course mandate reduced instructional attention in other subject areas leading to lower student SAT scores. However, to date, no one has examined the effects of local school district mandates within a state. Given the natural diversity of educational environments across states, an intra-state examination of local mandates may provide a more accurate measure of the impact of requiring a specific course of study.

THE DATA

During the Fall of 2006, students from ten high schools across the state of Mississippi were tested and surveyed. Five classes of students were taking the one semester stand-alone economics course mandated by their school district. Six classes of students were taking the same course as a social studies elective offered by their school district. An additional two classes were studying economics only as an infusion into their required United States History course. All of the courses were taught by teachers who had participated in a multi-day summer workshop produced by the Mississippi Council on Economic Education (MCEE).

Each of the participating teachers pre- and post-tested their students using the Test of Economic Literacy (TEL) (Walstad and Rebeck, 2001a). All of the students also completed a survey that collected their basic demographic characteristics and a limited amount of information about their families and school experiences and activities. The final sample consisted of 211 student observations. All testing and data collection procedures were conducted in accordance with the federal regulations for human subjects research involving minors.

DESCRIPTIVE ANALYSIS

Table 1 provides the mean pre-course and post-course TEL scores for each of the major student groups within the sample. Due to student absences, out of the 211 original observations, there were 189 paired pre-course and post-course scores available for analysis. Overall, the full-sample of students demonstrated a 2.68 item increase in mean score, on the 40-item TEL, between the beginning and end of the academic semester. A paired sample t-test revealed that this difference was statistically significant. Likewise, a significant increase in raw TEL scores was also found for the non-mandate group of students. In fact, the non-mandate students improved their mean score by 3.80 points--more than a full point greater than for the full-sample. Table 1 shows that this result is due to the large and significant increase observed for the non-mandate students taking a stand-alone economics course as an elective. These students ended the semester with the largest gain of any group--5.22 points. The non-mandate students receiving economic instruction infused into their history course did not achieve a statistically significant improvement in economic understanding, as reflected in their mean TEL scores. And most importantly, the students who were mandated to take a stand-alone economics course improved their mean score by only about one item--a paired sample t-test revealed that this was not a statistically significant difference between pre-course and post-course scores.

Thus, the descriptive analysis indicates that the largest improvements in student understanding of economics occurred in schools which offered economics as an elective course and not as a graduation requirement. The least amount of learning occurred in schools where economic content was infused into another required course. Furthermore, analysis of the mean scores suggest that mandating an economics course does not guarantee that significant learning gains will be observed. However, numerous factors contribute to the performance of students on standardized tests. To account for these factors a multivariate regression analysis was conducted.

THE REGRESSION MODEL

In keeping with the long tradition of regression-based educational production functions (Becker and Walstad, 1987), and echoing Soper and Lynn's (1994) mandate model, the following relationship was postulated:

POST TEL = f (Student Demographics, Student Educational Attributes, Student Market Experiences, School Attributes) [1]

where, the right hand side is composed of vectors of variables representing each of the factors assumed to determine post-course student performance on the TEL. Table 2 lists the individual variables within each vector and reports their empirical specifications. Table 3 provides the means and standard deviations for each variable broken down across mandate or non-mandate status, and for the full-sample. Based on surveys of prior high school-level research (see Walstad (2000) and Walstad and Rebeck (2001b)) the expected sign for each variable's regression coefficient is also reported in Table 3.

As noted above, not all students completed both the pre-course and postcourse TEL due to absence from school on test day. To account for the possibility of selection bias due to this attrition, Equation [1] was estimated using Heckman's (1979) two stage self-selection technique as recommended by Becker and Walstad (1990). The first stage of this approach requires the estimation of a probit equation designed to capture the effect of independent variables on the probability of remaining in the sample. Table 4 reports the results from this first stage. Only two probit coefficients were found to be statistically significant. MOTHER'S EDUCATION and CLUBS were both found to have a positive effect on completing both the pre- and post-course TEL. Thus, students from families with relatively strong investments in human capital and students with a demonstrated attachment to academic activities were less likely to be absent on test day.

Table 5 reports the second stage regression results. The LAMBDA coefficient captures the self-selection effect estimated from the first stage probit results. In this case, it is statistically insignificant indicating that the observed student absences did not structurally affect the overall results. The estimated equation obtained a significant F-statistic and a very reasonable cross-sectional adjusted [R.sup.2] of .475.

EMPIRICAL RESULTS

Before turning to the primary results of interest concerning the effect of course mandates on student learning, it is important to note several interesting findings revealed by the estimated coefficients for the control variables. All of the independent variables obtained coefficients with the a prori expected sign. As seen in Table 5, a student's prior understanding of economics was an important determinant of end-of-course understanding. The PRE TEL coefficient obtained the largest positive magnitude of any significant control variable. This is consistent with previous studies that include pre-course measures of understanding on the right hand side (Becker and Walstad, 1987). The variable found to have the largest negative effect on POST TEL performance was BLACK. Thus, holding all else constant, students who identified themselves as a member of a racial minority had lower end-of-course TEL scores, relative to their white cohorts. While this finding is also consistent with previous research, more work needs to be done to determine what underlying factors may be responsible.

A statistically significant coefficient was estimated for each of the student educational attribute variables. Students who had completed a calculus course scored almost six percentage points higher on the POST TEL, all else being the same. Although joining an additional extracurricular organization was associated with a one percentage improvement in score, students who participated in organized school sports demonstrated a five percentage point drop in score, holding all else constant. This result was likely due to the significant opportunity costs of the time commitment necessary to play on a high school athletic team.

The STOCK MARKET variable was included to capture the spillover effect of a student's previous participation in an MCEE sponsored program. As in many other states, Mississippi's annual simulated stock market competition serves as an entry-level program offered by the state council on economic education. Only about seven percent of the students in our sample had participated in a previous competition (see Table 3). However, participation was found to have a positive effect on POST TEL scores. The STOCK MARKET coefficient reported in Table 5 indicates that, holding all else the same, prior participation in the stock market competition was associated with an eight percent increase in end-of-course test performance. This result is consistent with results from a state-wide analysis of student achievement in Georgia (Swinton, DeBerry, Scafidi and Woodard, 2007) and findings from the recent National Assessment of Educational Progress (NAEP) in Economics (Walstad and Buckles, 2008).

The BANKING variable was included in our model to capture the potential effects of "real life" participation in the economy on economic understanding. However, students who held a bank account and credit card in their own name did not perform differently from those who did not have such accounts, ceteris paribus. Likewise, no peer effect was uncovered for students whose friends earned relatively high grades.

For this study, the two most important coefficients were those estimated for the INFUSED and MANDATE variables. As seen in Table 5, both of these coefficients were estimated to be negative and statistically significant. The magnitude of the INFUSED coefficient indicates that students studying economics through infusion in a history course scored 17.7 percent below their cohorts, holding all else the same. Clearly, this result suggests that the one course infusion approach is not the optimal strategy to implement successful economic education at the high school level. The magnitude of the MANDATE coefficient indicates that students who were required to take economics as a graduation requirement scored 8.2 percent below their cohorts, holding all else constant. This result is consistent with the previously cited inter-state research on economics course mandates. Thus, there appears to be something about implementing a course mandate that results in the observation of significantly lower student performance scores relative to those observed for students when the same course is offered as an elective.

CONCLUSIONS

Our empirical examination revealed that high school student learning of economics varies according to course structure. The least effective structure was the infusion approach whereby students studied economics within the context of a required United States history course. The regression model estimates that, holding all else constant, students taught via infusion scored almost 18 percent below their cohorts who took a stand-alone economics course as an elective. Likewise, students who took a mandated stand-alone economics course scored eight percent below those same cohorts who took the course as an elective. Apparently, requiring an economics class for graduation is not the most effective course structure to generate high end-of-course standardized test scores.

While our analysis focused on district level course mandates within one state, the results are consistent with previous research on state level mandates. Therefore, to date, the evidence suggests that economics course mandates are not the optimal policy to maximize student learning. What are the causal factors behind these findings? As Marlin (1991) pointed out, when courses are required to be offered, schools may be forced to place teachers lacking the requisite skill base into the classroom. This then may lead to ineffective teaching, poor learning, and frustrated teachers and students. On the other hand, elective courses are more likely to be taught by teachers who "champion" the subject and have the necessary skill base for that discipline.

The empirical results may also reflect student selection processes. When a course is offered only as an elective, it is natural that students with an interest and proclivity in the subject are more likely to enroll. Thus, teachers of elective courses face classrooms of students who have a higher probability of success. On the other hand, when a course is mandated and all students are required to enroll, classrooms reflect the entire distribution of student abilities. Thus, students in elective courses are being drawn from the upper tail of the ability distribution while students in mandated courses are drawn from across the entire distribution. Additional investigations and richer data sources are needed to sort out this particular selection process.

It is important to note that the current results do not suggest that an economics course mandate is always a bad idea. Although student learning in a mandated course may not be optimal, without course mandates many students would never be exposed to any formal economics. What the results do suggest is that economics instructors in a mandated course environment may face tougher teaching challenges relative those who teach elective classes.

ACKNOWLEDGMENTS

Financial support for this project provided by an Excellence in Economic Education subgrant from the National Council on Economic Education through funding from the United States Department of Education, Office of Innovation. The authors thank Josh D. Pitts for invaluable research assistance and Kristi James of the Mississippi Council on Economic Education for data collection. Thanks are also extended to Marybeth Grimes for editorial assistance.

REFERENCES

Belfield, C. R. and H. M. Levin (2004). Should high school economics courses be compulsory? Economics of Education Review, 23(4), 351-360.

Becker, W. E. and W. B. Walstad (1987). Econometric modeling in economic education research. Boston, MA: Kluwer-Nijhoff Publishing.

Becker, W. E. and W. B. Walstad (1990). Data loss from pretest to posttest as a sample selection problem. Review of Economics and Statistics, 72(1), 184-188.

Grimes, P. W. and M. J. Millea. (2003). Economic education as public policy: The determinants of state-level mandates. Journal of Economics and Economic Education Research, 4(2), 3-18.

Heckman, J. (1979). Sample bias as a specification error. Econometrica, 47(1), 153-162. Marlin, J. W. (1991). State-mandated economic education, teacher attitudes, and student learning. Journal of Economic Education, 22(1), 5-14.

National Council on Economic Education. (1997). Voluntary national content standards in economics. New York: National Council on Economic Education.

Rhine, S. L. W. (1989). The effects of state mandates on student performance. American Economic Review, 79(2), 231-236.

Soper, J. C. and M. P. Lynn (1994). Should we mandate the teaching of economics? The Journal of Private Enterprise, 10(1), 65-79.

Swinton, J. R., T. W. De Berry, B. Scafidi and H. C. Woodard (2007). The impact of financial education workshops for teachers on students' economic achievement. Journal of Consumer Education, 27(1), 63-77.

Walstad, W. B. (2001). Economic education in U.S. high schools. Journal of Economic Education, 15(3), 195-210.

Walstad, W. B. and S. Buckles (2008). The national assessment of educational progress in economics: Findings for general economics. American Economic Review, 98(2), Forthcoming.

Walstad, W. B. and K. Rebeck (2001a). Test of economic literacy (Third Edition). New York: National Council on Economic Education.

Walstad, W. B. and K. Rebeck (2001b). Assessing the economic understanding of U.S. high school students. American Economic Review, 91(2), 452-457.

Paul W. Grimes, Mississippi State University

Meghan J. Millea, Mississippi State University

M. Kathleen Thomas, Mississippi State University Table 1: Mean Test of Economic Literacy Scores by Group (Paired Sample) Group Pre-Test Post-Test N Difference t-Value Mandate 18.29 19.27 75 0.98 0.52 (5.49) (7.29) (6.50) Non- 15.74 19.54 114 3.80 Mandate (7.64) (8.70) (7.07) 2.50 *** Course 17.49 22.71 77 5.22 (8.10) (8.17) (6.92) 6.62 *** Infused 12.08 12.95 37 0.865 0.807 (4.92) (5.52) (6.52) Full- 16.75 19.43 189 2.68 Sample (6.96) (8.15) (6.97) 5.29 *** ( ) -- Standard deviations *** Statistically significant at the .01 level, two-tailed test. Table 2: Specification of Variables Included in the Model Variable Empirical Specification Economic Understanding PRE TEL Student's pre-course score, as a percentage, on Test of Economic Literacy POST TEL Student's post-course score, as a percentage, on Test of Economic Literacy Student Demographics SEX Male = 1; Female = 0 AGE Student's age in years BLACK Student is black or other racial minority = 1; student is white = 0 SENIOR Student is in 12th grade = 1; otherwise = 0 HIGH INCOME Family income is greater than $50K = 1; otherwise = 0 MOTHER'S Mother holds college degree or higher = 1; EDUCATION otherwise = 0 Student Educational Attributes CALCULUS Student has taken pre-calculus course or above = 1; otherwise = 0 STUDY Student studies everyday = 1; otherwise = 0 CLUBS Number of extracurricular organizations to which student belongs SPORTS Participates in varsity or junior varsity sports = 1; otherwise = 0 HIGH GRADES Student earns mostly A's and B's = 1; otherwise = 0 STOCK MARKET Student had previously participated in the statewide Stock Market Simulation = 1; otherwise = 0 Student Market Experiences BANKING Student maintains individual bank account and credit card in his or her name = 1; otherwise = 0 WORK Student currently holds part-time job = 1; otherwise = 0 School Attributes PEERS Student's friends earn mostly A's and B's = 1; otherwise = 0 INFUSED Economics not taught as a stand-alone course = 1; otherwise = 0 MANDATE School district requires economics course for graduation = 1; otherwise = 0 Table 3: Means and Standard Deviations of Variables by Sample Group Variable Mandate Non-Mandate Full Sample Economic Understanding PRE TEL [+] 0.438 0.390 0.410 (0.144) (0.189) (0.173) POST TEL 0.479 0.488 0.484 (0.182) (0.217) (0.203) Student Demographics SEX [+] 0.453 0.475 0.466 (0.524) (0.501) (0.510) AGE [+] 17.179 16.667 16.874 (0.519) (0.709) (0.685) BLACK [-] 0.500 0.440 0.464 (0.503) (0.498) (0.499) SENIOR {+} 0.977 0.472 0.677 (0.152) (0.501) (0.468) HIGH INCOME [+] 0.430 0.544 0.498 (0.498) (0.500) (0.502) MOTHER'S 0.384 0.312 0.341 EDUCATION {+} (0.489) (0.465) (0.475) Student Educational Attributes CALCULUS [+] 0.151 0.152 0.152 (0.360) (0.360) (0.360) STUDY {+} 0.186 0.248 0.223 (0.391) (0.434) (0.417) CLUBS [+] 1.738 2.431 2.150 (1.883) (2.797) (2.485) SPORTS [-] 0.393 0.320 0.350 (0.491) (0.468) (0.478) HIGH GRADES [+] 0.570 0.640 0.611 (0.498) (0.482) (0.489) STOCK MARKET [+] 0.060 0.074 0.068 (0.238) (0.262) (0.252) Student Market Experiences BANKING [+] 0.116 0.176 0.152 (0.322) (0.382) (0.360) WORK {-} 0.667 0.541 0.592 (0.474) (0.500) (0.493) School Attributes PEERS [+] 0.605 0.592 0.597 (0.492) (0.493) (0.492) INFUSED [-] 0.00 0.328 0.194 (0.00) (0.471) (0.397) MANDATE [-] 1.000 0.000 0.408 (0.000) (0.000) (0.493) [ ] -- Expected sign of variable's coefficient in regression equation. { } -- Expected sign of variable's probit coefficient in selection equation. Table 4: Selection Equation: Probit Estimation Variable Probit Coefficient CONSTANT 0.124 (0.028) SEX 0.039 (0.126) AGE 0.049 (0.183) BLACK -0.240 (0.797) SENIOR 0.069 (0.160) MOTHER'S EDUCATION 0.683 ** (1.665) STUDY 0.058 (0.143) CLUBS 0.543 *** (2.891) SPORTS -0.192 (0.496) WORK -0.377 (1.092) PEERS 0.163 (0.531) % Correct Predictions 91.262 Restricted Log-likelihood -61.065 ( ) - Absolute value of t-statistic. *** Statistically significant at the .01 level, one-tailed test. ** Statistically significant at the .05 level, one-tailed test. Table 5: Regression Results: The Determinants of Post-Course Economic Understanding (Dependent Variable = POST TEL) Variable Regression Coefficient CONSTANT 0.291 (0.830) Economic Understanding PRE TEL 0.456 *** (6.022) Student Demographics SEX 0.023 (1.052) AGE 0.003 (0.163) BLACK -0.059 *** (2.572) HIGH INCOME 0.010 (0.415) Student Educational Attributes CALCULUS 0.057 ** (1.707) CLUBS 0.010 ** (1.354) SPORTS -0.051 ** (2.151) HIGH GRADES 0.046 * (1.619) STOCK MARKET 0.080 * (1.895) Student Market Experiences BANKING 0.008 (0.257) School Attributes PEERS -0.024 (0.908) INFUSED -0.177 *** (5.085) MANDATE -0.082 *** (3.310) Selection Term LAMBDA -0.050 (0.504) F-Statistic 12.280 Adjusted [R.sup.2] 0.475 ( ) - Absolute value of t-statistic. *** Statistically significant at the .01 level, one-tailed test. ** Statistically significant at the .05 level, one-tailed test. * Statistically significant at the .10 level, one-tailed test.