A double dose of algebra: intensive math instruction has long-term benefits.

Cortes, Kalena ; Goodman, Joshua ; Nomi, Takako 等

In 2008, president-elect Barack Obama declared that preparing the nation for the 1121st-century economy" required making "math and science education a national priority." He later signed legislation that provided incentives for states to adopt common standards intended to increase curricular rigor in these and other subjects. Encouraging more students to take advanced classes seems laudable, but concerns have arisen about the ability of many students to complete such course work successfully.

Students in urban high schools are of particular concern. Populated predominantly by low-income and minority students, these schools struggle with two related problems. First, many students do not earn passing grades in early courses that are thought to be prerequisites for more-advanced subjects. Second, students are at high risk of failing to earn their high school diplomas at all. In fact, only 65 percent of black and Hispanic students graduate high school, with little evidence that the graduation gap between them and white students has changed in the last few decades. One theory for these low highschool completion rates is that failures in early courses, such as algebra, interfere with subsequent course work, placing students on a path that makes graduation quite difficult.

One increasingly popular approach to improving students' math skills is "algebra for all," which encourages more students to take algebra and at earlier ages. The best study of this approach, using evidence from Charlotte, North Carolina (see "Solving America's Math Problem," features, Winter 2013), shows that pushing students into course work for which they are ill prepared actually harms their subsequent academic achievement. A potentially promising alternative, and one we focus on here, is "double-dose" algebra, in which struggling students are given twice as much instructional time as they would normally receive. The best study of this approach, by Takako Nomi and Elaine Allensworth, examined the short-term impact of such a policy in the Chicago Public Schools (CPS), where double-dose algebra was implemented in 2003. Under that policy, students scoring below the national median on the 8th-grade math exam were required to take two periods of algebra a day during 9th grade instead of one, with the second class providing support and extra practice. Students placed in the extra classes thus received substantially more algebra instruction than other students. Nomi and Allensworth reported no improvement in 9th-grade algebra failure rates as a result of this intervention, a disappointing result for CPS. The time frame of their study did not, however, allow them to explore longer-run outcomes of even greater importance to students, parents, and policymakers.

Our study extends this work to examine the impact of CPS's double-dose algebra policy on such longer-run outcomes as advanced math course work and performance, ACT scores, high-school graduation rates, and college enrollment rates. Using data that track students from 8th grade through college enrollment, we analyze the effect of this innovative policy by comparing the outcomes for students just above and just below the double-dose threshold. These two groups of students are nearly identical in terms of academic skills and other characteristics, but differ in the extent to which they were exposed to this new approach to algebra. Comparing the two groups thus provides unusually rigorous evidence on the policy's impact.

We find positive and substantial longer-run impacts of double-dose algebra on college entrance exam scores, high school graduation rates, and college enrollment rates, suggesting that the policy had significant benefits that were not easily observable in the first couple of years of its existence. The benefits of double-dose algebra were largest for students with decent math skills but below-average reading skills, perhaps because the intervention focused on written expression of mathematical concepts.

Double-Dose Algebra

Since the late 1990s, Chicago Public Schools has been attempting to increase the rigor of student course work and prepare students for college entrance. Starting with students entering high school in the fall of 1997, CPS eliminated lower-level and remedial courses so that all first-time freshmen would enroll in algebra in 9th grade, geometry in 10th grade, and algebra II or trigonometry in 11th grade. Soon after, it became apparent to CPS officials that many students were unable to master the new curriculum, resulting in very low passing rates in 9th-grade algebra. The cause of this high failure rate was thought to lie largely with the poor math skills with which students entered high school.

In response to the low passing rates, CPS launched the double-dose algebra policy for all students entering high school in the fall of 2003. Instead of reinstating the traditional remedial courses from previous years, CPS required enrollment in two periods of algebra for all first-time 9th graders testing below the national median on the math portion of the 8th-grade Iowa Tests of Basic Skills (ITBS). Students enrolled for two math credits, a full-year regular algebra class plus a full-year algebra support class. Prior to the double-dose policy, algebra curricula varied considerably across CPS high schools, due to the decentralized nature of the district. With the new policy, CPS offered teachers of double-dose algebra two specific curricula called Agile Mind and Cognitive Tutor, standalone lesson plans they could use, and three professional development workshops each year, where teachers were given suggestions about how to take advantage of the extra instructional time.

Though it is difficult to know precisely what occurred in these extra classes, students assigned to double-dose algebra reported more frequently writing sentences to show how they solved a math problem; explaining how they solved a problem to the class; writing math problems for other students to figure out; discussing possible solutions with other students; and applying math to situations in life outside of school. The additional time spent building verbal and analytical skills may have conferred benefits in subjects other than math.

CPS also strongly advised schools to schedule their algebra support courses in three specific ways. First, double-dose algebra students should have the same teacher for their two periods of algebra. Second, the two algebra periods should be offered consecutively. Third, double-dose students should take the algebra support class with the same students who are in their regular algebra class. Most schools followed these recommendations in the initial year. In the second year, schools began to object to the scheduling difficulties of assigning the same teacher to both periods, so CPS removed that recommendation.

The policy we study had many components. Assignment to double-dose algebra doubled the amount of instructional time and exposed students to the curricula and activities discussed above. The recommendation that students take the two classes with the same set of peers increased tracking by skill level. All of these factors were likely to, if anything, improve student outcomes. We will also show, however, that the increased tracking by skill placed double-dose students among substantially lower-skilled classmates than non-double-dose students, which could have hurt student outcomes. Our results will capture the net impact of all of these factors.

Data

Our analysis focuses on the first two cohorts of students subject to the double-dose algebra policy, those entering high school in the fall of 2003 and in the fall of 2004. These two cohorts included more than 41,000 students overall. Our primary results are based on the 11,507 students with 8th-grade math test scores within 10 percentile points of the cutoff used to assign students to double-dose algebra.

As discussed above, the implementation of the policy differed somewhat between the two cohorts. For the 2003 cohort, 80 percent of double-dose students had the same teacher for both courses, 72 percent took the two courses consecutively, and rates of overlap between the two classes' rosters exceeded 90 percent. For the 2004 cohort, only 54 percent of double-dose students had the same teacher for both courses, and only 48 percent took the two courses consecutively. Overlap between the rosters remained, however, close to 90 percent. We find that the policy's impacts were similar for the two cohorts, so we combine them for the purpose of presenting our results.

Longitudinal data from CPS enable us to track students from 8th grade through college enrollment. These data include demographic information, detailed high-school transcripts, numerous standardized test scores, and graduation and college enrollment information. We include in our analysis all students who entered 9th grade for the first time in the fall of 2003 or 2004, who had valid 8th-grade math test scores, and who enrolled in freshman algebra in a high school in which at least one classroom of students was assigned to double-dose algebra. These CPS students were primarily from disadvantaged socioeconomic backgrounds and minority groups. About 90 percent were black or Hispanic, and 20 percent received special education services. The average CPS student scored at approximately the 45th percentile (a little below average) on the nationally normed exam used to determine which students were required to take double-dose algebra.

We focus on two sets of student outcomes. The first set measures students' academic achievement and includes grades, course work, and standardized test scores. For example, our data include pass rates for algebra and higher-level math courses and students' scores on the ACT (a college-entrance exam). The second set captures educational attainment, including high school graduation and college enrollment rates. We consider students to be high school graduates if they received a regular CPS diploma within five years of starting high school. About 50 percent of CPS students in our data graduated high school within four years, with another 5 percent graduating in the fifth year. College enrollment is measured using data on CPS high-school graduates from the National Student Clearinghouse. We count students as college matriculants if they enrolled in college by October 1 of the fifth year after starting high school. Only 28 percent of students in our data both graduated from a CPS high school and enrolled in college within this time frame. Of these college matriculants, 46 percent enrolled in two-year colleges and 54 percent enrolled in four-year colleges.

Method

Given the substantial differences between students who were and were not assigned to double-dose algebra, simply comparing their later outcomes would likely produce misleading evidence on the policy's impact. To eliminate this bias, we take advantage of the fact that students scoring below the 50th percentile on the 8th-grade ITBS math test were supposed to enroll in double-dose algebra. This rule allows us to isolate the impact of double-dose algebra by comparing the outcomes of students who scored just below the cutoff to those who scored just above the cutoff. These two groups of students were very similar--their scores differed by a tiny amount--but only one group was required to take double-dose algebra. And there were no differences in the outcomes of students scoring just below and above the assignment cutoff among earlier cohorts of CPS students, suggesting that any differences we identify can be attributed to the policy.

Overall, 55 percent of CPS students scored below the 50th percentile and thus should have been assigned to double-dose algebra, but only 42 percent were actually assigned to the support class. In addition, some students took double-dose algebra, even though they scored above the cutoff on the exam. As a result, the difference in double-dose assignment rates between students just below the cutoff and students just above the cutoff was about 40 percentage points--not 100 percentage points, as would be the case if the rule were followed without exceptions. We adjust our results to reflect the fact that some above-threshold students were double-dosed and some who scored below the threshold were not. Our results should therefore be interpreted as the effect of taking double-dose algebra among students who took it because they scored below the cutoff--not for some other reason, such as a desire for additional instruction in math.

Before turning to our results on student outcomes, we first examine how the double-dose algebra policy changed students' freshman-year experiences. Students assigned to double-dose algebra obviously spent more class periods learning math. The second class did not replace core academic courses in English, social studies, or science; it did replace other courses, such as fine arts and foreign languages. The net result was a small increase in the total number of courses taken freshman year.

Students in the double-dose classes were also more likely to be grouped with classmates of similar academic skill. Assignment to double-dose decreased the average achievement of a student's classmates by more than 19 percentile points, and increased the size of regular algebra classes by 2.4 students. Doubling instructional time in math by replacing other course work thus increased tracking by ability and class size.

Results

Double-dosing had an immediate impact on student performance in algebra, increasing the proportion of students earning at least a B by 9.4 percentage points, or more than 65 percent. It did not have a significant impact on passing rates in 9th-grade algebra, however, or in geometry (usually taken the next year). Double-dosed students were, however, substantially more likely to pass trigonometry, a course typically taken in 11th grade. The mean GPA across all math courses taken after freshman year increased by 0.14 grade points on a 4.0 scale.

As a whole, these results imply that the double-dose policy greatly improved freshman algebra grades for the higher-achieving double-dosed students, but had relatively little impact on passing rates for the lower-achieving students. The latter fact is one of the primary reasons that CPS has continued to refine its algebra instruction policy. There is, however, some evidence of improved passing rates and GPAs in later math courses, suggesting the possibility of benefits beyond 9th grade.

Though course work and grades matter for students' academic trajectories, the subjective nature of course grading suggests that standardized tests may be a better measure of the impact of double-dosing on math skill. We do not find consistent evidence of impacts on student performance on the preliminary ACT (called PLAN) exam taken in the fall of 10th grade, but we do find impacts on the 11th-grade PLAN (see Figure 1). On that exam, double-dosing was found to increase algebra scores by 0.15 standard deviations and overall math scores by 0.16 standard deviations. Perhaps more importantly, a nearly identical effect is seen on the math portion of the ACT (taken in the spring of 11th grade), with double-dose algebra raising scores by 0.15 standard deviations on an exam used by many colleges as part of the admissions process. This is equivalent to closing roughly 15 percent of the black-white gap in ACT scores.

Test-Score Boost (Figure 1)

Students who doubled up on algebra had higher scores on standardized
tests taken after 10th grade.

Effect of double-dose algebra on standardized test scores in math,
algebra, and geometry Standard deviations]

Standard Fall 10th Fall 11th Spring 11th
deviations grade grade grade

PLAN Math 0.09 0.16 * 0.15 *

PLAN Algebra 0.09 0.15 *

PLAN Geometry 0.01 0.10

* indicates statistical significance at the.05 level

NOTE: PLAN is a test students take in 10th grade in preparation for
taking the ACT college-entrance exam the following year. SOURCE:
Authors' calculations based on Chicago Public Schools data

Note: Table made from bar graph.

These results from standardized tests suggest that double-dosed students experienced few, if any, short-run achievement gains but did experience larger gains that persisted at least two years after the end of double-dose classes. Did these gains translate into improved educational attainment? We find that they did, with double-dosing increasing four- and five-year high-school graduation rates by 8.7 and 7.9 percentage points, respectively, a 17 percent improvement (see Figure 2).

Figure 2 also shows that double-dosed students were 8.6 percentage points more likely to enroll in college within five years of starting high school, a nearly 30 percent increase over the base college enrollment rate of 29 percent. Nearly all of this increase comes from enrollment in two-year colleges, with more than half of that resulting from part-time enrollment in such colleges. Given the relatively low academic skills and high poverty rates of double-dosed CPS students, it is unsurprising that double-dosing improved college enrollment rates at relatively inexpensive and nonselective two-year postsecondary institutions.

High Impact (Figure 2)

Double-dose algebra increased the percentage of students who
graduated from high school and of those who enrolled in college,
with most choosing two-year institutions.

Effect of double-dose algebra on high school graduation and
college-enrollment rates

 Percentage points

Graduated within 5 years 7.9 *

Enrolled in college 8.6 *

Enrolled in two-year college 7.9 *

* indicates statistical significance at the .05 level

SOURCE: Authors calculations based on Chicago Public Schools data
and National Student Clearinghouse data

Note: Table made from bar graph.

It is important to note that many of these results are much stronger for students with weaker reading skills, as measured by their 8th -grade reading scores. For example, double-dosing raised the ACT scores of students with below-average reading scores by 0.22 standard deviations but raised above-average readers' ACT scores by only 0.09 standard deviations. The overall impact of double-dosing on college enrollment is almost entirely due to its 13-percentage-point impact on below-average readers (see Figure 3). This unexpected pattern may reflect the intervention's focus on reading and writing skills in the context of learning algebra.

Reading and Writing in Algebra (Figure 3)

Students with weak reading skills benefited more from the algebra
support class than otherwise similar students, perhaps because
reading and writing were central to the instructional model.

Effect of double-dose algebra on high school graduation and
college enrollment rates, by reading skill level

Percentage Points Graduated within five years Enrolled in college

Below-median reader 12.7 * 13.3 *

Above-median reader 3.1 3.9

* indicates statistical significance at the .05 level

SOURCE: Authors calculations based on Chicago Public Schools
data and National Student Clearinghouse data

Note: Table made from bar graph.

Finally, we consider the possibility that the increased focus on algebra at the expense of other course work may have affected achievement in other subjects. We find strong evidence that rather than harming achievement in reading and science, double-dosing had positive effects across the board. Double-dosed students scored nearly 0.20 standard deviations higher on the verbal portion of the ACT, were substantially more likely to pass chemistry classes usually taken in 10th or 11th grade, and earned modestly higher GPAs across all of their nonmath classes in the years after 9th grade. In other words, the skills gained in double-dose algebra seem to have helped students in other subjects and in subsequent years.

Conclusion

Our study provides the first evidence of positive and substantial long-run impacts of intensive math instruction on college entrance exam scores, high school graduation rates, and college enrollment rates. We also show that the intervention was most successful for students with relatively high math skills but relatively low reading skills. Although the intervention was not particularly effective for the average affected student, the fact that it improved high school graduation and college enrollment rates for even a subset of low-performing and at-risk students is extraordinarily promising when targeted at the appropriate students. In this case, those were students with only moderately low math skills but below-average reading skills.

This double-dose strategy has become an increasingly popular way to aid students struggling in mathematics. Today, nearly half of large urban districts in the United States report double math instruction as the most common form of support for students with lower skills. The central concern of urban school districts is that algebra may be a gateway for later academic success, so early high-school failure in math may have large effects on subsequent academic achievement and graduation rates. With the current policy environment calling for "algebra for all" in 9th grade or earlier, effective and proactive intervention is particularly critical for those who lack foundational mathematical skills. A successful early intervention may be the best way to boost students' long-term academic success.

Kalena Cortes is assistant professor at the Bush School of Government and Public Service at Texas A&M University. Joshua Goodman is assistant professor of public policy at the Harvard Kennedy School. Takako Nomi is assistant professor of education at St. Louis University.