The impact of school vouchers on college enrollment.

Chingos, Matthew M. ; Peterson, Paul E.

In 1996, Cardinal John J. O'Connor, archbishop of New York, proposed to Rudy Crew, chancellor of the New York City public school system, that the city's most troubled public-school students be sent to Catholic schools, where he would see that they were given an education. New York City's mayor at that time, Rudolph Giuliani, a voucher supporter, attempted to secure public funds that would allow Catholic schools to fulfill the cardinal's offer. But voucher opponents condemned the idea on the grounds that it violated the no establishment of religion clause of the First Amendment. It was only several years later, in 2002, that the U.S. Supreme Court found vouchers constitutional.

As the controversy raged in the late 1990s, a group of philanthropists created the New York School Choice Scholarships Foundation (SCSF), which offered three-year vouchers worth up to $1,400 annually to as many as 1,000 low-income families with children who were either entering 1st grade or were public school students about to enter grades two through five. Due to excess demand, SCSF established a lottery for interested families. If a family met the eligibility criteria and won the SCSF lottery, all of that family's children entering grades one through five would receive a voucher. Recipients could attend any one of the hundreds of participating private schools, religious or secular, within New York City.

According to the Archdiocese of New York, average tuition in the city's Catholic schools, the city's largest private provider, was $1,728, which was 72 percent of the total per-pupil cost of $2,400 to educate a child at these schools. The scholarship would thus cover only a portion of the costs of the private education of eligible students. SCSF initially committed to making the scholarships available for a period of three years.

SCSF asked an independent research team to conduct an experimental evaluation of the impact of the intervention on student achievement and other outcomes, such as school climate and school quality, as reported by the students' parents or other guardians. More than 20,000 students expressed interest in a voucher and were invited to one of five separate eligibility verification and testing sessions. To participate in the lottery, students other than those who had yet to begin 1st grade were required to take a standardized test. While students were taking the test, the adult accompanying the child answered questions about the child's family background and the current school the child attended. All families were asked to supply identifying information for each child applying for a scholarship, including full name and date of birth.

Families who won the voucher lottery were told that scholarship renewal was dependent on participation in annual testing at a designated site other than the child's school. Families who lost the lottery were compensated for participating in subsequent testing sessions, and their children were given additional chances to win the lottery. Those who won a subsequent lottery were dropped from the evaluation control group. Those families who won the lottery but who did not make use of the scholarship were also compensated for participating in subsequent testing sessions. The original evaluation identified, after three years, large positive effects of the voucher opportunity on the test scores of African Americans but not on the test scores of students from other ethnic groups.

In this paper, we extend the original evaluation of the SCSF program by estimating impacts of the offer of a voucher on college enrollment. Our results provide the first experimental evidence of the effects of a voucher intervention on this outcome. The study is also notable for obtaining information on college enrollments for 99 percent of study participants, greatly reducing the potential for bias due to attrition from the evaluation. We find large positive impacts on college enrollment for African American students but not for Hispanic students. Impact data for the small group of students from other backgrounds are too noisy to produce reliable evidence.

Evidence on College Enrollment

Few experimental evaluations have estimated the long-term impacts of interventions taking place during the regular years of schooling. Public school choice for disadvantaged students in the Charlotte-Mecklenburg school district in North Carolina was shown to reduce incarceration rates, especially among high-risk students (see "Does School Choice Reduce Crime?" research, Spring 2012). Another study found that class-size reduction in Tennessee's K-3 classrooms increased college enrollment rates by about 6 percentage points among African American students, although no impacts were observed for white students.

The scarcity of experimental studies of long-term outcomes is especially true when it comes to school voucher research. One recent study using data from Washington, D.C., did identify positive impacts of a voucher program on high-school graduation rates. No studies have yet reported impacts on college enrollment, due in part to the challenges of following students long enough and obtaining accurate information on their postsecondary careers.

Fortunately, almost all colleges and universities in the United States, representing more than 96 percent of all college students, now submit enrollment information to the National Student Clearinghouse (NSC). We used the names and dates of birth of SCSF scholarship applicants, collected at eligibility verification sessions, to match them to NSC records. The information needed to make this match was available for 2,637 of the 2,666 students in the original sample.

Methods

Our primary outcome of interest is the overall (part-time and full-time) college enrollment within three years of expected (i.e., on-time) high-school graduation. We focus on this three-year window (the exact dates of which vary according to the student's grade when enrolling in the study) because the most recent enrollment data available are for fall 2011 and the youngest cohort was expected to graduate high school in 2009. We also report the effects of the voucher offer on full-time enrollment; enrollment in four-year colleges; enrollment in private colleges; and enrollment in selective colleges.

We identify students as not having enrolled in college if they are not matched to any NSC records. Some measurement error of college enrollment is possible. For example, a student who enrolled in college but whose birth date was incorrect in our records would be counted as a nonenrollee. This type of measurement error is unlikely to bias our estimates because there is no reason to believe it is related to whether a student won the school-choice lottery. Our results could be biased, however, if being offered a voucher affected enrollment in the small share of colleges that do not participate in the NSC.

We estimate the effects on college enrollment of simply being offered a voucher, even if it is not used to enroll in a private school, as well as the effects of actual voucher use. The effect of the voucher offer is referred to as an intent-to-treat (ITT) estimate, as offering a voucher to a family is an attempt by SCSF to induce the family to make use of a private school. The ITT effect includes both the effect of voucher use for those who used it and any effects on those who were offered the voucher but declined. The impact of actually using the voucher is referred to as a treatment-on-treated (TOT) estimate, as it identifies the effects on those actually treated, that is, those who used the voucher to attend a private school. The TOT analysis assumes that winning the lottery had no impact on college enrollment among students who never used a voucher.

[ILLUSTRATION OMITTED]

Of the 2,637 students included in our analysis, 1,358 students won the lottery and were therefore assigned to the treatment group. The remaining 1,279 students were assigned to the control group. All the students who applied for a voucher were socioeconomically disadvantaged, as only low-income families were eligible to participate. Nearly half of the students came from families in which neither parent had attended college. The vast majority of students were African American or Hispanic; the performance of the average student when tested before names were entered into the lottery was between the 17th and 25th percentile of students nationwide.

Although African American and Hispanic students had fairly similar scores on the baseline achievement test, students in these groups differed in a number of respects. While 42 percent of all students in the control group enrolled in college within three years of expected high-school graduation, only 36 percent of African American students in the control group did so, compared to 45 percent of Hispanic students. African American students in the treatment and control groups were more likely than Hispanic students to be male and more likely to have a parent with a college education. They were also more likely to come from one-child families and from families with four or more children.

As would be expected in a randomized experiment, students in the treatment and control groups--both overall and across African Americans and Hispanics--had similar characteristics on average. In other words, it appears that the randomization worked in producing groups of students that were comparable before the intervention began.

Most, but not all, students offered a voucher used it at some point. The share of lottery winners using the scholarship they were offered declined from 74 percent in the first year after the initial offer to 55 percent in the third year. Over the first three years after the initial offer, the average member of the treatment group used a scholarship for 1.9 years. Among students who used the scholarship for any of the first three years, the average length of time a scholarship was used was 2.5 years. SCSF later extended its initial three-year commitment and, over all of the years observed in our data, the average member of the treatment group used a scholarship for 2.6 years. Among those who used the scholarship for at least 1 year, the average is 3.4 years. Scholarship usage patterns did not vary notably between African American and Hispanic students.

Results

We find that the offer of a voucher increased college enrollment within three years of the student's expected graduation from high school by 0.7 percentage points, an insignificant impact. This finding, however, masks substantial variation in impacts among students from different ethnic groups. We find evidence of large, statistically significant impacts on African Americans, but fairly small and statistically insignificant impacts on Hispanic students. We discuss results for the small number of students from other groups below.

The SCSF-NSC linked data indicate that a voucher offer increased the college-enrollment rate of African Americans by 7 percentage points, an increase of 20 percent. If an African American student used the scholarship to attend private school for any amount of time, the estimated impact on college enrollment was 9 percentage points, a 24 percent increase over the college enrollment rate among comparable African American students assigned to the control group (see Figure 1). This corresponds to 3 percentage points for every year the voucher was used.

Voucher Impact (Figure 1)

Using a voucher to attend private school boosted the college-going
rates of African American, but not Hispanic students.

African American

                    Voucher User  Non-recipient

Any College                45 **             36
Full-Time                  34 **             26
Four-Year                  30 **             24
Private, Four-Year          16 *              9
Selective                   8 **              3

Hispanic

                    Voucher User  Non-recipient

Any College                   47             45
Full-Time                     34             34
Four-Year                     28             30
Private, Four-Year            11             12
Selective                      8              8

* (**) Indicates the estimated impact of using a voucher Is
statistically significant at the 90 (95) percent confidence level.

NOTE: College enrollment rates for voucher users were calculated by
adding the estimated effect of using a voucher to the enrollment
rates observed among students In the control group.

SOURCE: Author' calculations

Note: Table made from bar graph.

The impact of a voucher offer on the college-enrollment rate of Hispanic students is a statistically insignificant 2 percentage points. Although that estimate is much smaller than the one observed for African Americans, the impacts on the two ethnic groups are not significantly different from one another.

We obtain similar results for full-time college enrollment. Among African Americans, 26 percent of the control group attended college full-time at some point within three years of expected high-school graduation. The impact of a voucher offer was to increase this rate by 7 percentage points, a 25 percent increment. Among students using the voucher to attend a private school, the estimated impact was 8 percentage points, or roughly 31 percent. No statistically significant impact on full-time college enrollment was evident for Hispanic students.

Only 9 percent of the African American students in the control group attended a private four-year college. The offer of a voucher raised that proportion by 5 percentage points, an increase of 58 percent. That extraordinary increment may reflect the tight connections between private elementary and secondary schools and private institutions of higher education.

The percentage of African American students in the control group who attended a selective four-year college was 3 percent. That increased by 4 percentage points if the student received the offer of a voucher, a better than 100 percent increment in the percentage enrolled in a selective college, a very large increment from a very low baseline. Once again, no impacts were detected for Hispanic students.

Explaining Group Differences

The estimated impact of the voucher offer on college enrollment was roughly 5 percentage points greater for African American students than for Hispanic students, raising the question of why such a difference is observed between these two groups, both of which came from socioeconomically disadvantaged families.

We do not know for sure why larger impacts were observed for African American students than for Hispanic students, but it appears that the African American students in the study had fewer educational opportunities in the absence of a voucher. As noted above, Hispanic students were considerably more likely to attend college in the absence of a voucher opportunity. There is also some evidence that the public schools attended by Hispanic students were superior to those attended by African American students. When asked to rate the overall quality of the child's school at baseline, the parents of Hispanic students gave an average rating of 2.63 (on a 4-point, GPA-type scale), compared to the 2.29 rating given by the parents of African Americans.

Given this disparity, it is not surprising that the impact of a voucher offer on school quality (as reported by parents) was generally larger for African American students than it was for Hispanic students. Survey data from the first-year follow-up indicate that a voucher offer reduced the number of reported problems at the school attended by 1.1 (out of 6 problems listed) for African Americans but by only 0.5 problems for Hispanics. Also, Hispanic parents in the control group continued to rate their children's schools more favorably than did African American parents. All in all, it seems that the voucher option was less critical for Hispanic students than for African American students.

A possible alternative explanation focuses on motivations for moving from public to private school. Many Hispanic families may have been seeking a voucher opportunity for religious reasons, while most African American families had secular education objectives in mind. Eighty-five percent of Hispanic students were Catholic, the same religion as that of the most extensive network of private schools in New York City. Only 19 percent of African American families said their religious affiliation was Catholic. Sixty-five percent said they were Protestant, but there are very few Protestant and other non-Catholic religious schools in New York City.

Further, 71 percent of the Hispanic respondents said they attended religious services weekly, while only 47 percent of African American respondents said they did. When treatment-group parents with children in private schools were asked in the third-year follow-up study which type of school their child was attending, 93 percent of Hispanic respondents said it was a Catholic school and 71 percent of the African American respondents gave the same response. In that same follow-up survey, 39 percent of the Hispanic respondents listed religious considerations as one of the reasons they had sought a scholarship, compared to just 33 percent of African American respondents (though this last difference is not statistically significant).

The small group of students in the study from other ethnic backgrounds was diverse and less likely to use the voucher when it was offered to them, so we are hesitant to interpret their results. The group consists of 196 treatment and 127 control students, including 91 white students, 14 Asian students, 78 students from another background, and 140 students for whom information on ethnicity was not supplied. For this group as a whole, the estimated impact of the voucher offer on college enrollment within three years of expected graduation has a negative sign but is imprecisely estimated.

If we separate out white or Asian students, other-race students, and those for whom information on race is unavailable, the estimated effects of the voucher offer are all negative, but only the effect for white or Asian students is statistically significant. This group includes only 105 students, however, and we find that the treatment and control groups did not have similar characteristics at the beginning of the study. Consequently, we do not place much weight on this negative effect.

Conclusions

The magnitude of the voucher impact on African American students may seem unexpectedly large given the modest nature of the intervention: a partial-tuition scholarship of no more than $1,400 annually. Among all those offered a voucher, the average length of time a voucher was used was less than three years.

The impact is not substantially greater than that observed in other studies, however. Using a similar definition of scholarship use (receipt of any scholarship assistance), the evaluators of the federally funded Washington, D.C., voucher program estimated a positive impact of 21 percent on the high-school graduation rates of study participants, 88 percent of whom were African Americans. That is just short of the 24 percent impact on college-going for the New York City African American students in our study.

The impacts on college enrollment we estimate are somewhat larger than those of the much more costly class-size intervention in Tennessee. Susan Dynarski and her colleagues find that being assigned to a smaller class in the early elementary grades increased college enrollment rates among African Americans by 19 percent (6 percentage points on a base of 31 percent). Reduction of class size in Tennessee cost roughly $12,000 per student, whereas the SCSF voucher intervention cost the foundation about $4,200 per student, but reduced costs to the taxpayer by lowering the number of students who required instruction in public schools. Had the government paid for the voucher, the expenditure could have taken the form of a simple transfer from the public sector to the private sector, which in the long run need not add to the per-pupil cost of education.

[ILLUSTRATION OMITTED]

The impact of the voucher offer we observe for African American students is also much larger than the impact of exposure to a highly effective teacher. Raj Chetty and his colleagues (see "Great Teaching," research, Summer 2012) report that being assigned to an elementary school teacher who is 1 standard deviation more effective than the average teacher boosted college enrollment for students in a very large city by 0.5 percentage points at age 20, relative to a base of 38 percent, an increment of 1.25 percent. If one extrapolates that finding (as those researchers do not) to three years of highly effective teaching, the impact is 3.75 percent. The 24 percent impact we identify for African American students is many times as large.

The reader should be cautioned, however, that the results from any experiment cannot be easily generalized to other settings. For example, scaling up voucher programs would surely change the social composition of private schools. To the extent that student learning depends on peer characteristics, the impacts reported here could change. But the results of this investigation nonetheless advance our understanding of the effects of school choice policies by providing the first experimentally generated information on the long-term impact of a voucher intervention.

African Americans benefit the most

Matthew M. Chingos is a fellow in the Brookings Institution's Brown Center on Education Policy. Paul E. Peterson is professor of government and director of the Program on Education Policy and Governance at Harvard University and senior fellow at the Hoover Institution at Stanford University.