Taking Stock of PRIVATE-SCHOOL CHOICE.

Wolf, Patrick J. ; Harris, Douglas N. ; Berends, Mark 等

In the past few years, four states have established programs that provide public financial support to students who choose to attend a private school. These programs-a tax-credit-funded scholarship initiative in Florida and voucher programs in Indiana, Louisiana, and Ohio-offer a glimpse of what expansive statewide choice might look like. What have we learned about the students and schools who choose to participate in these programs? What academic outcomes have students reaped? And what does research tell us about how states should design and oversee voucher programs-if indeed they should do so at all? In this forum, we hear from Patrick J. Wolf, education policy professor at the University of Arkansas, Douglas N. Harris, professor of economics at Tulane, and the trio of Mark Berends, professor of sociology at the University of Notre Dame, R. Joseph Waddington, assistant professor at the College of Education, University of Kentucky, and Megan Austin, researcher at the American Institutes for Research, Chicago.

SCHOOL VOUCHER PROGRAMS, which allow eligible families to send their children to private schools with the help of public funds, have sparked controversy since the first such initiative was launched in Milwaukee in 1991. Today, 28 states and the District of Columbia (D.C.) operate 54 private-school-choice programs, which include not only government-issued vouchers but also tax-credit scholarships, education savings accounts (ESAs), and town-tuitioning programs for rural families. In spite of, or perhaps because of, the spread of such programs across the country, the debate surrounding their merits continues. Fortunately, many studies on the outcomes of private-school-choice initiatives have enabled us to begin evaluating their effectiveness. While the jury is still out on the effects of these programs on student test scores, there is significant evidence that they positively influence how far students continue in their schooling.

Who Participates?

Private-school-choice programs disproportionately attract students from disadvantaged backgrounds. Choice participants are considerably more likely to be low-income, lower-achieving, and African American, and much less likely to be white, as compared to the average public-school student in their area. Moreover, 12 percent of the 446,000 participants in private-school-choice programs in 2016-17 were in initiatives limited to students with disabilities, which is slightly higher than the 11 percent average rate of student disability in public schools nationally.

These participation trends are not surprising, since most voucher programs are targeted to low-income urban students or students with disabilities. Even in the Indiana Choice

Scholarship Program, however, which is open to low- and middle-income families statewide, the percentage of low-income students enrolled is slightly higher than their percentage of the overall K-12 population.

The private schools that participate in choice programs also are distinctive. Yujie Sude, Corey DeAngelis, and I examined the patterns of private-school participation in choice programs in D.C., Florida, and Louisiana. Private schools were more likely to participate if the gap between their tuition level and the usually lower voucher amount was smaller, if they already had experience serving disadvantaged students, and if they were Catholic schools. Stringent regulations appear to dissuade some schools from opting in: more than two thirds of private schools participate in the relatively low-regulation Florida Tax Credit Scholarship Program, while only one third participate in the relatively high-regulation Louisiana Scholarship Program. Brian Kisida, Evan Rhinesmith, and I surveyed private-school leaders who declined to participate in their state's choice program and found that most of them feared that regulations would increase in the future. They also viewed certain regulations as restricting their independence and organizational identity, especially mandates involving curriculum and requirements to administer the state accountability test to their choice students.

Douglas N. Harris, in his contribution to this forum, states that even religious schools willing to participate in school voucher programs "often have academic and behavioral admissions requirements." Only three voucher programs--those in D.C., Indiana, and Ohio--permit participating private schools to apply test-based admissions standards to applicants using vouchers. None of the U.S. voucher programs permit schools to deny admission to students based on their disciplinary records. Private schools can decline to participate in voucher programs, but if they agree to serve students on vouchers, in most cases they must accept all comers.

Voucher Effects

The effects of private-school-choice programs on the achievement of student participants have been extensively studied using a variety of research designs. Sixteen evaluations of eight programs in Charlotte, North Carolina; Dayton and Toledo, Ohio; D.C.; Louisiana; Milwaukee, Wisconsin; and New York City used "gold standard" experimental designs. Danish Shakeel, Kaitlin Anderson, and I conducted a meta-analysis of the 16 experimental studies, finding that the private-school-choice programs evaluated in the United States have increased student achievement by an average of 0.13 standard deviations in reading by the fourth year after the study started. In total, programs have had no significant effect on average math scores. The reading effect represents a gain of about four months of learning, depending on student grade level and background. The achievement effects from school-choice experiments follow a consistent pattern. They begin slightly negative, then turn positive and cumulate over time (see Figure 1).

Four recent non-experimental studies of choice programs also tended to report positive effects in reading achievement, with some qualifications. David Figlio concluded that the Florida Tax Credit Scholarship program had a statistically significant positive effect on reading outcomes for students close to the program's income eligibility cutoff. Mark Berends and colleagues, as reported in their essay for this forum, found that students who persisted in the Indiana Choice Scholarship Program for four years experienced reading gains. I led a research team that concluded that the combination of access to school vouchers and a high-stakes-testing policy boosted the reading test scores of students in the Milwaukee Parental Choice Program. An evaluation of the Ohio EdChoice Scholarship Program, conducted by David Figlio and Krzysztof Karbownik, was an exception, reporting negative effects of that voucher program on both reading and math scores that persisted over time.

That study, while reporting negative achievement effects for participants in Ohio's largest voucher program, also found that students remaining in public schools performed higher on tests, owing to program-induced competition. Their study is the 15th evaluation of the competitive effects of vouchers to report consistently positive results. Six other such studies reported that competitive pressure from vouchers had effects that ranged from neutral to positive. Only one study, conducted by Jay Greene and Marcus Winters and focusing on the D.C. voucher program, found that voucher competition had no effect on the test scores of non-participants, while no empirical study of acceptable rigor has found that a U.S. private-school-choice program decreased the achievement of public school students.

The effects of choice programs on educational attainment--how far an individual goes in school--are both larger and more consistent than their achievement effects. Attainment is typically measured by benchmarks such as high school graduation, college enrollment, persistence in college, and college graduation. Higher levels of educational attainment are associated with a longer, healthier life; higher lifetime earnings; and lower probabilities of divorce, welfare receipt, and incarceration.

Fewer choice studies have examined attainment than achievement because doing so requires tracking students for many years. The five studies undertaken so far all report positive effects of private-school-choice on attainment for all participants or key subgroups, and these effects are both statistically significant and substantively large. An experimental study I led for the U.S. Department of Education of the D.C. Opportunity Scholarship Program found that using a voucher increased the likelihood of high-school graduation by 21 percentage points, representing a 30 percent boost. In a similarly rigorous experimental evaluation, Matthew Chingos and Paul Peterson reported that participating in the New York City private-school scholarship program increased college enrollment rates for African American and Hispanic students by 6 percentage points, which represented a 10 percent hike. The program also increased those students' college-graduation rates by 3.5 percentage points, an increment of 35 percent. In a non-experimental analysis, Chingos and Daniel Kuehn found that participation in the Florida Tax Credit Scholarship Program increased the student rate of college enrollment by 15-43 percent, depending on how many years the individual used a scholarship. Two non-experimental studies of the Milwaukee Parental Choice Program reported that it increased high-school graduation rates, but by smaller amounts than the programs in D.C., New York, and Florida. One of those studies, by my research team, also followed students into college, finding that voucher students enrolled and persisted in four-year colleges at higher rates than their matched public-school peers. The fact that Milwaukee voucher students advanced through their college years at better rates than the comparison group indicates that their higher high-school graduation rate was not driven by possibly-lower diploma standards in the private-school sector.

Why Do Effects Vary?

Private-school choice begins with a school switch for all participants except rising kindergartners who did not attend a preschool. Eric Hanushek and Steven Rivkin have established that student achievement tends to drop the year after a school switch, as students adjust to their new schools. Such a decline is likely larger for voucher students who move to a private school immediately after a choice program is created, because the schools also have to adjust--to an influx of new, disadvantaged students. The achievement effects of choice programs after just one or two years may well turn out to be misleading indicators of the longer-term effects on test scores and attainment. Parental choice is a commitment to a journey that takes time to deliver clear learning benefits to students.

It should not surprise us that private schooling boosts student attainment more than it does test scores. Most private schools focus on educating the whole child: intellectually, behaviorally, and spiritually. For example, the Alliance for Catholic Education program based at the University of Notre Dame, which trains college graduates to teach in Catholic schools, speaks of "preparing students for college and heaven." Although research on the question remains at a formative stage, private schools tend to focus on molding student character by fostering grit, conscientiousness, and tolerance of others. Such character traits are more predictive of educational attainment than of future educational achievement. The attainment effects of choice programs may be outstripping their achievement effects because private schools prioritize character over test scores. That prioritization of long-term over short-term outcomes likely pleases their customers: parents.

The evidence also suggests, though by no means conclusively, that voucher programs targeted to low-income urban students have larger and more consistent positive effects on participants than do statewide programs that are less narrowly targeted. Because of the entrenched practice of assigning students to public schools based on their neighborhood of residence, urban public schools tend to concentrate highly disadvantaged students in schools characterized by low levels of safety and achievement. Prior research by William Howell and Paul Peterson suggested that the reason low-income inner-city African Americans benefit most from private-school choice is that moving to the new school represents a more dramatic improvement in the school environment for them than for less-disadvantaged white and Hispanic students.

Statewide choice programs are too new to generate a clear comparison with the more established urban voucher programs. Statewide programs in Florida, Louisiana, and Ohio, however, already have demonstrated clear positive effects on the achievement of students who remain in public schools, confirming Caroline Hoxby's claim (see "Rising Tide," features, Winter 2001) that competition from choice generates "a rising tide that lifts all boats."

Program Design

We know precious little about what makes some private-school-choice programs more successful than others--and success itself can be defined in various ways. A choice program can be called a success if it serves a large number of students, attracts a high percentage of private-school providers, improves racial integration in the community's schools, increases test scores of participants, improves the test scores of non-participants, boosts student attainment, or enhances student civic values. Different program designs are likely to favor some of these desirable outcomes at the expense of others. There is great risk in thinking, with so little evidence, that we know exactly how to design voucher programs to optimize student outcomes.

Still, I will offer a few humble suggestions. Voucher programs narrowly targeted to income-disadvantaged urban students reach a particular student population that appears to benefit most from access to private schooling. Meanwhile, programs broadly available to both low- and middle-income students statewide attract a diverse and likely higher-quality set of participating schools. Combining these two features in creative ways, such as by providing higher-value vouchers to lower-income students, might be the best way to match disadvantaged students with a wide array of private schools to serve their educational needs. Education savings accounts could prove to be the most effective mechanism for delivering private-school choice: they provide parents with more flexibility to customize the educational experience of their child, potentially drawing from multiple schooling providers. The ESA model also encourages parents to obtain the best value for their child's education dollars, as unspent money in one year can be rolled over to the next and even be spent on college costs.

A specific debate rages over what forms of government accountability to impose on private schools participating in choice programs, which already are accountable to parents, who can vote for or against them with their feet. There is merit to the arguments on both sides of this dispute. My main concern is that, in trying to perfect private-school choice, we could accidentally destroy it. Policies requiring private schools to administer the official state tests, which are aligned with the public-school curriculum, appear to discourage distinctive private schools from participating. Such policies also create incentives for schools that do accept voucher students to change their educational programs to match what the state tests. The fundamental purpose of vouchers is to permit parents to choose from among a diverse array of educational models for forming their children into successful adults. If policymakers impose regulations that limit the range of choice--either by disqualifying certain types of schools or by encouraging uniformity in curriculum and school identity--they will ultimately narrow family options rather than expanding them. That would be a regrettable choice.

STILL WAITING FOR CONVINCING EVIDENCE

DO PUBLIC-SCHOOL STUDENTS who move to a private school with a government-funded voucher benefit from making this switch? A growing body of research is shedding light on this question. Of particular interest are findings coming out of three states and the District of Columbia, all of which have implemented ambitious voucher programs over the past dozen or so years. That evidence does not seem to justify the fervor for vouchers displayed by many education reformers and now by U.S. secretary of education Betsy DeVos.

First, what do we know about the students who choose to participate in voucher programs? It almost goes without saying that families who choose to use tuition vouchers are less satisfied with their traditional public schools than those who stay. And since the vast majority of private schools are religiously affiliated, it comes as little surprise that voucher users tend to be more religious.

Participation in voucher programs is also driven by eligibility requirements. For example, programs that target low-income families directly or indirectly (by virtue of being based in urban areas) will enroll low-income students. Even when limited to low-income populations, though, vouchers tend to serve a socioeconomically advantaged portion of that group, those who are best positioned to leverage choice. Why? Mostly because this is how markets work. Economic research shows that more-educated adults are more likely to get what they want in the marketplace writ large.

This dynamic intensifies in the schooling market. In a practical sense, families who lack personal transportation or live far away from private schools do not have access to alternatives. Also, most private schools, in the absence of vouchers, are designed for the wealthy and middle class. It is wealthier families who can afford to pay tuition, and school eligibility requirements often exclude students who have academic and disciplinary challenges. That is largely because parents know what researchers have confirmed: students benefit from attending school with more-advantaged classmates. School reputations therefore depend to a substantial degree on exclusivity. This is also why we must view parent satisfaction cautiously. Research shows that white or affluent parents often avoid schools that have high concentrations of minority and low-income students. This might make them more "satisfied," but it is hardly a reason to celebrate.

Some evidence about exclusivity comes from my fellow participants in this forum, who have shown that private schools considering whether or not to accept voucher students often worry about having to lower their academic standards. Given the correlation between family socioeconomic advantage and the student characteristics that schools look for, this concern on the part of private schools will restrict access for voucher-bearing students. Religious schools, a partial exception, are more willing to participate in means-tested voucher programs, but they, too, often have academic and behavioral admissions requirements. In some cases, schools institute these policies in a well-intentioned effort to build strong scholastic communities, but their criteria effectively serve to exclude many students. Even though voucher programs often officially prohibit selection practices, these rules are rarely enforced, and research shows that schools have many ways to shape enrollment that fall outside the rules.

Vouchers that are targeted to disadvantaged students could theoretically help address the affordability barrier, but, in general, when governments target attractive financial benefits to one part of the population, politically powerful groups will exert pressure to loosen eligibility requirements to gain their own access. We have seen this in Indiana, Louisiana, and other places where small-scale, means-tested voucher programs gradually expanded to include families closer to middle class. The trend is toward voucher programs becoming less well targeted, with funding shifting to socioeconomically advantaged students who already have some degree of choice.

Effects of Vouchers

And what do we know about the academic outcomes of students participating in voucher programs? Most of the rigorous research, now dating back more than a decade, focuses on programs in large urban areas, such as Milwaukee and New York City. Averaging across 16 U.S.-based programs, Patrick Wolf and colleagues find that these small-scale voucher programs have statistically insignificant effects on standardized test scores across academic subjects.

This interpretation is different from what Wolf concludes from the same results in his contribution to this forum. The reason for the difference is noteworthy. In his Figure 1, Wolf shows that studies of voucher programs that examine students' outcomes longer after they switch to a private school produce more positive results. However, this conclusion assumes that the programs had no effect on the achievement of students who switched to a private school only temporarily and, even if valid, could only be generalized to the small subset of students who used a voucher consistently when given the opportunity to do so, ignoring the smaller and perhaps negative effects on the majority of voucher users. The analysis also includes a now dated study of the Washington, D.C., program, the effects of which have since turned negative (more on this below). Moreover, the other handful of studies that lasted for four years might only have done so because they were relatively successful, so that the less successful ones are omitted. In contrast, my interpretation--that there is no statistically significant effect--places equal weight on all students observed using a voucher to enroll in a private school, including all studies and students regardless of how long they were followed, therefore capturing all the positive and negative effects at work. Note, too, that even Wolf's more optimistic interpretation asserts a positive achievement effect only for reading, not for math.

These early programs' effects on non-achievement outcomes, such as graduating from high school, tend to be somewhat more positive. Even if we place more weight on these outcomes, however, we need to keep in mind that effects found in small-scale programs often do not generalize to larger scales. School-choice initiatives, including charter schools, seem to work better in cities than statewide because it is easier to exercise choice where there is better mass transit and higher population density, and the performance of traditional public schools is generally worse in urban areas, making it less challenging for choice programs to improve on baseline student outcomes.

The limited scale of the programs examined in most prior studies is important, because the United States is now in the midst of a full-scale nationwide expansion of these policies. Twenty-five states and the District of Columbia have some type of voucher program. Just four statewide voucher programs have been formally evaluated, and only one has shown any signs of success. Ohio's statewide program has shown clear negative effects on test scores. Two others, in Indiana and Louisiana, started off with some of the worst test-score results any education program has ever demonstrated, though these subsequently improved somewhat so that the net effects are now essentially zero. In the fourth program (Florida), the authors conclude that test-score effects cannot be determined with any confidence because of the program design.

Why are the results for test scores not more positive? One possible reason is that the state-mandated tests were not well aligned to the curriculum taught in private schools. However, the most recent experimental evaluation of the D.C. voucher program showed negative test-score effects after one year, even though the study did not rely on a state-mandated test--and despite the fact that an earlier study of the program showed no effects. The more likely explanation is that private schools in the city are competing with a public-and charter-school system that has demonstrated substantial academic improvements in recent years. More generally, where vouchers are competing with charter schools--which have produced increasingly positive results over time--the voucher results are likely to continue to be less positive. It is harder to look good against stronger competition.

Another possible reason for the uninspiring results is that the private schools that participate in statewide voucher programs are simply not very effective. This could be interpreted as a failure of the voucher concept or, as voucher supporters have asserted, it could be that the regulatory burden of the programs, while very small compared with those of public or charter schools, kept the best private schools from joining. Research by Wolf and colleagues does not seem to support the latter interpretation, however. Based on ratings from the organization GreatSchools, the schools participating in the Louisiana voucher program were not of lesser quality than those that did not participate, though the voucher-accepting schools did charge lower tuition.

The effects of the Florida Tax Credit (FTC) scholarship program on college outcomes have been widely cited as a success story, but several caveats apply here. First, unlike the other studies mentioned above, the design of the FTC program precludes a rigorous research design. In their Florida study, Matthew Chingos and Daniel Kuehn do their best by matching students on observable characteristics that are somewhat removed from the outcome of interest. But this strategy is unlikely to yield an apples-to-apples comparison. In studies using test scores as the outcome, matching is much more effective, because the treatment and comparison groups of students can be matched on their scores--the variable of interest--before students receive vouchers. This is not possible when studying college-going. Also, because the assignment to the FTC is not random, the more positive effects they see for students participating for more years may, as they acknowledge, reflect selection bias; that is, any student who stays in the same school for more years is likely to have better outcomes.

Earlier research in D.C. provides evidence of positive effects on another important long-term outcome, high school graduation, but these findings are now difficult to interpret for another reason. The downward trend in test-score results in D.C. calls into question whether prior outcomes still reflect the current reality, given increased competition from charter schools.

Still Waiting

What has all this taught us about how states ought to design and oversee voucher programs--and, indeed, whether they should do so at all? How about slow down? The latest results should give everyone pause.

Try this exercise: Let's drop the word "voucher" and simply say, "Statewide--programs show a mix of tenuously positive and negative results." Now, fill in the blank with your favorite non-performing program. It is hard to imagine that any objective observer would respond by saying, "Great, lets expand this to states across the country." Yet this seems to be what DeVos and half the states in the country have concluded about vouchers.

Robert Pondiscio, senior fellow at the Thomas B. Fordham Institute, has argued that empirical evidence is largely irrelevant to determining whether vouchers and other forms of choice programs should be adopted, because choice aligns schooling with a core American value: freedom. Certainly, it is desirable that education policy support our most fundamental principles, and freedom is at the top of the list. However, this argument assumes that choice policies, by definition, increase freedom. Whether that assumption holds true depends on what form of freedom we mean and how policies are designed. A voucher program that allows schools to set their own rules and does not provide transportation will increase freedom only for those students who meet the schools' standards and who can find their own way to and from school.

Even that interpretation assumes we mean freedom in the libertarian sense--that is, freedom from restrictions on individual choice. School attendance zones, which assign children to a particular school by their neighborhood of residence, do curtail this kind of freedom. But the freedom that comes from promoting educational opportunity is also important. Unlike the libertarian form of freedom, opportunity is mostly an empirical issue, as it requires not only that families be unfettered by government policies in selecting schools for their children, but also that they are able to choose from among accessible, high-quality options. We can debate what criteria define quality--strong test scores versus parent satisfaction, for example--but the assertion that opportunity is an empirical issue is hard to dispute.

It is also important to consider how voucher programs contribute to or detract from other salient cultural values such as equity, community, and democracy. The fact that vouchers are likely to open access only for some creates an immediate concern for equity. Apprehension that vouchers will undermine neighborhood schools--and the neighborhoods themselves--is also well founded at a time when geographically based communities are already under great stress.

The voucher debate, therefore, is a question not just of values but also of effectiveness, and research should play a significant role. So how should we interpret the available evidence? At most, only one of the more than two dozen states that have tried statewide vouchers and tuition tax credits has yet to demonstrate convincing, measurable success with them, Given this reality, it is hard to make a case for substantially replacing our system of public schooling on a national scale. The American workforce continues to be the most productive and creative in the world. This does not mean we cannot do better, but it does indicate that we should proceed with caution and care.

Finally, we cannot interpret the voucher evidence without thinking about the alternative policy options. Vouchers represent just one form of choice. Given the multitude of ways in which we would expect a free market in schooling to fail, perhaps other forms of choice that strike a different balance between government and market forces would be more effective. The evidence on charter schools, for example, is increasingly positive--even at scale. Perhaps what some call the "portfolio model," and what I have called "managed competition," will do more to increase freedom, equity, efficiency, and community. A system of managed competition gives families genuine choice in schooling, but it also ensures 1) true accessibility to these options; 2) transparency, including data reporting and open board meetings; 3) coordination of school operations with a government body that has some degree of authority; and 4) government enforcement of the rules and protection of students' civil rights. It also seems likely that different localities need different systems, and many might be best served by maintaining traditional public schools.

We have been debating vouchers for decades, even centuries, without much evidence to inform those debates. Today, policy advocates are way out in front of the evidence, especially with the current proliferation of statewide voucher programs. The new federal expansion of tax-favored 529 savings plans to include tuition for private schools, a move that constitutes reverse targeting to the affluent, has even less justification. It would be wise to put a hold on further broad-based experiments until we see whether the dozens of relatively new programs yield more positive results than the older ones. When it comes to convincing evidence, we are still waiting.

LESSONS LEARNED FROM INDIANA

THE INDIANA CHOICE Scholarship Program, launched in 2011, offers a rich opportunity to study how a large-scale tuition-voucher program works and to analyze the results it has produced in its first few years. As we consider the merits of private-school choice and what it would take to make it succeed, this initiative deserves particular attention: it is the nation's largest voucher program, accounting for nearly 20 percent of all voucher students nationwide, with 34,299 students receiving vouchers and 313 private schools participating during the 2016-17 academic year. And in terms of student eligibility, Indiana's program is the broadest, with its vouchers aimed at both low- and moderate-income families and no cap on the number of students who can take part. The average scholarship amount, based on the public-school district in which students live, ranges from nearly $4,500 in kindergarten to $5,600 in high school.

Our four-year evaluation of the Indiana program is one of a few recent studies that finds statistically significant negative effects on student achievement of using a voucher to switch from a public to a private school in the first years after a choice program's launch. But that is only part of the story. Our research also shows that voucher students begin to recoup their academic losses in their third and fourth years of attending a private school. Students transitioning to a private school may need time to acclimate to what are usually more-rigorous academic standards and higher expectations for homework and schoolwork. Our findings also speak only to the achievement gains of students using vouchers to switch to a private school in grades 5-8. Starting students in private schools in earlier grade levels, and thus giving them more time to adjust, might produce better outcomes.

Given that many state and federal policymakers support the expansion of private-school choice, Indiana's experience can offer lessons for the design of future voucher programs.

Who Participates?

About 76 percent of Indiana's private schools--and almost 100 percent of its Catholic schools--participate in the voucher program. Not long after the program began, we conducted a survey of all Indiana private schools to learn why they did or did not decide to participate. Those that chose to join most frequently specified their mission, service orientation, and school finances as reasons (see Figure 1). Schools that opted not to participate cited reasons such as the desire to maintain autonomy from government, their lack of required accreditation (and unwillingness to become accredited), and concern that enrolling voucher students would require the schools to lower their academic standards.

When examining student participation, we looked at those who joined the program early and those who are participating now, because the program has changed over time. Initially, students could receive a voucher only if they had attended a public school for at least one year, or if they had attended a private school with the help of the state's less-generous tax credit for some parents paying private-school tuition. In 2013, the program dropped the public-school attendance requirement. Thus, in its first year, 90 percent of Indiana Scholarship students had previously attended a public school, but by 2016-17, over half--55 percent--had never attended one. In other words, the program started out serving students who wanted to leave public schools, but it now serves a majority of students who have attended private schools from day one. Why Do Schools Participate in the Indiana Scholarship Program? (Figure 1) Three-quarters of Indiana's private schools participate in the state's voucher program, and schools that chose to join most frequently specified their mission, service orientation, and school finances as reasons for doing so. Percentage citing factor as... * A reason for * One of the participating top three reasons for participating Expand mission to 91 66 larger community Help existing voucher-eligible 65 33 families Help needy children 64 30 in the community Bring back former families who could not afford tuition 57 28 Fill open seats in 47 17 some of our classes Strengthen the school's 41 18 financial situation A lot of eligible families 40 19 interested in having child attend the school Prevent school from being 5 2 closed or consolidated SOURCE: 2012 Survey of Indiana private schools Note: Table made from bar graph.

As the program changed, so too did the demographics of participating students (see Figure 2). In the first year, 24 percent were African American, but this number declined to 12 percent in 2016-17. Conversely, the percentage of white students receiving vouchers increased from 46 percent in the first year to 60 percent in 2016-17. The shares of Hispanic students (20 percent) and multiracial students (6-7 percent) remained consistent over time. Statewide in 2016-17, the K-12 student population was 69 percent white, 12 percent African American, 11 percent Hispanic, and 5 percent multiracial.

In the programs first year, 69 percent of its students lived in metropolitan areas, 16 percent in suburbs, 7 percent in rural areas, and 7 percent in small towns. By 2016-17, the proportion of students from metropolitan areas decreased to 61 percent, while the share of those from suburban areas increased to 23 percent. Statewide in 2016, about 31 percent of students in grades 3-8 lived in metropolitan areas, 27 percent in suburbs, 14 percent in small towns, and 28 percent in rural areas. (Corresponding figures for the full K-12 enrollment were not readily available.)

How the Program Works

When the voucher program first began, we conducted about 100 interviews with principals, teachers, parents, and students in 13 participating private schools. We sought to learn how schools were providing educational opportunities to students using vouchers; how schools adapted their curriculum and instruction; and how students were integrating into their new environments academically, behaviorally, and religiously.

Statewide, students receiving vouchers were low-achieving before entering private schools (on average, performing at the 42nd percentile compared to public- and private-school students statewide). Principals, teachers, and students we interviewed said that students who transferred into private schools using a voucher had not been required to do much homework in the public schools. These students moved to private schools whose students were performing, on average, at the 53rd to 57th percentile in mathematics and English language arts (ELA), respectively. Schools responded to their new students by providing more individualized instruction and in some cases, by adding an "ability" group. One teacher, for example, observed a much larger gap between her high-achieving and low-achieving students than in previous years. In response, she provided more differentiated instruction and worked with smaller groups when possible.

Voucher students themselves also had to adapt to the higher academic and behavioral expectations in their new private schools. A principal's comment typifies what we heard in our interviews: while the school did not change its expectations, he said, teachers worked to help the new students adjust to the expectations and understand that such standards applied not just to them but to all the students.

Effects on Achievement

Because Indiana does not cap the number of vouchers awarded, it has no lottery process to determine who's in and who's out. Without the benefit of random assignment, we used a variety of statistical approaches to determine the program's impact on student achievement immediately after its launch. We focused on students using a voucher to switch from a public to a private school in grades 5-8 during the program's first four years (2011-12 through 2014-15). Because Indiana public and private schools use the same assessment in grades 3-8, we could identify public-school students who shared similar achievement trajectories and demographic characteristics with these voucher students at baseline (the year prior to a student switching from a public to a private school) and track both groups' academic progress for up to four subsequent years.

Overall, we found an average loss in mathematics of 0.12 standard deviations (roughly 3-4 percentile points) from baseline for students who used a voucher to transfer from public to private schools (see Figure 3a). The largest losses occurred during years one and two. However, voucher students began to show signs of improvement by their fourth year in a private school, and in that year there was no statistically significant difference between them and their public-school peers in terms of total achievement gains from baseline. The negative math effects in the early years are similar to recent findings for students participating in new statewide voucher programs in Louisiana and Ohio, though smaller in magnitude. In ELA, we find no statistically significant average difference in the performance of voucher and public-school students across all four years (see Figure 3b).

Our estimates of the effects of voucher use after three and four years are based on a relatively small number of students: fewer than 200 in year four, as compared to roughly 3,000 in year one and 1,700 in year two. That's mainly because state test scores are not available for voucher students who had reached high school by that time. Nearly 15 percent of voucher students also return to a public school within one or two years, so our longer-term estimates represent the most persistent students. Furthermore, our main results are averages across all participating private schools, and estimates of the effects of using a voucher to attend specific private schools vary widely (that is, voucher students excel in some private schools and perform poorly in others). Even so, our analysis provides the most complete picture to date of the early effects on student achievement of a voucher program operating at scale.

Implications for Program Design

Although further research is necessary to understand the variation in the Indiana program's effects--including the organizational and instructional environments of voucher schools--our research to date offers some clear policy implications:

Allow enough time for preparation. Implementing voucher programs incrementally might be more prudent than scaling up quickly. The state legislature passed the Indiana Choice Scholarship Program in the spring of 2011, and students began using vouchers that fall. Our interviews reveal that, despite their strong support for the program, private schools felt rushed in their communication with families and enrollment of students. Giving schools, families, and students more time to prepare for change might ease adjustment all the way around.

Start smaller. Before expanding voucher programs statewide, it might be better to start with smaller programs using a lottery process. This way, the randomization can allow policymakers, educators, and researchers to assess more definitively the programs' effects.

Measure broadly, and begin in early grades. States should rely on a wider set of measures to evaluate voucher programs and begin this appraisal at earlier grade levels. Certainly, test scores are important proxies for what students are learning, but currently there is no standardized assessment taken by both public- and private-school students in grades K-2 in Indiana. What's more, researchers know that other outcomes (for instance, social-emotional skills, engagement, motivation, and progression through K-12 schooling) are critical when considering the impact of education reforms on students of all ages. Going forward, our study will examine the impacts of vouchers on high-school students' achievement and attainment.

Start students early. As we mentioned at the outset, it may be that implementing voucher programs in early grade levels would better acclimate students to private schools. Voucher students in upper grade levels appear to need time to adjust to the more demanding homework and high expectations in their new schools. In this regard, it is worth noting that a majority of students currently using vouchers in Indiana have enrolled in private schools from the start.

Make it easy for good schools to participate. States implementing voucher programs should ensure the quality and number of private schools willing to take part. About three quarters of those in Indiana are participating. This contrasts with Louisiana, where the supply of private schools has posed a challenge, likely due to the regulatory burden they would face if they signed on.

Consider teacher training. Private-school educators may need additional training to prepare for serving voucher students. Our findings suggest that teachers would benefit from professional development in mathematics curriculum and instruction and in learning to lead more-diverse classrooms.

Although we have much to discover about the impacts of statewide voucher programs, we are beginning to understand what policymakers and educators should consider when implementing new ones. Until research can show the conditions under which voucher programs succeed, the policy debates will rage on. In the meantime, Indiana's experience to date provides useful guidance for states intent on moving ahead now.

by PATRICK J. WOLF

by DOUGLAS N. HARRIS

by MARK BERENDS, R. JOSEPH WADDING, and MEGAN AUSTIN