The New Orleans OneApp: centralized enrollment matches students and schools of choice.

Harris, Douglas N. ; Valant, Jon ; Gross, Betheny 等

In most of the U.S., the process for assigning children to public schools is straightforward: take a student's home address, determine which school serves that address, and assign the student accordingly. However, states and cities are increasingly providing families with school choices. A key question facing policymakers is exactly how to place students in schools in the absence of residential school assignment.

In the immediate aftermath of Hurricane Katrina, New Orleans families could choose from an assortment of charter, magnet, and traditional public schools. The city initially took a decentralized approach to choice, letting families submit an application to each school individually and allowing schools to manage their own enrollment processes. This approach proved burdensome for parents, who had to navigate multiple application deadlines, forms, and requirements. Moreover, the system lacked a mechanism for efficiently matching students to schools and ensuring fair and transparent enrollment practices. The city has since upped the ante with an unprecedented degree of school choice and a highly sophisticated, centralized approach to school assignment.

Today, New Orleans families can apply to 89 percent of the city's public schools by ranking their preferred schools on a single application known as the OneApp (see Figure 1). The city no longer assigns a default school based on students' home addresses. Instead, a computer algorithm matches students to schools based on families' ranked requests, schools' admission priorities, and seat availability. Experience with the OneApp in New Orleans reveals both the significant promise of centralized enrollment and the complications in designing a system that is technically sound but clear to the public, and fair to families but acceptable to schools. The OneApp continues to evolve as its administrators learn more about school-choosing families and school-choosing families learn more about the OneApp. The approach remains novel, and some New Orleanians have misunderstood or distrusted the choice process. The system's long-term success will require both continued learning and growth in the number of schools families perceive to be high-quality options.

The OneApp's Design

Early centralized enrollment systems, and the matching algorithms at their core, suffered from a key flaw: the lotteries were designed so that if a family ranked its most-preferred school first and that school was in high demand, then the family could lose its second-ranked option. In this situation, it could be rational for families to rank less-preferred options first. This is precisely what families did in cities like Boston that used this approach to match students to district schools, and it likely produced inefficient outcomes.

The challenge that faced the state entity that oversees most of the New Orleans schools, the Louisiana Recovery School District (RSD), was how to build a centralized, market-like enrollment system without inducing inefficient strategic behaviors. The solution was found in the Nobel Prize-winning research of Stanford economist Al Roth. He, along with fellow Nobel Prize winner Lloyd Shapley, showed that a system could be designed to elicit true preferences just as prices would in a normal market. New Orleans and Denver became the first cities to use this Roth/Shapley-inspired centralized enrollment system across charter and district sectors. In New Orleans, this enrollment system is called the OneApp. To develop and run the OneApp, the RSD contracted with the Institute for Innovation in Public School Choice (IIPSC), an organization for which Roth has served as an adviser and board member.

For families, the OneApp process begins by acquiring an application packet with details about the application process, profiles of participating schools, and the application itself. Parents can request up to eight schools by submitting a ranked list to the RSD, in paper or online. The RSD then assigns students to schools based on families' preferences, schools' enrollment criteria, and seat availability. Families that do not submit a "Main Round" application, are not assigned to a school, or would like to try for a better placement may apply in a subsequent round. Families still lacking a satisfactory placement after the second round can go through a late enrollment process managed by the RSD to select from schools with available seats.

The machinery driving these placements is the RSD's "deferred acceptance" computer algorithm. The first step of the process is to assign every student a lottery number for use when seats in oversubscribed schools must be allocated at random. The algorithm then tentatively assigns students to their first-choice schools, provided that students satisfy the entry criteria. If the school cannot accommodate all families applying for that grade, then the algorithm makes tentative assignments based on the school's priority groupings (e.g., whether the student lives within the school's broad catchment area) and students' lottery numbers. At this point, students who were not assigned to their first-choice school are rejected from that school. Importantly, however, the algorithm leaves all assignments tentative until the final step. This means that students tentatively assigned to their first-choice school might later lose their seats to students who ranked that school lower than first but were rejected from all higher-ranked schools. This is key to the algorithm's strategy-proof design.

[ILLUSTRATION OMITTED]

In the next step of the process, all students who were rejected from their first-choice school are considered for their second-choice school. The algorithm considers them along with other second-choice applicants and those who were tentatively assigned to their first-choice schools. These steps are repeated for third choices and so on until no available seats remain. The algorithm's final step is to actually assign all students to the schools to which they are tentatively assigned. Only then are families notified of the results.

The OneApp has many useful properties as a system for assigning students to schools of choice, including its strategy-proof design. To maximize the probability of receiving a desired placement, applicants have an incentive to rank as many schools as possible (eight) in their true order of preference. In fact, deviating from that strategy only makes it less likely that applicants will be assigned to their most-preferred schools. Yet even a technically elegant system--and especially one this difficult to explain--faces challenges when it confronts families making decisions for their children in actual choice settings.

The OneApp in the School Choice Context

The RSD set three goals for the OneApp: efficiency, fairness, and transparency. Here, we consider the OneApp and centralized enrollment in the context of these goals, at times defining them differently from how the RSD does. We examine not just the technical process of assigning students to schools, but also the relationship with the city's broader school-choice setting, since the OneApp is so intertwined with New Orleans overall education policy. To incorporate empirical evidence when possible, we draw on data from interviews with 21 parents and surveys of 504 parents about the OneApp and school choice, conducted in the spring of 2014 by the Center on Reinventing Public Education (CRPE). We also utilize de-identified OneApp data containing families' school requests and assignments for the 2013-14 school year.

[ILLUSTRATION OMITTED]

Efficiency. A centralized enrollment system like the OneApp may improve efficiency both in how families choose schools and how the broader market for schools operates. The RSD's stated definition of efficiency is reasonable, if incomplete. It states that the OneApp can improve efficiency by making the enrollment process easier for parents to navigate, reducing the costs associated with choosing and enrolling in a school. We favor a definition that also considers how successfully the system matches families to the schools they want. Economists emphasize the importance of matching preferences with products--in this case, matching what families want with the available schools. Given the available schooling options, the OneApp algorithm is designed to do that.

How well the OneApp stacks up on this two-pronged definition of efficiency depends on the alternative to which it is compared. Relative to traditional zone-based assignment, the OneApp requires somewhat more effort from families. Families are asked to gather information and think about the many options in front of them before actively selecting a school and ranking their preferred schools. Families could incorporate school considerations into decisions about where to live, but once a residential decision is made, the school-housing linkage sharply limits a family's options. Traditional zoned-based assignments may be less able to match family preferences than the OneApp, especially for those who don't have the means to purchase or rent a home in a neighborhood with desirable public schools.

[FIGURE 1 OMITTED]

Compared with decentralized choice, where families apply to every school separately, centralized enrollment should be easier on families by reducing the applications and deadlines they have to navigate. It also should more efficiently match families to schools via a centralized matching algorithm. Perhaps surprisingly then, CRPE's surveys of New Orleans parents in spring 2014 found that families that chose schools after the OneApp was instituted in 2012 reported greater difficulty with the number of applications and deadlines involved than families that chose schools before the OneApp. This may have been due to families adjusting to an unfamiliar process early in the OneApp's tenure. It will be worth tracking future surveys to see if parents grow more comfortable with the procedures as these procedures grow more familiar.

In general, most families that enter the OneApp are getting the schools they request. The RSD reports that 54 percent of Main Round applicants received their first-choice school and 75 percent got one of their top three choices for the 2015-16 school year (see Figure 2). While these results are encouraging, no comparable metric exists for zone-based assignment or decentralized choice, and these metrics can be misleading. They indicate how well participating families are being matched to participating schools. These measures cannot gauge families' true satisfaction with their school options and their matches. For example, if an extremely popular school joins the OneApp and many families rank that school first, the percentage of families receiving their first choice might fall even as the system's ability to match families to desirable schools improves. For this reason, the OneApp data provide limited, though useful, information about family satisfaction. Continued surveys and discussions with school-choosing New Orleans families can complement the information from these publicized metrics.

Fairness. Defining fairness requires normative judgment. A high standard might hold that access to high-quality schools does not vary by students' socioeconomic status. Every modern enrollment system would fall far short of this standard. Traditional zone-based systems generally leave low-income and minority students heavily concentrated in low-performing schools. Decentralized systems typically favor parents who have strong social networks and resources to understand, navigate, and even manipulate the many different enrollment processes in a city. The centralized OneApp system is not devoid of problems either. Students receive preference within their geographic catchment areas, and students from affluent families are more likely to have the preparation needed for admissions to selective schools. Moreover, the early deadline for schools with special entrance requirements--in December of the year before enrollment, two months before other Main Round applications are due--requires early awareness that may disadvantage all but the most well-informed or socially connected parents. On the other hand, families of all backgrounds at least have a chance to enter lotteries for the vast majority of schools, and even though some of the most desirable schools have early deadlines and additional requirements, simply including these schools in the OneApp likely makes them more visible and accessible than they would have been otherwise.

A more attainable definition of fairness, and the one adopted by the RSD, is that a system is fair if it sets rules governing enrollment and assignment in advance and then applies those rules consistently to all students. Residence-based school-assignment systems generally treat students within their zones equally for purposes of admission, though there have been cases of skirting the rules with incorrect addresses or special treatment. More significant problems arise in schools of choice when, for example, school leaders hide open seats from certain types of students or manipulate their lotteries or waitlists--problems that are especially likely when schools manage their own enrollment processes amid significant accountability pressure. Prior to the implementation of the OneApp, a study by Huriya Jabbar found that roughly one-third of New Orleans principals admitted to practices that kept certain students out. The OneApp has reduced opportunities for schools to engage in these behaviors by transferring decisionmaking authority in admissions from schools to the centralized process. While system leaders report that these behaviors became less common after the OneApp, it did not completely eliminate opportunities for unfair enrollment behaviors, as schools still might dissuade certain families from applying or enrolling. But these behaviors cannot be remedied with an application system alone.

[FIGURE 2 OMITTED]

Transparency ... and Clarity. The RSD also includes transparency among its primary goals, and for good reason. Being open and honest about the rules governing enrollment and the strategies for effective participation is an essential element of the responsible administration of a centralized enrollment system. We submit, however, that simply being transparent is not enough with a program as unfamiliar and potentially confusing as a centralized enrollment system. A transparent system can still be unclear, and a lack of clarity can produce misunderstandings and distrust that undermine even the most transparent system.

To assess transparency, we again compare a centralized enrollment system with the alternatives. Attendance zones are extremely transparent, despite obvious questions about equity. At the other extreme, decentralized choice systems can have severe transparency concerns, with schools individually managing their lotteries and waitlists outside the view of the public or an oversight agency. State or local rules requiring public lotteries and equal treatment may be helpful but difficult to enforce, as Jabbar's evidence on pre-OneApp principal behavior attests.

The OneApp, in contrast, requires that all rules and criteria determining admission are set in advance and, in fact, coded into a computer algorithm. The criteria are also included in the OneApp enrollment packet for the public to see. Some schools still give priority for criteria such as being the child of a school staff member, but these criteria at least are made known to the public. Putting this information in the OneApp booklet helps families understand the enrollment processes, and may discourage schools from adopting enrollment criteria or processes to strategically manipulate their pools of incoming students.

Being clear about certain elements of the OneApp has proven more difficult than being transparent. In some ways this is understandable, since at the core of the OneApp lies an algorithm that is difficult to explain to even the most interested audience. Yet clearly communicating to families information about the matching process and instructions for correctly filling out an application is essential, since misunderstandings or mistrust may lead parents to approach the OneApp in ways that undermine its goals. To examine the possibility of misunderstandings or mistrust, we analyzed patterns in OneApp rankings and interviews and surveys with parents. Useful, if limited, evidence of the OneApp's clarity can be found by identifying application behaviors that reduce applicants' probability of getting their desired placements.

We find evidence that many families do not approach the OneApp as its designers likely expected. The OneApp allows families to rank up to eight schools, and given the algorithm's strategy-proof design, families cannot gain by ranking fewer than the allowed number. Yet most families rank far fewer than eight. Applicants seeking nonguaranteed kindergarten or 9th-grade Main Round placements for the 2013-14 school year submitted forms with only 3.1 schools ranked, on average. (Students are guaranteed slots in the schools they currently attend.) Perhaps these families were considering only a few OneApp schools before seeking out private schools or non-OneApp public schools. For many applicants, this did not seem to be the case. In the Main Round, 315 families that requested nonguaranteed kindergarten or 9th-grade placements with applications listing fewer than eight schools did not get placed at all. Of these families, about half (164) applied to at least one additional school in a subsequent round of the OneApp, which indicates a willingness to enroll in a school not originally ranked. Many of these families likely would have been better off listing additional schools in their Main Round application, when more schools were available to them. While this amounts to a small proportion of total OneApp applicants, others who ranked fewer than eight schools and yet received a Main Round placement might have simply been fortunate.

[ILLUSTRATION OMITTED]

One possible explanation for this behavior is that many parents do not understand or believe the OneApp's strategy-proof design. Parents interviewed by CRPE researchers described efforts to outwit the OneApp's matching algorithm by ranking fewer than eight schools. For example, many interviewed parents reasoned that by ranking only their most-preferred schools, they gave the RSD little alternative but to assign them to one of their top choices. While such decisionmaking is hard to observe in the OneApp data, this kind of strategy puts parents at a greater risk of not matching to any school.

The number of families that do not submit an application at all suggests that many families, despite the RSD's efforts to publicize the OneApp and provide information on procedures, may still be unclear about the OneApp process. For the 2013-14 school year, 2,881 applicants requested a nonguaranteed kindergarten or 9th-grade placement during the Main Round in February. However, another 774 applicants first requested a nonguaranteed kindergarten or 9th-grade placement in either Round 2 (in May) or Round 3 (in July), before the final administrative matching process. With some highly regarded schools filling up during the Main Round, these families' access to desirable schools was limited. For many, missing the Main Round was likely the result of imperfect information about either the OneApp process or their own plans for the coming school year. And certain populations are especially vulnerable. Families just arriving in New Orleans, families with children just reaching school age, and families without access to informed social networks could struggle to learn about the OneApp process in time.

Centralized Enrollment and Education Policy

In many ways, the OneApp is more efficient, fair, and transparent than the decentralized choice system that preceded it. Despite this, some New Orleanians remain skeptical of the new system, often for reasons only tangentially related to the city's enrollment process. For example, in one parent's words, "This [common enrollment] would be great...if we had better choices." We argue that these impressions tend to emerge not from the OneApp itself but from the larger choice system, especially the closely connected "supply side" of the market. Yet these impressions can have direct implications for the OneApp. How the public feels about the school choice setting in New Orleans can shape education policy, and education policy can shape the OneApp's role, now and in the future.

[ILLUSTRATION OMITTED]

Examples of supply-side issues that can affect public perception include transportation, selective admissions, and nonparticipation in the OneApp. If families cannot access the schools they want because commuting to those schools is too difficult, their children do not meet performance requirements, or those schools do not appear in the OneApp, then families are unlikely to believe that centralized enrollment gives them real choice.

These supply-side issues intersect in New Orleans, where it can feel like a decentralized school-choice system operates alongside a centralized one. Most public schools in New Orleans are administered by the RSD, but among other public schools are those run directly by the traditional school district (the Orleans Parish School Board, or OPSB), OPSB-authorized charter schools, and charter schools authorized by the state's Board of Elementary and Secondary Education (BESE). Whereas all RSD schools participate in the OneApp and do so without academic entrance requirements, the same is not true of OPSB and BESE schools. Several OPSB and BESE public schools have selective admissions based on entrance exams, language proficiency exams, prior grades, essays, and other criteria. Some of these selective admissions schools do not currently participate in the OneApp, and school bus service is less consistently provided by them. This multi-part system can give rise to confusion and frustration, particularly among families trying to reconcile claims that they have unprecedented choice with the reality that their children may not have access to some of the city's most desired public schools.

Parents also indicated a slim possibility of receiving a seat in a high-quality school. While New Orleans schools have improved considerably since pre-Katrina (see "Good News for New Orleans," features, Fall 2015) and families seem to have a variety of schooling options (see "Many Options in New Orleans Choice System," research, Fall 2015), only 22 of the 90 schools in the 2015-16 OneApp received a letter grade of A or B under the state's accountability system. Of the four schools that received an A, three are full-immersion Spanish or French language schools that required applications during the Main Round's Early Window period because they mandated language proficiency tests.

[ILLUSTRATION OMITTED]

Moreover, while 89 percent of New Orleans public schools appeared in the OneApp, a few of the city's highest-rated, most-desired schools constitute the 11 percent of New Orleans public schools that have chosen to handle enrollment processes on their own, outside of the OneApp. Some of these same schools have complex application requirements and ambiguous selection procedures, heightening the sense that the best schools in New Orleans are not truly accessible to all families.

In the long run, parental perceptions will also depend on how the school system responds to market demand. The OneApp can help in this regard, since it collects information about family preferences. Ideally, system leaders use this information--along with other data on school quality--to increase the number of high-quality seats (e.g., by adding seats to desirable schools or opening more schools like them) and reduce the number of low-quality seats (e.g., by closing low-performing, undesirable schools). Indeed, the RSD has incorporated demand data in judgments about school sites, placing popular schools in buildings that can accommodate future growth. However, responses through the portfolio management process can be slow to develop, and some high-demand schools, feeling effective at their current scale, have expressed reluctance to increase their enrollment substantially. Individual school leaders may be able to adjust to demand signals more quickly by better aligning their offerings with community needs, though research on schools' responses to market pressures generally shows that schools make some programmatic improvements in response to demand pressures but focus more intently on superficial changes like improved marketing.

The OneApp will likely enjoy long-term public support only if it is woven into a larger fabric of school options and choice. These examples show that some important threads in this fabric are still missing. No matter how well thought out and carefully constructed the OneApp itself might be, families that find their preferred schools inaccessible or their options undesirable are likely to experience frustration and confusion. Some may judge the enrollment system using metrics of efficiency, fairness, and transparency, but parents will judge it based on their own experiences and interests.

[ILLUSTRATION OMITTED]

The OneApp represents an ambitious policy shift, requiring families and educators to think in an entirely new way about how students are assigned to schools. Given this, and the fact that the OneApp is still in its early years, misunderstandings are not surprising. With most families getting one of their top-ranked schools, the number of satisfied parents could give system and school leaders time to improve the application process further as well as the quality of schools offered. There are signs in New Orleans that such learning and improvement are underway. RSD administrators routinely consider the system's successes and failures, and modify it accordingly for the next iteration, all while the public continues to acclimate and learns how to better leverage the choice system. Continued learning and adaptation will be essential to the OneApp's sustained success and the ability of New Orleans to provide the country with a model for student enrollment that is worthy of replication elsewhere.

by DOUGLAS N. HARRIS, JON VALANT, AND BETHENY GROSS

Douglas N. Harris is professor of economics and founder and director of the Education Research Alliance for New Orleans at Tulane University. Jon Valant is postdoctoral fellow in the department of economics at Tulane University and at ERA-New Orleans. Betheny Gross is senior analyst and research director at the Center on Reinventing Public Education at the University of Washington Bothell.