For-profit and nonprofit management in Philadelphia schools: what kind of management does better than the district-run schools?

Peterson, Paul E. ; Chingos, Matthew M.

The federal law No Child Left Behind (NCLB) requires states to "restructure" any school that fails for six years running to make Adequate Yearly Progress (AYP) toward full proficiency on the part of all students by the year 2014. The law provides a number of restructuring options, including turning over the school's management to a private for-profit or nonprofit entity. Only a few school districts nationwide have sought help from either type of organization in the management of low-performing schools. But in 2002 the School District of Philadelphia, at the request of the state of Pennsylvania, asked entities of both types to participate in a substantial restructuring of many of its lowest-performing schools. The restructuring initiative was directed by the Philadelphia School Reform Commission (SRC), which contracted with for-profit organizations to manage 30 elementary and middle schools and with nonprofit organizations to manage 16 schools.

The policy intervention in Philadelphia raises questions of general interest: Do students at schools assigned to for-profit or nonprofit managers learn more than would be expected had those schools remained under school district management? Is for-profit management more or less effective at raising achievement than nonprofit management?

Told most simply, the Philadelphia story provides a threefold answer to these questions: 1) for-profits outperform district-managed schools in math but not in reading; 2) nonprofits probably fall short of district schools in both reading and math instruction; and 3) for-profits outperform nonprofits in both subjects. However, the answers require both explication and qualification.

The Theoretical Debate

The distinction between for-profit and nonprofit management has been a topic of continuing discussion in the scholarly literature on school reform. Nobel Prize-winning economist Milton Friedman theorized that for-profit firms are more effective because they have clear economic incentives to lift student performance. The firm can build its reputation (and in the long run generate a profit) only if it becomes known for running effective schools. Others have suggested, however, that for-profit firms are likely to cut costs and thereby shortchange students in order to benefit the firm's owners and shareholders. The debate over nonprofit organizations takes a different form. Some have argued that nonprofit managers are likely to be effective because they have close ties to the community in which they are embedded and can enlist the energies of committed entrepreneurs, who devote all available resources to enhance student performance. But others caution that nonprofit managers may not have the experience, resources, or economic incentives necessary for building quality educational institutions.

The Intervention

Only after an intense political struggle did the Philadelphia school district ask for-profit and nonprofit managers to assume responsibility for a number of its schools. In 2001 Pennsylvania governor Tom Ridge, a Republican and school voucher supporter, indicated he would not support any increase in funding for the Philadelphia school district until an independent entity had assessed its financial practices and educational effectiveness. Philadelphia's mayor at the time, Democrat John Street, knowing the district was facing a $215 million deficit, agreed to the study, and the state department of education asked Edison Schools to carry it out. Edison, a for-profit firm that manages charters and other schools under contracts with school districts, reported that the Philadelphia school district had spent $10 billion over a decade without being held accountable for the results. Governor Ridge refused to distribute any more state aid beyond current levels unless the school district agreed to a new partnership with the state.

When the local press reported that Edison was expected to assume management of many of the city's schools, the local teachers union mobilized in opposition, and groups of parent and student activists held rallies throughout the city. As the turmoil was reaching its climax, Governor Ridge resigned from office to become the nation's secretary of homeland security, and Pennsylvania's lieutenant governor, Mark Schweiker, became governor. Upon assuming office, Schweiker said he would not serve beyond the current term, ending in 2002, a decision that weakened his leverage vis-a-vis the Philadelphia school district.

The stage was set for a compromise that would save face for all the parties involved- It was agreed that the school board should be replaced by a School Reform Commission, three members of which would be appointed by the governor and two by the mayor. The SRC decided that only a limited number of the lowest-performing schools in the district would be turned over to private management. Edison Schools was not to be the only private provider. Instead, seven entities--three for-profit and four nonprofit--were chosen. The SRC explained its decision by saying that multiple providers would yield information on the kind of management that was most effective. The SRC asked Edison Schools to manage 20 of the schools; another 5 each would become the responsibility of two other for-profit companies, Victory Schools and Chancellor Beacon Academies. Sixteen of the low-performing schools would be managed by nonprofit entities--the University of Pennsylvania (3 schools), Temple University (5 schools), Foundations (5 schools), and Universal (3 schools). The SRC also appointed a reform-minded superintendent, the energetic and outspoken Paul Vallas, who had instituted a series of reforms in Chicago at the behest of Mayor Richard Daley.

The for-profit firms were more experienced in running schools but they had fewer local political connections than did the nonprofit entities, as none had operated programs within the city itself. Though Edison was held in high regard by the Republican state secretary of education, it faced strong opposition within Philadelphia, especially from the local teachers union. Edison Schools could claim considerable experience at running schools, however, as it was the manager of 100 district and charter schools nationwide. Victory Schools, a company that offered single-sex education within classrooms, was the manager of schools in New York state and Baltimore, Maryland. To strengthen its local connections, Victory hired a former district employee to head up its Philadelphia effort. Chancellor Beacon operated some 80 private and charter schools, but it had not previously managed schools under contract with a school district. Shortly after the intervention began, Superintendent Vallas canceled the district's contract with Chancellor Beacon, and its five schools were either brought back under district control or assigned to other providers.

By contrast, the nonprofit entities were--and have remained--politically well-connected institutions. The University of Pennsylvania is a Philadelphia icon, a highly prestigious Ivy League institution with a history dating back to Benjamin Franklin. Temple University's status is less exalted, but it is nonetheless an established Philadelphia institution of higher learning. Foundations was created by one of the school district's former associate superintendents and is staffed by many former district employees. Foundations also had close ties to a politically influential state representative active in community development programs. Universal, a community development corporation founded by Kenny Gamble, an immensely successful writer of soul music, has strong ties to Islamic leaders within Philadelphia's black community.

Temple University and the University of Pennsylvania drew on the resources of their schools of education. Rather than taking on a general reorganization of the schools, they focused mainly on providing professional development to teachers, formal and informal assessment feedback to teachers, and within-classroom coaching services to students. Foundations operated afterschool programs and favored a teaching approach that relied on computer-based learning in which students progress at their own pace. Universal was known for boosting economic development and providing social services, but it had only limited experience managing schools.

The school district restricted the prerogatives of both for-profit and nonprofit managers in a number of ways. Management had to operate within the framework of the district's collective bargaining agreements with its union employees, and teachers were allowed to transfer to other schools within the district if they wished. Many teachers chose to transfer, as could be expected given the fact that the schools in which they had been teaching were considered most in need of new management. The district also retained control over many aspects of school management, including the school calendar and the code of conduct for teachers and students.

All of that happened in 2002. Then over the following six years, several leadership changes occurred. Sandra Dungee Glenn was appointed by Democratic governor Edward Rendell (who had succeeded Schweiker) as the SRC's new chair. Glenn was a former community organizer and active in Democratic politics. Superintendent Paul Vallas left for New Orleans, where he took on the daunting task of rebuilding the city's post-Katrina school system. And Arlene Ackerman, who had previously served as superintendent in San Francisco and Washington, D.C., was hired by the SRC as the new superintendent.

Under its new leadership, the SRC removed five schools from the management of the for-profit firms (four from Edison, one from Victory) and one school from the management of a nonprofit entity (Temple). Acting upon the superintendent's recommendation, the SRC decided that the six schools should come under direct district control once again because they had not made Adequate Yearly Progress (AYP), as required by No Child Left Behind. "It's been six years--it is time to sort it out," said SRC chair Glenn. Ackerman indicated that many more schools could come back under district control once a full-scale review had been undertaken.

[ILLUSTRATION OMITTED]

Factors other than educational considerations may have influenced the selection of schools the district brought back under its control. As noted above, the nonprofit schools were well connected politically, while the for-profit firms were not. But it is also possible that the nonprofits were the more effective educational institutions. To see whether that was the case, we use a rigorous research design to estimate the impact of for-profit and nonprofit management in Philadelphia.

The Data

The Philadelphia school district supplied the information on which we base our analysis. Test-score, demographic, and school enrollment information on Philadelphia students in grades 2 to 8 from 2001 through 2008 are available for each student. The test-score data come from three different tests. The Pennsylvania System of School Assessment (PSSA) is the test currently used for holding schools accountable for improved student learning in Philadelphia. But when the private management intervention began in the fall of 2002, that system of measuring school performance was still a work in progress. However, two other tests were given to some Philadelphia students between 2001 and 2006: the Stanford 9 and the TerraNova, both of which are nationally normed. In order to place the information from these tests on a common scale, we followed the standard practice of standardizing all scores by test, grade, and year to have a mean of zero and standard deviation of one.

We next classified schools as under for-profit management, under nonprofit management, or under regular district management. The average combined reading and math test scores one year prior to the management change at schools assigned to for-profit and nonprofit entities were 0.39 and 0.13 standard deviations below the Philadelphia average, respectively, while the pre-intervention scores of the full set of 142 regular public schools were 0.19 standard deviations above the district average. Because of that disparity, we limited the schools included in the comparison group to the lower half of all regular district schools. Those 71 schools had prior test scores that were 0.15 standard deviations below the district average, a level of performance much closer to those at the schools placed under new management. Restricting the comparison group in this way allows us to make a cleaner, if not an exact, comparison while maintaining a sufficient number of schools to be able to detect sizable management impacts at conventional levels of statistical significance.

Method

The Philadelphia intervention does not provide the opportunity for a random assignment study of the impact of for-profit and nonprofit management. Schools assigned to intervention status were not chosen randomly, but selected on the grounds that they were in greatest need of intervention. We therefore employed a "difference-in-differences" analysis to estimate the impact of attending a for-profit or nonprofit privately managed school (relative to attending that school had it remained under district management). The treatment groups consist of schools managed by each type of private provider, and the comparison group includes the regular public schools with test scores below the median for all regular district schools, as discussed above. To identify the effect of treatment, we calculate the difference between average annual test-score gains made by students at treated and comparison schools before and after the intervention began. So, for example, if test-score gains at the schools treated by for-profit management were 20 percent of a standard deviation higher than gains before treatment, while comparably measured gains at the comparison schools were only 15 percent of a standard deviation higher, the estimated effect of for-profit management would be the difference between them, or 5 percent of a standard deviation.

Our use of annual gain scores provides an estimate of treatment effects based on the extent to which students at each school do better or worse than would be expected, given their initial test scores. We also include student fixed effects, which account for changes in the composition of the schools' student populations over time that cannot be explained by the limited set of student characteristics for which information was available in the district's database. Finally, we control for the demographic characteristics of students' peers and whether students have recently moved from one school to another.

The inclusion of student fixed effects means that students are compared only to themselves over time when estimating the effect of each kind of management. Estimates that include student fixed effects require at least three years of scores, which provide two changes in scores from one period to another, typically called gain scores (even though in some cases they are losses, not gains). In our analysis, there are 68,677 students for whom the data allow us to compute at least two gain scores in math (and a similar number for the reading estimation). For another 46,875 students one gain score is available. With one gain score, it is possible to use models that control for observable background characteristics, but those models cannot adjust for unobserved student characteristics (such as parental education or student commitment). We nonetheless use such models to estimate informally the impacts on all students for whom at least one gain score can be calculated. Those informal estimates require assumptions that are quite restrictive, however, so the best estimates available to us are only for the approximately 68,000 students that could be included in the model that employed a fixed-effects analysis.

Despite the tens of thousands of student observations, those in schools under private management are clustered within only 30 schools managed by for-profit organizations and 16 managed by nonprofit ones. As a result, the annual impacts of the intervention must be as much as 20 to 30 percent of a standard deviation in order to be detected at conventional levels of statistical significance. That substantial an annual impact is seldom detected for large-scale structural interventions in education. For that reason, we also discuss large impacts that fall short of statistical significance when the pattern of results is consistent over the six-year time period.

Results

The impact of nonprofit management appears to have been negative. At schools under nonprofit management, students learned, on average for the six years, 21 percent of a standard deviation less in math each year than they would have had their school remained under district management. In terms of years of schooling, the negative impacts on math performance were, on average, approximately half a year's worth of learning each year, a large effect (see Figure 1). However, the negative impact was statistically significant in only the first year after the intervention began. In reading, the average adverse impact of nonprofit management was roughly 10 percent of a standard deviation annually, about 32 percent of a year's worth of learning. The adverse effect on reading performance was statistically significant in only the first year after the intervention began.

[FIGURE 1 OMITTED]

As mentioned earlier, impacts may have been somewhat different for the students with fewer test scores, who could not be included in our preferred model. Adjusting for that possibility yields an adverse impact of nonprofit management of 18 percent of a standard deviation on math performance and 14 percent on reading scores.

The effect of for-profit management was generally positive, although only the math impacts are statistically significant. At schools under for-profit management, students learned on average 25 percent of a standard deviation more in math each year of the six years of the intervention than they would have had the school been under district management. The estimated impact each year was roughly 60 percent of a year's worth of learning, a large, statistically significant impact. Our adjustment using results from an alternative model that includes the larger number of students yields a positive annual impact of for-profit management on math performance of 12 percent of a standard deviation, 29 percent of a year's worth of learning.

The estimated average annual impact on reading performance of for-profit management relative to district management is a positive 10 percent of a standard deviation, approximately 36 percent of a year's worth of reading. However, that impact is not statistically significant. The adjusted impact was just 4 percent of a standard deviation, about 14 percent of a year's worth of learning.

The differential impact of for-profit and nonprofit management is especially sizable. Using the estimates given above, students in schools under for-profit management gained between 70 percent and greater than a full year's worth of learning in math more each year than they would have had the schools been under nonprofit management. All of these differences are statistically significant. In reading, students learned approximately two-thirds of a year more in a for-profit school than they would have had the school been under nonprofit management. All but one of the differences are statistically significant.

[ILLUSTRATION OMITTED]

Taking Back Five Schools

Our analysis provides compelling evidence that schools do much better under for-profit than under nonprofit management. Year after year, students learned substantially more in reading and math if they attended a school under for-profit rather than one under nonprofit management. Yet in 2008 the district reassumed control of only one school under nonprofit management while not renewing the contracts for five schools under for-profit management. To ascertain whether that decision had a strong educational basis in the district's own test-score database, we used the same methodology to estimate the impact on student learning of the five schools for which the for-profit management contract had been terminated.

The results are mixed but provide little support for the district's decisions. The reading performance of students at the five schools was, on average, 18 percent of a standard deviation below what could have been expected had the schools been under district management, a difference that is statistically significant in three of the six years. Also, nonprofit schools whose contracts were not revoked had an impact on reading performance that was 10 percent of a standard deviation more positive than that of the for-profit schools whose contracts were terminated. However, that difference is not statistically significant in any year.

Math performance of students at the for-profit schools was 35 percent of a standard deviation higher than would have been the case had the schools been under district management. Also, it was 56 percent of a standard deviation higher than would have been the case had the schools been under nonprofit management. Those large differences are statistically significant in most years.

Our results indicate that nonprofits outperformed the five for-profits in reading in five out of six years, although the difference was only 3 percent of a standard deviation in 2008, and in no year were the differences statistically significant. In math, the five for-profits had strongly positive impacts in all years, while the nonprofits had decidedly negative ones, leading to very large, statistically significant differences between the two groups of schools in all years. The large differences in math clearly offset the statistically insignificant differences in reading.

If the Philadelphia school district cares only about reading results, and places no weight on math results, our data could be used to support the policy choice that was made, provided no attention is paid to the statistical insignificance of the reading finding. But if the two subjects are given equal weight in evaluating a school, our results provide no support for the decisions made by the school district with respect to renewing for-profit and nonprofit management contracts.

Discussion

Care should be taken before generalizing from the Philadelphia experience concerning the relative advantage of for-profit and nonprofit management. It is possible that for-profit entities have a greater vested interest in enhancing student achievement, because only in that way are they likely to survive over the longer run. But other factors in Philadelphia could easily account for the same result. The two main for-profit providers had much more experience with school management than did any of the nonprofit organizations. The nonprofits seem to have been selected more for their strong political ties than for any history of effectiveness at delivering educational services. Others have reported that newly formed charter schools under both for-profit and nonprofit management appear to become more effective as they gain in experience. That could easily account for the pattern of results reported here. Still, it is disconcerting to discover that impacts of non-profits compared unfavorably with those of the for-profits six years after the intervention began, presumably a long enough period for a new school manager to learn from experience.

Edison was asked to manage 20 schools; another 10 became the responsibility of two other for-profits. Sixteen schools would be managed by 4 nonprofits--the University of Pennsylvania, Temple University, Foundations, and Universal.

Paul E. Peterson, director of the Harvard Program on Education Policy and Governance (PEPG), is editor-in-chief of Education Next. Matthew M. Chingos is a PEPG research fellow.