Expanding college opportunities: intervention yields strong returns for low-income high-achievers.

Hoxby, Caroline ; Turner, Sarah

Ask any high school student in a well-heeled suburban community around the United States the best strategy for applying to college, and chances are you'll hear something like this: apply to several schools, most with students whose grades and test scores are similar to your own. But be sure to include one or two "safeties" at which admission is all but guaranteed and a couple of "reaches." And data on the colleges to which high-achieving, high-income students apply and that they attend suggest that they are paying attention.

The situation for low-income students appears to be quite different. The vast majority of even very high achieving students from low-income families do not apply to a single selective college or university. In other words, having worked hard in high school to prepare themselves well for college, they do not even apply to the colleges whose curriculum is most geared toward students with their level of preparation.

This is particularly puzzling because there are good reasons why many of these students should attend more-selective colleges. First, they are likely to succeed if they do. The high-achieving, low-income students who do apply are admitted, enroll, progress, and graduate at the same rates as high-income students with equivalent test scores and grades. Second, taking into account financial aid, low-income students generally face lower net costs at selective institutions than at the far less-selective institutions with fewer resources that most of them attend (see Figure 1).

[FIGURE 1 OMITTED]

One potential explanation for this pattern of behavior is that high-achieving, low-income students do not have access to good information about college quality and costs. These students are quite dispersed throughout the country and are often the only high-achieving student or one of just a few such students in their school. Thus, their high school counselor is unlikely to have much expertise regarding selective colleges and likely to be focused on other issues. Nor are recruiting visits to their high school or community likely to be cost-effective for college admissions staff. Moreover, it is often the case that neither parents nor other trusted adults are able to fill the deficit in information about college quality and costs for high-achieving low-income students. In short, traditional information channels may bypasss, low-income students, even if counselors and admissions staff conscientiously do everything that they can for these students.

Many low-income students may therefore be poorly informed about their college opportunities or deterred by apparently small barriers such as the paperwork required to request a waiver for application fees. Although a great deal of relevant information is available on the Internet, it is not easy for an inexperienced student to distinguish reliable sources of information on college admission standards, curricula, and net costs from the numerous unreliable (sometimes egregiously misleading) sources that are also online. Furthermore, many available information sources assume that low-income students are low-achieving and offer guidance that reflects this assumption. Because high-achieving, low-income students are atypical, these materials, aimed at students who are at the margin of attending any college, will provide little assistance.

For this study, we designed an experiment to test whether some high-achieving, low-income students would change their behavior if they knew more about colleges and, more importantly, whether we can construct a cost-effective way to help such students realize their full array of college opportunities. We do so by randomly assigning interventions that provide different types of information to roughly 18,000 students, including 3,000 students who serve as controls. The most comprehensive form of the intervention, which we call the Expanding College Opportunities-Comprehensive (ECO-C) Intervention, combined application guidance, semicustomized information about the net cost of attending different colleges, and no-paperwork application fee waivers.

The ECO-C Intervention costs just $6 per student, yet we find that it causes high-achieving, low-income students to apply and be admitted to more colleges, especially to more of those with high graduation rates and generous instructional resources. The students who receive the ECO-C Intervention respond to their expanded opportunities by enrolling in colleges that have students with stronger academic records, more instructional resources, and higher graduation rates. Their first-year grades in college are as good as those of the control students, despite the fact that the control students attend less-selective colleges, where the other students' preparation for college is substantially inferior to their own.

The Expanding College Opportunities Project

We designed the Expanding College Opportunities Project to test several hypotheses about why most high-achieving, low-income students do not apply to and attend selective colleges. The application guidance component of ECO-C provides the kind of advice that an expert college counselor would give a high-achieving student. An expert counselor would advise such a student to apply to eight or more colleges, including a combination of "safety," "match," and "reach" colleges. We call this group of colleges that are within an appropriate range for a given student's achievement "peer" colleges.

An expert counselor would also advise a student to obtain letters of reference; take college assessments on schedule; send verified assessment scores to colleges; write application essays; complete the Free Application for Federal Student Aid and the CSS Profile (an additional form required by many colleges that offer the most generous financial aid); and meet all other deadlines and requirements of selective colleges' applications. Finally, an expert college counselor would advise a student to compare colleges on the basis of their curricula, instructional resources, other resources (housing, extracurricular opportunities), and outcomes (such as graduation rates).

ECO-C includes application guidance along these lines and gives students timely and customized reminders about deadlines and requirements. It also provides students with comparative information on colleges' graduation rates and other resources tailored to where students live. The student is always presented with the graduation rates of his nearest colleges, his state's flagship public university, other in-state selective colleges, and a small number of out-of state selective colleges.

Even with this information, some students may focus unduly on colleges' "list prices" (the tuition and fees that an affluent student who received no aid would pay) and fail to understand that net costs for students like themselves are much lower. Many low-income students may not realize that they would generally pay less to attend colleges that are more selective and have richer instructional and other resources.

ECO-C therefore provides students with information about net costs for low- to middle-income students at an array of colleges. This information is again semicustomized in that a student always receives the list prices, instructional spending per student, and net costs of his state's public flagship university, at least one other in-state's public college, nearby colleges, a selective private college in his state, one out-of-state private liberal arts college, and one out-of-state private selective university. The net-cost information is shown for hypothetical families with incomes of $20,000, $40,000, and $60,000.

The net-cost materials are not intended to give a student precise information but, rather, to demonstrate the fact that list prices are often substantially greater than net costs, especially at selective institutions. The materials emphasize the importance of application as a student will not learn exactly how much a given college will cost him unless he applies. The net-cost materials also explain how financial aid works, emphasize how crucial it is to complete the FAFSA and CSS Profile on time, clarify how a student's Expected Family Contribution is computed, decipher a typical financial aid offer, and illustrate the trade-offs between loans, grants, and working while in college.

Finally, some low-income students may be deterred from applying to college by application fees. Such students may fail to realize that application fee waivers are available to them, or they may balk at filling out financial aid forms that will reveal their family income to a counselor. Or counselors may be too busy to do their part of the fee waiver process. ECO-C therefore provides students with no-paperwork fee waivers that allow them to apply to 171 selective colleges.

Data and Methods

In our main experiment, we randomly assigned each of 3,000 high-achieving, low-income 2011-12 high school seniors to the ECO-C Intervention and the same number of students to the control group. To be defined as high-achieving, we required that students score in the top 10 percent of test-takers on the College Board's SAT I or the ACT (1,300 math plus verbal on the SAT, 28 on the ACT).

We identified low-income students by combining student data from the College Board and ACT with data from an array of sources that allow us to estimate whether a student comes from a low-income family. We started with data that contain a student's SAT I or ACT scores, neighborhood, and high school. We then matched each student to 454 additional variables that describe the sociodemographics of his neighborhood, the sociodemographics and other characteristics of his high school, the history of college application and attendance among former students of his high school, the scores of former students of his high school on college assessments and statewide high school exams, and incomes in his zip code. We used all of this information to produce an estimate of each student's family income. We then focused our analysis on students with estimated family incomes in the bottom one-third of the income distribution for families with a 12th grader.

Finally, we exclude from our main analysis students who attended a "feeder" high school, which we define as one in which more than 30 students in each grade typically score in the top 10 percent on college assessment exams. We focused on high-achieving, low-income students from nonfeeder schools because we hypothesized (and the early data confirmed) that they would be more affected by the ECO-C Intervention than students who attend a high school with a critical mass of high-achieving students.

To study students' responses to the ECO-C Intervention, we obtained two sources of data on their application behavior, admissions outcomes, and college enrollment. First, we surveyed students each summer after they were selected for an ECO treatment or control group. Second, we collected information on their enrollment, persistence, and progress toward a degree from the National Student Clearinghouse. These data are reported by postsecondary institutions and cover 96 percent of students enrolled in colleges and universities in the United States.

In a large randomized experiment such as this, we can estimate the effect of receiving the intervention by simply comparing the average outcomes of the treatment and control groups. We present the results of these comparisons in two different ways. First, we present some "intent to treat" results that compare outcomes for the treatment and control groups, regardless of whether they actually experienced the intervention. Second, we discuss in full the intervention's effects on the 40 percent of students surveyed who could recall ever seeing ECO materials. We believe the latter results are more relevant for policy because a scaled-up version of the ECO-C Intervention would likely attract more attention from students and their families if it came from a widely known organization such as the College Board or ACT.

Effects of the ECO-C Intervention

The ECO-C Intervention has substantial effects on students' behavior at each stage of the process of applying to and enrolling in college. For example, we find that the ECO-C Intervention causes an increase of 19 percent in the number of applications students submit (see Figure 2). It increases by 22 percent the likelihood that they apply to at least one peer college, which we define here as a college with students whose median SAT scores are within 5 percentile points of the applicants' own scores.

Seeking the Right Match (Figure 2)

The ECO-C Intervention led high-achieving, low-income students to
apply and gain admission to more colleges and, more importantly,
to more colleges with similarly qualified students.

                                                 Percent increase

Number of Probability of applications submitted               48%

Probability of applying to a peer college                     56%

Number of colleges to which admitted                          31%

Probability of admission to a peer college                    78%

NOTE: The figure shows estimated treatment effects for students who
recalled seeing the ECO-C Intervention materials. Each estimate is
statistically significant at the 99 percent confidence level. Peer
colleges enroll students whose median SAT scores are within 5
percentile points of the applicant's own score.

SOURCE: Authors' calculations

Note: Table made from bar graph.

Still, these results likely represent a lower bound on the effectiveness of the program. Many of the students may have disregarded the mailings as they did not recognize the ECO organization. We expect that the effectiveness of the program would have been greater had the materials been distributed by a well-known organization such as the College Board or ACT. Indeed, based on our surveys, roughly 60 percent of students assigned to receive ECO intervention materials could not recall receiving them. To the extent that students disregarded the materials, the effects of the program were diminished. To correct for this, we perform what economists call a "treatment on the treated" analysis to produce estimates of the effects that a trusted organization such as the College Board or ACT would achieve were it to conduct the intervention. Thus, if a student could at least recall having seen ECO materials, the ECO-C Intervention caused her to increase the number of applications submitted by nearly 48 percent and be 55 percent more likely to apply to a peer college. In the text and figures that follow, we focus on the estimates that adjust for the likelihood of exposure to the materials.

Because the students targeted by the ECO program have high college assessment scores and grades, we expected that they would be admitted to more-selective colleges if the intervention did, in fact, cause them to apply to such colleges. This expectation was correct. Students receiving the ECO-C intervention were admitted to 31 percent more colleges and were 78 percent more likely to be admitted to a peer college.

It is not obvious that the ECO-C Intervention should have affected college enrollment outcomes simply because it affected the colleges to which students applied and were admitted. After all, a student might be willing to invest the time and effort to apply to a college in order to learn about it and the financial aid package it would offer. The same student might, upon receiving this information, decide that the college was, after all, not for him.

But the ECO-C Intervention did, in fact, alter students' enrollment decisions (see Figure 3). Students receiving the ECO-C materials enrolled in a college that was 46 percent more likely to be a peer institution, with a graduation rate 15 percent higher, instructional spending that was 22 percent higher, and student-related spending that was 26 percent higher.

... and Finding It (Figure 3)

Students who recalled seeing the ECO-C Intervention materials were
substantially more likely to enroll in peer institutions and in
institutions with higher graduation rates and more spending on
instruction.

                                              Percent increase

Probability of enrolling in a peer college                 46%

Graduation rate of college attended                        15%

Instructional spending at college attended                 22%

Student-related spending at college attended               26%

NOTE: The figure shows estimated treatment effects for students
who recalled seeing the ECO-C Intervention materials. Each estimate
is statistically significant at the 99 percent confidence level.
Peer colleges enroll students whose median SAT scores are within
5 percentile points of the applicant's own score.

SOURCE: Authors' calculations

Note: Table made from bar graph.

Finally, we test whether students who attended more-selective colleges as a result of the ECO-C Intervention struggle in the more demanding environment. Although it is too soon to address this issue definitively, our preliminary results provide little cause for concern: despite being in a more competitive environment, these students earn similar grades and persist to the sophomore year at similar rates to those of their peers who did not receive the ECO-C intervention and attended less-selective colleges.

More Experiments

In addition to our main experiment testing the ECO-C Intervention's effects on our target group of high-achieving, low-income students, we also used the same approach to study its effects on students who meet the same test-score criteria but who have estimated family income above the bottom one-third or attended a feeder high school. Although these students are outside our target group, this enabled us to test whether the effects of the ECO-C Intervention are different for the target students than for nontarget students. And, in fact, the results of this separate experiment confirmed that ECO-C generally had larger effects on our target group than on these other high achievers.

We also randomly assigned three groups of 3,000 students who met the criteria for our target group to receive just one of the three ECO-C components (application guidance, information on net costs, or fee waivers) rather than all three. This allowed us to test whether some parts of the ECO-C Intervention were more important than others. We found that the fee waivers tend to have larger effects on application behaviors, whereas the application guidance information tends to have larger effects on enrollment behaviors. The bottom line, however, is that the ECO-C Intervention as a whole tends to have larger effects than any of its parts. We therefore see no reason why an intervention based on our results should not incorporate all three components.

Costs and Benefits

The costs of the ECO-C Intervention are quite modest: approximately $6 per student to whom we sent materials. Because 60 percent of students could not recall looking at the materials (our minimal definition of treatment), the cost of actually treating a student was $15. We believe, however, that a reputable organization like the College Board or ACT could achieve a cost of treatment of approximately $6 simply because mail from such an organization would likely be opened and at least cursorily reviewed. Such an organization would presumably also have lower mailing and in-house printing costs than our small experimental organization had.

Even without those advantages, the benefits our intervention produced far exceeded its costs. For every $10 we spent, the ECO-C Intervention caused students to apply to four more colleges and to be 51 percentage points more likely to apply to a peer college. The same $10 caused students to enroll in colleges where graduation rates were 13 percentage points higher, instructional spending was $5,906 greater, and median SAT scores were 65 points higher. A growing body of evidence suggests that these differences in college quality will translate into substantial differences in the college graduation rates and lifetime earnings of the students who received the ECO-C Intervention.

The most prominent alternative strategy for influencing college-going behavior of low-income students, in-person counseling, typically costs upwards of $600 per student. Thus, in order to be as cost-effective as the ECO-C Intervention, such interventions would need to have effects that are at least 100 times as large. Needless to say, no existing in-person counseling interventions have been demonstrated to have this sort of impact.

It is worth noting that the ECO-C Intervention is likely a much more cost-effective means of changing students' college-going behavior than reducing the cost of college through tuition reductions, grants, and other forms of aid. Importantly, the successful provision of information related to college choice through initiatives like ECO-C is likely to magnify the return to existing federal and state aid policies, while the return to high-cost interventions such as expanding the Pell grant program is likely to be very limited unless students possess sufficient information about college alternatives.

Conclusions

Using random assignment of thousands of students, we successfully demonstrated that a low-cost, fully scalable intervention can help many high-achieving, low-income students recognize their full array of college opportunities. The ECO-C Intervention leads students to apply to and enroll in colleges with higher graduation rates, greater instructional resources, and curricula that are more geared toward students with very strong preparation like their own. Put another way, the ECO-C Intervention closes part of the college-going behavior "gap" between low-income and high-income students with the same level of achievement. The high-achieving, low-income students who are induced to attend more-selective colleges do not earn lower grades than they would if they had enrolled at the less-selective colleges attended by the control students. Under any reasonable assumptions about the value to these students of attending a more-selective college, the benefits of the ECO-C Intervention far exceed its costs.

The social benefits of the ECO-C Intervention are harder to define in dollar terms, but they are the benefits associated with increased income and sociodemographic mobility for high-achieving students from low-income families. For instance, such students may "pave the way" to selective colleges for other students from their high schools or neighborhoods. Or, such students may inspire other low-income students to study more because their experience makes the benefits of high achievement more salient.

We are often asked why some large-scale intervention akin to the ECO-C Intervention does not already exist. Our answer is twofold. First, the database capabilities that power the intervention (but are extremely inexpensive per student) did not always exist. Second, no one postsecondary institution has the incentive to implement such an intervention, since many of the benefits would accrue to its competitors. That is, the benefits ECO-C Intervention produces are largely of a public nature. Thus, a natural host for such an intervention would be a consortium of colleges and universities or a related organization with social goals.

Caroline Hoxby is professor of economics at Stanford University. Sarah Turner is professor of economics and education at the University of Virginia.