Learning from live theater: students realize gains in knowledge, tolerance, and more.

Greene, Jay P.

AS SCHOOLS NARROW THEIR FOCUS on improving performance on math and reading standardized tests, they have greater difficulty justifying taking students out of the classroom for experiences that are not related to improving those test scores. Schools are either attending fewer field trips or shifting toward field trips to places they know students already enjoy. When testing is over, schools are often inclined to take students on "reward" field trips to places like amusement parks, bowling alleys, and movie theaters.

The nature of culturally enriching field trips is that they are often to places that students don't yet know they might enjoy. In a previous study, we examined the impact of field trips to an art museum. We found significant benefits in the form of knowledge, future cultural consumption, tolerance, historical empathy, and critical thinking for students assigned by lottery to visit Crystal Bridges Museum of American Art (see "The Educational Value of Field Trips," research, Winter 2014). In the current study, we examine the impact of assigning student groups by lottery to see high-qullity theater productions of Hamlet or A Christmas Carol. This is the first randomized experiment to discover what students get out of seeing live theater. Our results are generally similar to those found in the previous study. Culturally enriching field trips have significant educational benefits for students whether they are to see an art museum or live theater. Among students assigned by lottery to see live theater, we find enhanced knowledge of the plot and vocabulary in those plays, greater tolerance, and improved ability to read the emotions of others.

Our goal in pursuing research on the effects of culturally enriching field trips is to broaden the types of measures that education researchers, and in turn policymakers and practitioners, consider when judging the educational success or failure of schools. It requires significantly greater effort to collect new measures than to rely solely on state-provided math and reading tests, but we believe that this effort is worthwhile. By broadening the measures used to assess educational outcomes, we can also learn what role, if any, cultural institutions may play in producing those outcomes.

Research Design

The opportunity to study the effects on students of seeing live theater arose as part of a collaboration with TheatreSquared, an award-winning professional theater in Fayetteville, Arkansas. TheatreSquared agreed to add matinee performances of A Christmas Carol and Hamlet, and school groups in grades 7 through 12 were offered the opportunity to receive free tickets to one of those performances. A total of 49 school groups, with 670 students, completed the application process and participated in the study. Twenty-four of those groups applied to see A Christmas Carol, and 25 applied to see Hamlet. Applicants were organized into 24 matched groupings based on their similarity in terms of grade level, demographics, and whether they comprised a drama, English, or some other type of class. Lotteries were held within each of those matched groupings to determine which groups would receive the free tickets to see a play and which would serve as the control group. A total of 22 school groups attended one of the performances, and 27 were in the control group. As one would hope from a lottery-based research design, the resulting treatment- and control-group students are generally alike in terms of gender, race, and the grade in which they are enrolled.

During the winter and spring of the 2013-14 school year, surveys were administered by members of the research team to 330 treatment- and 340 control-group students. The information was collected from each matched grouping on average 47 days after the treatment students in that matched grouping saw the play. Of the students who completed the consent forms necessary to participate in the study, we collected surveys from 78 percent in the treatment group and from 79 percent of students in the control group.

The survey collected background information on students as well as a number of important outcomes. Each survey contained a series of items to assess student knowledge of the plot and vocabulary used in the plays. There were also several items to measure student tolerance as well as student interest in viewing and participating in live theater.

We also employed a measure known as the Reading the Mind in the Eyes Test (RMET), which captures the ability to infer what other people are thinking or feeling by looking at their eyes. The test was developed by British psychologist Simon Baron-Cohen and his colleagues as a tool for studying theory of mind, particularly for people with autism. It is now widely used by researchers interested in studying theory of mind and empathy for people developing typically, as well as for those with autism. Researchers using RMET have found that reading literary fiction or engaging in theatrical role-playing enhances people's ability to read the emotions of others. We suspected that watching live theater might have a similar effect and decided to include RMET in our survey. The version of RMET we employed was developed for use with adolescents and has 28 photographs cropped to show only people's eyes. Subjects are asked to pick one of four words that best describes what the photographed person is thinking or feeling.

[ILLUSTRATION OMITTED]

All of the scales used to measure student knowledge about the plays, tolerance, ability to read the emotions of others, as well as interest in watching or participating in live theater, are either established or were validated with conventional tests of scale construction. All of the results reported below are based on analyses that control for student grade level, gender, and minority status, and compare students only within each matched grouping, while taking into account the fact that students within a given group are likely to be similar in ways that we are unable to observe.

Knowledge

Among students who are assigned by lottery to see live theater, knowledge of the plots of those plays as well as the vocabulary used in those productions is significantly enhanced, above and beyond what they learn by reading those works or by seeing film versions (see Figure 1).

Gains from Live Theater (Fig 1)

Students who see live theater become more knowledgeable of the plot
and vocabulary of the plays, more tolerant, and better able to read
the emotions of others.

                                  Knowledge  Tolerance  Reading other
                                                           people's
                                                           emotions

Eftect of theater field trip        63**      26**         23**

Effect adlusted for having read     58**      31**          21*
or watched a movie of the play

* Effect is statistically significant at the 90% confidence level
** Effect Is statistically significant at the 95% confidence level

Source: Authors' ca1cuIatuns

For each play we asked students six questions about the plot and five questions about the vocabulary used. We combined those 11 items into a single scale measuring a student's knowledge of the plays. Students assigned by lottery to see the live productions improved their knowledge of those plays by 63 percent of a standard deviation, a dramatic increase.

It may be easier for readers to grasp the nature and magnitude of this knowledge result by describing some of the changes produced on some of the individual plot and vocabulary questions we asked. For example, we asked Hamlet students, "Who are Rosencrantz and Guildenstern?" and 83 percent of the students who were assigned by lottery to see the play could correctly identify them as Hamlet's friends, compared to 45 percent of the control group. Of the students who saw A Christmas Carol, 88 percent could correctly identify Jacob Marley as Ebenezer Scrooge's deceased business partner, compared to 66 percent of the control group. More than 94 percent of the treatment group knew that Ophelia drowns in Hamlet, compared to 62 percent of the control group. Of those who saw A Christmas Carol, 93 percent knew that humbug meant "nonsense or a trick" compared to 62 percent of the control group, and 66 percent knew that destitute meant "very poor," compared to 50 percent of the control group. And 71 percent of students who saw Hamlet knew that idle meant "not working, active, or being used," compared to 61 percent of the control group.

Even the control-group students did much better than chance in picking the correct answer out of four multiple-choice options for each question. Our sample contained a large number of drama and Advanced Placement (AP) English students, whose knowledge of the plots and vocabulary from these plays may have been relatively high even without seeing a live production. But another possible explanation for the relatively high amount of knowledge among the control group is that almost one-quarter of them had read Hamlet or A Christmas Carol or watched movie versions of those stories for school that year. An even higher rate of the treatment groups, 52 percent, reported having read or watched movies of Hamlet or A Christmas Carol for school that year. Teachers who knew their students would attend the play were almost twice as likely to assign students to read or watch movies to prepare for the theater field trip.

[ILLUSTRATION OMITTED]

This raises the question of whether treatment-group students acquired so much knowledge about the plays because they saw the live theater performances or because they had been prepared by reading and watching movies. Of course, student groups were not randomly assigned to read or watch the movies, so we can't have the same confidence in identifying causal relationships, but we can use information about reading and watching movies to try to separate the extent to which the benefits we observed were produced by seeing a live theater production, or by having read and watched movies of those same works in school.

It is very clear that reading or watching movies of Hamlet and A Christmas Carol cannot account for the increase in knowledge students experienced by winning the lottery to see the plays. Even when we control for watching the movie or reading the material for school, the estimated effect of winning the lottery to see the plays remains basically unchanged, producing an effect size of 58 percent of a standard deviation for the treatment group on knowledge of the plot and vocabulary of the plays. Students who were assigned to read Hamlet or A Christmas Carol did no better on tests of their knowledge of the plays than did other students. Watching the movies for school is associated with about half of the benefit (30 percent of a standard deviation) as seeing the live theater performance. If teachers want students to learn plays, it is much better for them to take students to a live theater performance than to have them read the material or watch a movie. Plays are taught best by seeing them performed live.

Tolerance

We hypothesized that culturally enriching field trips are broadening experiences that expose students to a diverse world populated with different people and ideas, making them more aware and accepting of those differences. As part of the previous study in which we randomly assigned school groups to tour an art museum on a field trip, we found a significant increase in tolerance among students who toured the museum. To test whether field trips to see live theater have a similar effect, we utilized a scale that measures tolerance of others.

Students assigned by lottery to see A Christmas Carol or Hamlet scored significantly higher on our tolerance measure than did the control-group students. The difference is a little more than one-quarter of a standard deviation (26 percent). To put this effect in context, consider that students were asked the extent to which they agreed or disagreed with the following statement: "Plays critical of America should not be allowed to be performed in our community."

If students won the lottery and went on the field trip to see the plays, only 9 percent agreed that plays critical of America should be forbidden, compared to 21 percent of the control group. Students were similarly asked the extent to which they agreed or disagreed with the statement: "People who disagree with my point of view bother me."

Only 22 percent of the treatment-group students agreed with that statement, compared to 30 percent of the control group.

There is something about cultural experiences that seems to promote tolerance. Is it possible that reading or watching movies of these stories account for the benefit we observed? Again, we reanalyzed our data controlling for students having been assigned to watch or read these works for school. Students who read A Christmas Carol or Hamlet for school were no more tolerant than students who had not done so, and students who watched the movies were actually somewhat less tolerant. Controlling for reading and watching movies strengthens slightly the estimated benefit of seeing live theater on tolerance to an effect of 31 percent of a standard deviation.

If it is true that seeing live theater increaser tolerance, then past theater exposure should also be positively related to tolerance. We randomly assigned student to only this theater experience, not past ones, so our confidence in making any causal claims about the effects of past experiences should be lower. Nevertheless, it would support our finding that being randomly assigned to see this show improved tolerance if past theater experiences were also positively related.

We do not have a direct measure of the cumulative exposure students have had to theater, but it is reasonable to assume that students who are more interested in live theater have also had more exposure to it. We did ask a series of items about students' interest in seeing live theater. As we discuss below, attending a play had no impact on interest in seeing live theater, so this measure is telling us about differences in students' interest in live theater independent of our experiment. When we modify our analysis to control for interest in theater, the benefit of seeing A Christmas Carol or Hamlet does not change much (22 percent of a standard deviation), but we do find that interest in seeing theater (our proxy for past exposure) is strongly related to tolerance. A one-standard-deviation increase in theater interest is associated with an increase of 37 percent of a standard deviation in tolerance. This evidence suggests the plausibility of our finding that being randomly assigned to go on a field trip to see live theater increases tolerance. We had a similar finding in our earlier study of field trips to an art museum.

Reading Other People's Emotions

Seeing live theater may be particularly beneficial in teaching students to recognize the emotions of others. Theater works best when the actors effectively convey to the audience what their characters are thinking and feeling. The intensity of that experience may provide the audience with practice in reading emotions that is not normally found in everyday experience. Some earlier research also suggested that literary and acting interventions are effective at increasing people's ability to read the emotions of others. We expected the same might be true of the students seeing live theater in our experiment, so we administered the youth version of the Reading the Mind in the Eyes Test (RMET).

Students who won the lottery to go on a field trip to the theater scored significantly higher on RMET than did the control group. Even the control group scored fairly well on the test, correctly identifying the emotions portrayed in the photographs of eyes 71 percent of the time. But the students who saw A Christmas Carol or Hamlet could correctly identify emotions 73.4 percent of the time. That translates into an increase of almost one-quarter of a standard deviation (23 percent).

Having read or watched movies of these works, in contrast, was not associated with students' ability to read the emotions of others. And adding controls for reading or watching films of these works did not substantially change the estimated effect of seeing the live performances. The intensity and immediacy of live performance appears to have conveyed this ability to recognize what other people are thinking and feeling in a way that watching a movie or reading a text could not.

We collected the RMET measure, on average, 47 days after treatment, which suggests more than a fleeting benefit. To test whether past theater experiences are also positively related to the ability to read the emotions of others, we ran another model controlling for interest in seeing live theater as a proxy for the cumulative effect of past theater exposure. Doing so does not change the estimated benefit of seeing A Christmas Carol or Hamlet, but it does show that past theater experience is also significantly and positively related to the ability to read the emotions of others. A one-standard-deviation increase in interest in live theater (as a proxy for past theater exposure) is associated with an increase of 9 percent of a standard deviation in the RMET score.

While our proxy for past theater experiences was not randomly assigned and cannot be used to make causal claims, the fact that past theater is positively related to the ability to recognize emotions helps confirm the plausibility of our finding. Seeing live theater seems to teach students how to be better at reading other people's emotions, a quite useful skill.

[ILLUSTRATION OMITTED]

Participation and Consumption

It is also important to note outcomes that we thought might be affected by winning the lottery to see live theater but that produced no significant effects. In particular, we suspected that seeing live theater might inspire students to become more interested in either participating in future theater productions or going to view them. We found no evidence of this. Students who saw A Christmas Carol or Hamlet were no different in their desire to participate in or view theater than those who did not see a live performance.

The lack of an effect on student interest in viewing theater differs from the findings from the art museum study. In the earlier study, students who toured an art museum expressed significantly stronger interest in future museum attendance and actually returned to the museum at higher rates than did the control group. But on closer examination, the results are not actually so different. The benefit in the earlier field-trip study was concentrated in students who lacked previous cultural experiences, specifically younger, rural, minority, and low-income students as well as those who had not previously been to the art museum. More-advantaged students showed no significant benefit in the art museum study in terms of their interest in future museum consumption. The subjects in the TheatreSquared study are similar to the more-advantaged students in the art museum study. They are older, on average in 9th grade, and many were in drama or AP English classes (particularly for Hamlet, since that play is often read for AP English). These students already had a fairly high prior exposure to live theater, so it is possible that the marginal benefit of this one experience on interest in theater consumption is not strong enough to be detectable.

Limitations

Several groups had to cancel their attendance of A Christmas Carol because their schools were closed by snowstorms on the days of the performances. In the results presented above, we have treated school closures caused by weather as random events and reassigned those schools that did not see the play to the control group. An alternative way to handle these missed performances is to leave those students in the treatment group, but to adjust the results for the fact that not all treatment-group students actually attended the play as intended. Doing so produces basically the same estimated effects, with the exception that the effect of live theater on tolerance falls short of being statistically significant. We provide these results and a more detailed explanation for why they are not our preferred analysis in the methodological appendix available on the Education Next website.

[ILLUSTRATION OMITTED]

Because the statistical power of this experiment is driven by the number of school groups, not the number of individual students, we are unable to conduct subgroup analyses to reveal how seeing a play may differently affect subsets of students. For example, with our limited number of school groups we cannot know whether minority students, female students, younger students, low-income students, or rural students receive different benefits from seeing live theater. Our ability to conduct these subgroup analyses is further constrained by the relative homogeneity of the students in our sample, with most being white and in advanced classes. However, this homogeneity does improve the similarity of students within our matched groupings, strengthening the overall results.

[ILLUSTRATION OMITTED]

We are unable to say much about how A Christmas Carol and Hamlet may have differed in their effects. Again, we have too few school groups. Also, school groups chose the play for which they applied, so we do not have the benefit of random assignment to make causal claims about how the two plays may have differed in their effects.

Conclusion

Culturally enriching field trips matter. They produce significant benefits for students on a variety of educational outcomes that schools and communities care about. This experiment on the effects of field trips to see live theater demonstrates that seeing plays is an effective way to teach academic content; increases student tolerance by providing exposure to a broader, more diverse world; and improves the ability of students to recognize what other people are thinking or feeling. These are significant benefits for students on specific educational outcomes that schools pursue and communities respect. Especially when considered alongside our previous experiment on field trips to art museums, this research shows that schools can draw upon the cultural institutions in their communities to assist in producing important educational outcomes. Not all learning occurs most effectively within the walls of a school building. Going on enriching field trips to cultural institutions makes effective use of all of a community's resources for teaching children.

Finally, this research helps demonstrate that schools produce important educational outcomes other than those captured by math and reading test scores, and that it is possible for researchers to collect measures of those other outcomes. If what's measured is what matters, then we need to measure more outcomes to expand the definition of what matters in education.

[ILLUSTRATION OMITTED]

by JAY P. GREENE, COLLIN HITT, ANNE KRAYBILL, AND CARI A. BOGULSKI

Jay P. Greene is professor of education reform at the University of Arkansas, where Collin Hitt and Anne Kraybill are doctoral students and Carl A. Bogulski is a researcher.