Subject area literacy instruction in low SES secondary schools in New Zealand.

Wilson, Aaron ; McNaughton, Stuart ; Zhu, Tong 等

The purpose of this study was to investigate patterns of student achievement and subject-area literacy teaching in a cluster of 22 New Zealand (NZ) secondary schools that serve low-to mid-socio-economic-status (SES) communities. We describe patterns of students' reading achievement in high stakes assessments in English, mathematics and science as well as patterns of literacy teaching across 104 Year 12 subject-area classrooms.

The wider context of the study is NZ's high quality but low equity compulsory education system (OECD, 2014). NZ students regularly rank well above the OECD average in mathematics, reading and science but there is a wider gap between high and low-achievers than most other countries (OECD, 2010; Vannier, 2012). Students from low SES backgrounds, Maori (indigenous) and Pasifika (first or second generation immigrants from Pacific island countries) are markedly overrepresented in the tail end of the achievement distributions (OECD, 2010). Maori and Pasifika ethnicities and low SES are also strongly positively correlated.

We were interested in understanding how patterns of literacy teaching in subject-areas might contribute to historic patterns of low achievement for students from this group of schools and in identifying teaching practices that could be developed to be more consistent with practices identified as effective in the literature.

The specific research questions were:

1. What are the patterns in low SES schools of student participation and achievement in high stakes English, mathematics and science assessments that have complex reading demands?

2. How does literacy instruction across different subject areas in low SES schools reflect increasingly specialised and sophisticated reading and writing challenges in the senior secondary school?

3. How does literacy instruction in low SES schools in different subject areas reflect currently known optimal practices?

Internationally, over the past two decades, there has been growing concern about an 'adolescent literacy crisis'. The crisis consists of high school students' reading achievement stagnating or declining, relative to previous cohorts, at a time when the literacy demands of education, work and society are rapidly increasing due to technological, social and economic changes (Jacobs, 2008; Moore, Bean, Birdyshaw & Rycik, 1999) such as the expansion of information-based technologies, the internationalisation of labour markets, and the dramatic decline in the number of jobs that do not require (much) reading or writing (Biancarosa & Snow, 2006; Kamil et al., 2008; Shanahan & Shanahan, 2008). The increasing importance of reading means that, regardless of whether overall levels of reading are actually declining, as they appear to be in the USA (Brozo, Moorman, Meyer & Stewart, 2013), or merely stagnating, as they are in New Zealand (Telford & May, 2010),

there is a growing gap between the literacy vision and the reality.

Leaving high school without the literacy needed to fully participate in workplace and life situations constrains the life chances of the individuals involved but having large numbers of students leave high school without advanced forms of literacy is also concerning from a community and national perspective. Particularly troubling is that some groups of students -notably minority, indigenous and students from poorer communities --are markedly overrepresented in the lowest achieving groups. Ethnicity and socio-economic based disparities in literacy outcomes are persistent features of the educational systems of the United States, the United Kingdom, Australia, New Zealand and other Anglophone nations.

Given the importance of literacy--and clear evidence that quality teaching has the potential to improve student learning (Alton-Lee, 2003; Hattie, 2008)--it is unsurprising that much attention has been directed toward better understanding the role that teaching can play in improving literacy outcomes. Historically, most of that focus was on understanding literacy teaching and learning in the elementary years but, more recently, attention has turned to instruction in the high schools themselves. This is largely because of a growing awareness that the literacy developed in the primary years is a necessary but not sufficient condition for literacy success in and beyond the senior schooling years; strong foundational literacy does not necessarily equip students for the changing and increasingly sophisticated demands of literacy in high school.

One set of changes students encounter in the move into and through secondary school is in the 'textual landscape': texts become longer; conceptual challenges increase; word, sentence and structural complexity increases; graphic representations become more important; and texts begin to vary widely across content-areas (Carnegie Council on Advancing Adolescent Literacy, 2010). What is more, students are expected to develop more sophisticated and critical readings of these increasingly complex texts using an expanded repertoire of knowledge, skills and strategies and in more deliberate ways. This is evident in English Language Arts curricula and related progression documents. Pearson (2013), in the USA, for example, identified five key progressions in the Core Common Standards for reading in English Language Arts. The progressions were in support (from more to less scaffolding), the number of entities involved in the reading process (e.g. from consideration of characters to consideration of characters, settings and events), the type of entities (e.g. from general to discipline-specific entities, from explaining entities to explaining relationships and interactions), cognitive demand (e.g. from description to explanation, from explanation to comparison) and evidentiary requirements. Similar progressions are evident in the New Zealand Curriculum. By Year 13 for example, students are expected to use a wide range of sources of information, processes and strategies 'purposefully, confidently, and precisely' to 'identify, form, and express increasingly sophisticated ideas' and to 'critically analyse' the ways that authors shape texts for different purposes and audiences (Ministry of Education, 2007, p. 44). Pearson (2013) notes that progressions such as those outlined have been developed using expert consensus, rather than empirical evidence that each stage does in fact precede the next. Nevertheless, it is clear that the literacy needed in the later years is much more complex than that required in the earlier years.

This complexity exists in English language arts curricula. But another way that literacy demands change is through increased subject-specialisation (Moje, 2008; Shanahan & Shanahan, 2012). Greater subject-specialisation becomes evident in the types of texts students are required to read and write and in the language, structure and forms of representation employed in those texts. There are also differences in the purposes students have for reading and writing in different subject areas. The complexity that exists within and between each subject means that the literacy challenges faced on any given day by high school students as they move between five or six different disciplines are considerable.

Effective literacy instruction

Given these developmental and curriculum demands, what do we know about the features of teaching that most effectively promote the development of advanced, subject-specialised, and increasingly independent forms of literacy for senior students? The existence of generic as well as more subject-specialised texts, text features, strategies and purposes implies a need to reteach or promote the generalisation of general aspects of reading and writing in the process of teaching in subject areas. There is also a need to teach the practices of reading and writing associated with specific subject areas (Lee & Spratley, 2010, p. 10).

Four types of learning opportunity that contribute to adolescent students' literacy development are identified in the literature, namely opportunities to: read and write appropriately challenging disciplinary texts; to learn about language and texts; to talk about texts; and to develop skills and strategies they can apply with increasing levels of independence to reading and writing tasks beyond the classroom.

Opportunities to read and write texts

Our assumption is that texts should be at the centre of literacy instruction and there is no skills-only approach to literacy instruction that can substitute for extensive reading (Schoenbach, Greenleaf, Hurwitz & Cziko, 1999). Although there has been limited research about how texts are or should be used in subject-area classrooms (Moje et al., 2011), repeated practice is an important principle of learning in general (Bransford, Derry, Berliner, Hammerness & Beckett, 2005; Darling-Hammond & Bransford, 2005) and reading mileage is associated with reading fluency and, in turn, with comprehension (Allington, 2014). Previous studies indicate that teachers sometimes avoid using complex written texts with groups of students who struggle, or who are perceived to struggle, with reading and opt instead to ameliorate reading challenges using modified texts and more direct forms of teaching (DiGisi & Willett, 1995). A range of evidence suggests that differential exposure to and engagement with texts are associated with cumulative achievement differences between low SES and culturally diverse groups (Guthrie et al., 2004). Tatum (2008) for example has described predominantly African-American students in some inner-city US schools as experiencing an 'in-school literacy underload'.

Learning about language and texts

The texts associated with individual subject areas have particular characteristics that present readers with different challenges and require subject-specialised as well as generic skills and knowledge. Some forms of representation, such as Punnett squares, poems and probability trees, will almost only ever be encountered in their 'home' subject of biology, English and mathematics respectively (Lee & Spratley, 2010; Moje,

Stockdill, Kim & Kim, 2011). Although other ways of representing information, such as in paragraphs of running prose, will be encountered in all subjects, the texts they are used within differ in their structures and organisational features. For example, scientific research reports tend to be non-continuous texts that incorporate abstracts, section headings, figures, tables, diagrams and running text in ways very unlike texts encountered in a typical English lesson (Lee & Spratley, 2010).

The vocabulary used in senior secondary subject texts is also highly specialised: consider pathogenesis in biology versus pathos in English versus polynomial in mathematics. Not only that, the relative impact of certain types of word on reading comprehension varies by subject. For example, insufficient attention to prepositions could be disastrous in mathematics if one were to transpose to and from when asked to calculate the time it took a person to travel from place A to place B (Cocking & Mestre, 1988). However, prepositions in English texts are more likely to be accompanied by other contextual information that could alert a student if such an error were made. On the other hand, having strategies for solving unfamiliar words may be more important in English because the range of vocabulary that students might encounter in a text in an external assessment, for example, is less constrained, less predictable and therefore less pre-teachable than it is in other subjects. That is, whereas students will encounter complex but non-subject-specific vocabulary in all subjects the relative number of non-subject-specific to subject-specific words is likely greater in the literary texts of English because the range of possible contexts is less constrained.

Different subjects also employ different language features at the level of sentences. For example, ellipsis (the omission of words, phrases, or clauses), the use of lengthy noun phrases, complex sentence structures, passive voice and nominalisation (the process by which verbs and adjectives are turned into nouns) have been identified as common features of science texts (Fang, 2006; Fang & Schleppegrell, 2010). Nominalisation is not as common a feature in subjects such as English or history because these subjects typically are very interested in human agents (Shanahan & Shanahan, 2012). Importantly, the common features of science language noted above were all identified in a recent study as characteristics of text that make comprehension more difficult (White, 2012).

Given the increasingly complex texts students are expected to read in the subject-areas, students need to develop knowledge about the complex vocabulary, structures and language features of specialised text types. Deliberate acts of teaching, including direct teaching, modelling and activities clearly are warranted, but much of this learning will likely occur incidentally as one reads and writes and talks about subject texts (Fang, 2006).

Talking about texts

Increasingly researchers claim that one of the most powerful ways to raise students' subject-literacy is through immersion in disciplinary rich language; for students and teachers to exchange ideas in more 'dialogic' open-ended discussions in ways that improve understanding and interpretation of texts (Wilkinson & Son, 2011). What the different dialogic approaches to comprehension development have in common is more time for open-ended discussion, greater use of authentic open teacher questions to explore rather than 'test' students' understanding, and attempts to increase 'uptake' whereby teachers prompt for elaboration and incorporate and build on students' ideas (Applebee, Langer, Nystrand & Gamoran, 2003). All of the approaches seek to disrupt the more typical recitation pattern of classroom talk known as Initiate-Respond-Evaluate (I-R-E) whereby teachers initiate discussions by asking question they already know the answer to, students respond, and the teacher evaluates responses against predetermined criteria of correctness (Applebee et al., 2003).

Dialogic talk may be particularly important in classrooms where there are students whose home cultural and language practices are not well matched with the canonical literacy and language practices of western classrooms. So for example in New Zealand classrooms with many students from the indigenous communities (Maori) or from Pacific islands cultures this kind of talk, as opposed to I-R-E pattern, can authorise and value students' own funds of knowledge (Moll, Amanti, Neff & Gonzalez, 1992) and represents a shift in power from a traditional transmission approach where the teacher's funds of knowledge are privileged toward ako, the Maori concept of a reciprocal process in which teaching and learning roles are shared (Bishop, Berryman, Wearmouth & Peter, 2012).

However despite its recognised pedagogical potential, teacher dominated talk is still the default discourse pattern in the majority of secondary (and primary) classrooms and teachers find it very difficult to change classroom discourse patterns, even when they want to (Alozie, Moje & Krajcik, 2010; Groenke, 2010).

Developing strategies

Understandably, teachers' immediate concern in a classroom context is to support students to read the particular text they have in front of them at that moment. To be effective in the long term though, literacy instruction must not only support students to read the text encountered in class in a particular lesson but to develop reading and writing knowledge and strategies they can apply in future-situations where access to support will be limited, such as in examinations, university studies and work places. Some efforts to 'scaffold' literacy in subject areas may inadvertently exacerbate students' literacy issues by removing cognitive challenge, constraining students' own responses and strategy use, and by fostering dependence and learned helplessness (Athanases & de Oliveira, 2014).

One approach for developing students' independent reading is teaching cognitive literacy strategies, defined as constructive interactions with texts in which good readers and writers continuously create meaning (Pressley, 2006). Well known cognitive strategies include asking questions about texts, making predictions, activating prior knowledge and summarising (Conley, 2008; Moje, Overby, Tysvaer & Morris, 2008). Pressley (2004) suggests that effective teaching of such strategies begins with a teacher modelling and explaining strategies, and then providing opportunities and some teacher scaffolding for students to practice the strategies. Extensive deliberate strategy instruction may have unintended effects such as that overemphasising strategies may be to the detriment of students' engagement in meaningful dialogue about text (Wilkinson & Son, 2011) and students in schools serving low SES communities may be more at risk of display type exchanges in which the focus is strategies (McNaughton & Lai, 2009).

Another approach for developing student independence in reading involves metacognitive approaches where students reflect on and discuss the processes and strategies they used themselves to identify and solve problems encountered whilst reading (Schoenbach, Greenleaf & Murphy, 2012).

Given that it is one of the key reading progressions, deliberate teaching to develop critical literacy is also warranted. Practices identified include teachers encouraging students to read from a resistant perspective, setting tasks where students produce counter texts, providing multiple perspectives on the same topic and conducting student-choice research projects (Behrman, 2006; Janks, 2013).

Summary

This review of the developmental features of literacy in the upper secondary and the evidence base for how to optimise development through instruction suggests two important conclusions. One is that optimising development requires effective literacy instruction to be in place across different content areas. The domain specificity of some aspects of literacy indicates that there would likely be limited transfer from generic ELA based programs to other content areas. To best understand this conclusion requires knowing details of the content area requirements, for example as enacted in assessments. The limited research base would indicate that literacy is best developed and enacted in the service of acquiring disciplinary expertise (Pearson, 2013). The second conclusion is that under best circumstances similar properties of instruction should be present across the content areas in order to optimise development.

These conclusions have led us to consider whether the quality of instruction, not just within ELA departments but within and across content areas, contributes to the systematic underachievement of some groups of students in the upper secondary school. Students at these schools from Maori and Pasifika families and low SES communities typically are at risk of lower achievement than peers in other schools. In so doing we also contribute to the wider evidence base of the nature of literacy instruction in the upper secondary school. The particular concern of this question is the instructional milieu experienced by students from culturally and linguistically diverse students and from low income communities, who may need even more high quality instruction to accelerate rates of learning.

Methods

The 22 schools in the study served low- to mid- socioeconomic communities in Auckland and Tai Tokerau (Northland). State schools in New Zealand are divided for funding purposes into decile groupings which indicate the extent to which the school draws its students from socio-economically deprived households. Decile 1 schools are the 10 percent of schools with the highest proportion of students from poor households, and conversely, decile 10 schools are the 10 percent of schools with the lowest proportion of such students (Ministry of Education, n.d.). Of the 22 schools, eight were decile 1, two decile 2, five decile 3, four decile 4, and three decile 5. Overall, the schools had higher proportions of Maori and Pasifika Year 12 students than schools nationally (40% Pasifika and 24% Maori on average for the four years versus 9% and 18% respectively nationally) and fewer New Zealand European students.

Measures of literacy participation and achievement: NCEA standards

Since 2002, the National Certificates of Educational Achievement (NCEA) have been the main qualification for students in state schools in NZ. To gain a NCEA qualification students are assessed against a range of 'standards' in different subjects. Each standard represents a particular skill, understanding or competency and is worth a specified number of credits which, if achieved, counts towards the 80 credits required for a national certificate at that level (New Zealand Qualifications Authority, nd, a). We analysed roll-based attainment rates for each of the NCEA levels (1, 2 and 3) and for University Entrance to investigate overall patterns of achievement. The roll-based attainment rate renders the number of students who achieved the qualification into a percentage of total number of Year 12 students.

We also selected a sub-set of 'high literacy' standards to serve as indicators of students' disciplinary reading, for each of the subjects. These standards were selected on the basis that they were externally assessed and had been identified by independent subject experts as required sophisticated reading to achieve. The subject experts had been contracted by the New Zealand Qualifications Authority (NZQA) to lead professional development nationally when the standards were introduced and were therefore assumed to have expert knowledge of the different standards. Externally assessed standards were used because these are indicative of the level of learning and independent disciplinary reading students need to attain in order to be able to succeed in more advanced study and could be more readily benchmarked against national norms. In order to illustrate the complex specialised reading demands, extracts from recent assessments of some of the standards are presented in Table 1.

We analysed participation rates and achievement rates in the high literacy standards for the four years prior to the classroom observations. This helped to answer question one, and contributed evidence for the first feature of instruction, access to reading and writing texts. It was important to analyse participation rates in the high literacy standards because of the flexibility inherent in NCEA. In each subject, subject leaders select standards to assess students' skills and knowledge from a larger pool of possible standards. In English, for example, students are typically entered for only five or six of the ten Level 2 achievement standards schools can select from (New Zealand Qualifications Authority, n.d., b). This flexibility allows schools to be responsive to the needs and interests of their students but it also creates complexity and the possibility for systematic differentiation at both structural and pedagogical levels, within and between schools (Madjar, McKinley, Jensen & Van Der Merwe, 2009).

We calculated the total number of students entered for each achievement standard as a percentage of the total Year 12 roll in the 22 schools to establish a participation rate. We used the total number of Year 12 students as the denominator even though some Year 11 and 13 students are also enrolled in Level 2 standards (NCEA allows for multi-level study) because calculating participation as a percentage of the combined roll of all Year levels would have produced a misleadingly low rate as the vast majority of students who attempt the standard are in Year 12.

The participation-based pass rate comprised the total number of participants who achieved the standard relative to the total number of participants enrolled in it. Because the denominator for the calculation of participation-based pass rates is the total number of students enrolled in that standard rather than the total number of students on the roll, participation-based pass rate data should be interpreted cautiously and with reference to the participation data. A school could, for example, improve its pass rate in a particular standard by excluding those students most at risk of failing, that is, by reducing its participation rate.

We introduced the time measure of 'years prior to literacy observation' because the 22 schools were from two cohorts who entered the study in different years: Group A schools were observed in 2011 and Group B in 2012. We first computed differences between achievement rates in the group of schools and national achievement rate for each calendar year, and then consolidated the differences by years prior to the literacy observations in proportion to the number of participants in the school group. For example, difference in achievement rate at one year prior to the literacy observations was computed as the weighted average of difference between the achievement rate in Group A schools and the national achievement rate in 2012, and difference between achievement rate in Group B schools and national achievement rate in 2013. Differences in participation rate at each year prior to the literacy observations were calculated in the same fashion.

We calculated the difference between the participation and achievement rates in the schools, and national participation and achievement rates for each of the years because, being standards-based, national NCEA achievement rates differ both for different qualifications / standards within the same year and for the same qualifications / standard across different years. Because of our focus on equity, the national rates we used as benchmarks were for all students; we did not, for example, compare the achievement rates of Maori students in the study to Maori students nationally or students in decile 1 schools in the study to students in decile 1 schools nationally. Our analyses of the NCEA data over time were primarily visual analyses of plotted line graphs. We examined graphed data visually to judge rates over time, to see if any changes had occurred, and what the magnitude, trend, latency and reliability of any changes were (Neuman & McCormick, 1995, p. 19).

Classroom observations

The literacy instruction data were derived from observations of 104 teachers working with Year 12 (approximately 17 years old) students comprising 28 biology, 39 English and 37 mathematics teachers. There were fewer observations in biology because not as many students take Level 2 biology as English and mathematics and fewer classes are offered. Each teacher was observed for one lesson of about 50 to 60 minutes duration. The purpose of the observations was to provide a snapshot of literacy teaching practices for a relatively large sample of teachers. Although a single observation would not be sufficient to make sound generalisations about the literacy teaching of a single teacher or a small group of teachers over time, we judged that a single observation of many teachers was more appropriate for our purpose than conducting several observations of fewer teachers. A possible limitation of the study was that the observed teachers were volunteers who had been informed beforehand that the focus of the observations was literacy teaching. We believed it ethically important to disclose the focus of the classroom observations to potential teacher participants and to respect their right to participate, or not participate. We acknowledge that the literacy teaching observed may well have been different if the teachers had not known that literacy was a focus of the observations and/or if we had observed non-volunteers as well. We predict that this would tend to overestimate the frequency and quality of literacy teaching but, of course, we have no way of testing that prediction. There was no attempt to observe predetermined content or teaching related to particular NCEA Level 2 achievement standards as the knowledge and skills that are assessed in the end-of-the-year examinations depend on learning across the year and across a range of learning experiences. Observations were conducted by three research assistants and the first named author, and took place in the second and third terms (in a four-term year which starts in February and culminates for Year 11-13 students with NCEA examinations which occur in the fourth term). Each teacher was observed using a tool designed to record instances of literacy teaching (Wilson, Jesson, Rosedale & Cockle, 2012). The observation tool is included as an appendix from the full report of a previous study downloadable here: https://www.educationcounts.govt.nz/ publications/series/Secondary_Literacy/Literacy_and_ Language_Pedagogy. In each teaching period, threeminute blocks of observation of the teaching alternated with three-minute blocks for data recording using precategorised codes as well as descriptive open recording. Thus a 60-minute lesson, for example, resulted in 10 three-minute blocks of observational data providing a 50 percent random sample of teaching within that lesson.

Observational data included details about teaching activities, how students were grouped, forms of differentiation, and the focus and characteristics of any literacy instruction. Observers photographed or recorded details about texts so that analyses of text type, source, length and audience could be conducted later.

The observer made a judgement as to the main teaching activity in each block and coded this as being one of question and answer, extended discussion, lecture, modelling, conference, roving, or management.

The observer noted whether there was any instance of teaching about literacy and language in that block and, if so, whether the focus was on vocabulary, text structure, or language features including syntax, spelling, or punctuation. The observer also noted whether the instruction focused on activation of prior knowledge, developing item knowledge (e.g., teaching word meanings), providing practice (e.g., word-meaning matching activities), developing students' strategies (e.g., students reflecting on word-solving strategies, direct teaching of context- and morphemic-based, word-solving strategies), or an aspect of critical literacy (e.g., identifying bias or how the author has positioned the reader).

To develop reliability, each observer was paired with a more experienced observer for their first three to five observations. Any disagreements in the first two observations were resolved through discussion led by the more experienced observer. Observers continued to work in pairs until inter-rater checks determined high reliability of observations (> 90% inter-rater agreement). The unit of analysis for inter-rater agreement checks was each researcher's decision on the teacher observation tool, for each block, for each lesson that was co-observed. In total there were 26 separate decisions to make per block. Data recorded on the classroom observation templates were entered manually into SPSS, and descriptive statistics (percentages) for each variable, cut by subject, were extracted. Fisher's exact tests were used to test significances of differences in dimensions across subject areas and Clopper-Pearson intervals for Binomial proportion were used to estimate and compare confidence intervals across subject areas. Exact methods of statistical testing and parameter estimation were applied instead of approximated approach in order to accommodate the likelihood of small expected frequencies (less than five) in some dimensions of classroom observations (Mehta & Patel, 1983; Wallis, 2013).

Results

Attainment rates for NCEA Levels One, Two and Three, and University Entrance

Analyses showed that, despite some improvements over time, the rates at which qualifications were attained were markedly lower than for schools nationally (Figure 1). Attainment rates for Level Two and Three NCEA had been trending upwards relative to national attainment rates but were still about 10% lower than national rates. University Entrance had also improved but was still 15% below national in the most recent year.

Participation and pass rates for 'high literacy' standards

The participation rates (Figure 2) show that markedly fewer percentages of the students in these schools than students nationally were entered for and hence participated in the high literacy standards. The pass rates (Figure 3) for high literacy standards in mathematics and biology were also below national rates, despite participation rates also being much lower. Although pass rates for the aggregated English standards were between two and six percent higher than national pass rates over the four years, participation rates were consistently about 40% lower than national participation rates meaning that selective participation policies of schools likely contributed to the seemingly positive pass rates. In the year immediately prior to the observations, mathematics participation and pass rates dipped to 35% lower and 16% lower than national respectively.

In the three years prior to that participation rates had been steady at about 30% lower than national whereas pass rates had fluctuated from 3% to 7% lower than national. Participation rates for biology were steady at about 16% lower than national and pass rates were between 11% and 16% lower than national.

Patterns of literacy instruction

Table 2 summarises the opportunities students were observed to have for literacy learning in biology, English and mathematics classrooms. Three key findings emerge. Firstly, there was more variability between teachers than there was between subjects; few statistically significant differences between subjects were identified (Table 2). The texts in mathematics were significantly shorter than those in biology and English classrooms and there was more vocabulary instruction in biology and significantly less in English. Secondly, there was a general mismatch between the properties of texts students were observed to read in subject-area classrooms and those implied in the literature, the curriculum and in NCEA assessments. Thirdly, there were, overall, low rates of those instructional acts identified in the literature as effective in developing senior secondary students' literacy in the subject-areas.

Key findings about text-use, classroom talk and literacy instruction follow.

Texts

On average, across all lessons, students were provided with some form of written text to read in just over half of observed blocks. Although on average there were more opportunities to read in biology (63% of blocks) than in English (51%) or mathematics (56%), the differences were not significant. Table 3 summarises the characteristics of texts provided students in the observed blocks. Overall, the Year 12 students in the study had many opportunities to read notes, task sheets and PowerPoint presentations but few opportunities to read the extended, authentic, published texts implied by the literature, the New Zealand Curriculum, and required by NCEA assessments.

Despite expectations in the literature, the curriculum and in assessments that students need to apply their knowledge of mathematical and scientific concepts to authentic, real world situations, they had few opportunities to read published print or online texts. Rather, the majority of texts in all subjects were teacher-designed texts such as notes and task-sheets. Texts were generally short texts with a median length of 51-100 words in biology and English, and a significantly shorter length of 11-50 words in mathematics.

Dialogic approaches

There were very few instances of extended discussion observed. Although rates were higher in English (1.1% of blocks) than in biology (0%) or mathematics (0.3%) the differences did not reach significance. Teacher-centred approaches to whole-class teaching, in the form of lecturing, modelling and question-and-answer sessions, were much more prevalent and these were coded as the main teaching activity in 41% of blocks. This pattern was more common in mathematics (55 % of blocks) and less common in English (25% of blocks). Teacher conferencing with individual and small groups of students was more common in English classrooms (32%) than in mathematics (16%) or biology (20%).

Literacy instruction

Some form of literacy instruction was observed in over half of all blocks with higher, but not significantly different, rates in biology (59%) and English (55%) than in mathematics (40%). Of all those blocks that had any literacy instruction, vocabulary was the most common focus especially in biology and mathematics where, respectively, 90% and 74% of the literacy blocks related to vocabulary; in English 31% of the literacy blocks did. As mentioned previously, vocabulary was one of the only aspects where we identified significant differences between subjects with significantly higher rates in biology (53% of all blocks) and significantly lower rates in English (17% of all blocks). Instruction aimed at developing students' knowledge of text structure and language was very rarely observed outside English but, again, differences between subjects were not statistically significant. Finally, many more of the acts of literacy instruction were focused on developing students' item knowledge (for example, knowing word meanings) than they were on developing students' strategic knowledge or critical literacy.

Discussion

The NCEA data reported in this paper highlight inequitable literacy outcomes--both in participation and achievement--for students in this cluster of lower SES schools and for Maori or Pasifika students. Despite some improvements in the overall attainment rates of NCEA, the participation-based pass rates for the high literacy achievement standards over the four year period had not improved. The reason why there can be an increase in the proportion of students gaining qualifications--without an accompanying increase in students achieving the high literacy standards--relates largely to the flexibility schools have in deciding what standards to offer. Decisions by schools not to enrol particular students in high literacy standards are completely consistent with NCEA rules and regulations. Indeed, such decisions are rational and strategic in a qualification system and social, media and policy environment where the main focus is on overall attainment rates for NCEA at the qualification level, rather than on the individual standards that comprise those qualifications (Wilson, Madjar & McNaughton, 2016). One contextual feature that incentivises schools in this manner is the government target of 85 percent of all students gaining at least the NCEA Level 2 qualification by age 18 (State Services Commission, 2013). Another is that media outlets regularly publish school 'league tables' based on NCEA achievement rates. Improving students' subject-area literacy through more effective instruction will almost certainly prove more challenging than guiding students away from academically challenging externally assessed standards and toward internally assessed and vocational standards judged by the school to be easier for those students to achieve.

How, then, might the observed patterns of literacy instruction contribute to inequitable literacy outcomes for students in lower-SES schools? Although we identify aspects of literacy where arguably there is a need to increase rates of instruction, the overall patterns may be more positive than that implied in previous literature which has included seminal titles such as, 'Why content area literacy is difficult to infuse into the secondary school' (O'Brien, Stewart & Moje, 1995). In contrast, the pattern here does not support a general unwillingness by content area teachers to 'resist' literacy teaching. Rather, we found evidence that teachers on average presented students with some texts to read in about half of all blocks observed and also included a focus on language or literacy in between 40% and 59% of instructional blocks. While we certainly see a need to change the types of texts that students read in those blocks, and to expand the focus of literacy instruction from a focus on vocabulary item teaching to include a broader linguistic, strategic and critical focus, there is no evidence that subject teachers have ruled out a focus on literacy out of hand. This stance provides a strong starting point for efforts to transform literacy instruction so that classroom practices are more aligned by those warranted in the literature.

Our analyses do reveal mismatches between actual practice and those practices identified as optimal in the literature. Examples of these mismatches include: a mismatch between the properties of texts commonly observed in the classrooms and accessible-yet-challenging texts (Moje et al., 2011); low rates of specific literacy instruction and particularly strategy instruction (Conley, 2008); and very low rates of extended discussion (Wilkinson & Son, 2011), even in English classrooms. There was also a mismatch between the observed instruction and the requirements of high-stakes assessments (such as those illustrated in Table 1). There was little instruction about language features such as lengthy noun phrases and nominalisation that are typical of and complicate students' comprehension of science and mathematics texts (Fang & Schleppegrell, 2010; White, 2012). Given that students in low SES schools are more likely to experience social marginalisation and injustice, we are also concerned by the lack of critical literacy instruction (Janks, 2013).

The differences between subjects' means were mostly in the direction we expected (more in English less in Biology even less in math) but there were exceptions to this. Although extended discussion and instruction to develop students' knowledge of language resources and critical literacy were more prevalent (albeit still low) in English than the other subjects this was not the case in other important aspects of literacy instruction. For example, the number of blocks in which students were provided with any kind of reading was less in English than it was in either of the two other subjects and there was less of a focus on strategy instruction as well. It has often been claimed that one barrier to content teachers teaching literacy is a belief that this should be the responsibility of English teachers (May & Wright, 2007). On the basis of the patterns identified here some common assumptions about the amount of reading and reading instruction English, and therefore the reliance we should place on that subject to develop reading, might be called into question.

The differences between subjects, however, were not statistically significant, mainly because there was so much variability between individual teachers. Investigating within-subject variability in literacy teaching will be important for future analysis as there appear to be, compared to the literature on optimal practices at any rate, pockets of promise in every subject. Previous work we have led has found that capitalising on this positive variability can be a powerful source of new understanding about what works for whom and under what circumstances (Jesson, McNaughton & Wilson, 2015).

This study was limited to literacy instruction in lowto mid- SES classrooms with achievement rates lower than national achievement rates. It was part of a design based approach to solving low achievement rates across the upper years of secondary schools serving low to mid SES communities with high percentages of Maori and Pasifika students; students traditionally not well served by schools. This focus limits our conclusions about literacy instruction in general and its role. It might be the case that students in higher SES schools receive similar instruction but still achieve higher rates of literacy in ELA and other subject areas, although we suspect that this is not the case. For example, in a previous paper (Wilson, Madjar & McNaughton, 2016) we compared patterns of instruction in ELA across three different SES groups within the cluster of low to mid SES schools and found markedly lower rates of time spent on reading and reading instruction in the lowest SES schools than the low-mid and middle SES schools. Nevertheless, even if the rates of literacy instruction were similar in higher achieving, higher SES schools, that does not mean addressing these patterns of instruction in low SES schools is not warranted; the lower SES schools after all have a higher proportion of students with higher literacy needs.

We also think the lack of evidence for differences between subject areas is in part a measurement issue and that the differences are more nuanced and fine-grained than the analyses reported here could detect. The design of the tool and our processes for data collection and analysis were informed by theoretical and practical considerations. There was inevitably a tradeoff in this study between depth and breadth: using the approach we have enabled us to identify broader patterns of literacy teaching across many more teachers and schools than we could have otherwise but taking this approach meant that our ability to conduct more detailed and nuanced analyses of individual cases was reduced. Clearly, the number of opportunities students have to read texts and participate in particular forms of literacy learning is not the only thing we would to know about and we need more rich studies which investigate its quality the precise nature of interactions between student and teachers, and students and students, and students, teachers and texts. Having said that, it is important to know about learning time and we would question for example whether even extremely high quality episodes of extended talk or critical literacy instruction could make up for the scarcity of those opportunities identified in this study.

Acknowledgements

This study was undertaken as part of the Starpath Project for Tertiary Participation and Success, established in 2005 as a Partnership for Excellence between the University of Auckland and the New Zealand Government and administered by the Tertiary Education Commission (www.starpath.auckland.ac.nz). We acknowledge the support of the Starpath team and the contribution of the teachers who volunteered for the observational part of the study.

References

Allington, R.L. (2014). How reading volume affects both reading fluency and reading achievement. International Electronic Journal of Elementary Education, 7(1), 13.

Alozie, N.M., Moje, E.B. & Krajcik, J.S. (2010). An analysis of the supports and constraints for scientific discussion in high school project-based science. Science Education, 94(3), 395-427.

Alton-Lee, A. (2003). Quality teaching for diverse students in schooling: Best evidence synthesis. Wellington: Ministry of Education.

Applebee, A.N., Langer, J.A., Nystrand, M. & Gamoran, A. (2003). Discussion-based approaches to developing understanding: Classroom instruction and student performance in middle and high school English. American Educational Research Journal, 40(3), 685-730.

Athanases, S.Z. & de Oliveira, L.C. (2014). Scaffolding Versus Routine Support for Latina/o Youth in an Urban School Tensions in Building Toward Disciplinary Literacy. Journal of Literacy Research, 46(2), 263-299.

Behrman, E.H. (2006). Teaching about language, power, and text: A review of classroom practices that support critical literacy. Journal of Adolescent & Adult Literacy, 49(6), 490-498.

Biancarosa, C. & Snow, C.E. (2006). Reading next--A vision for action and research in middle and high school literacy: A report to Carnegie Corporation of New York (2nd ed.J.Washington, DC: Alliance for Excellent Education.

Bishop, A.R., Berryman, M.A., Wearmouth, J.B. & Peter, M. (2012). Developing an effective education reform model for indigenous and other minoritized students. School Effectiveness and School Improvement, 23(1), 49-70.

Bransford, J., Derry, S., Berliner, D., Hammerness, K. & Beckett, K.L. (2005). Theories of learning and their roles in teaching. Preparing teachers for a changing world: What teachers should learn and be able to do, 40-87. San Francisco, CA: Jossey-Bass.

Brozo, W.G., Moorman, G., Meyer, C. & Stewart, T. (2013). Content area reading and disciplinary literacy: A case for the radical center. Journal of Adolescent & Adult Literacy, 56(5), 353-357.

Bryk, A.S., Gomez, L.M., Grunow, A. & LeMahieu, P.G. (2015). Learning to improve: How America's schools can get better at getting better. Cambridge, MA: Harvard Education Press.

Carnegie Council on Advancing Adolescent Literacy. (2010). Time to act: An agenda for advancing adolescent literacy for college and career success. New York, NY: Carnegie Corporation of New York.

Cocking, R.R. & Mestre, J.P. (1988). Linguistic and cultural influences on learning mathematics. Hillsdale,NJ: Lawrence Erlbaum Associates.

Conley, M. (2008). Cognitive strategy instruction for adolescents: What we know about the promise, what we don't know about the potential. Harvard Educational Review, 78(1), 84-106.

Darling-Hammond, L. & Bransford, J. (2005). Preparing teachers for a changing world: What teachers should learn and be able to do. San Francisco: Jossey-Bass Inc.

DiGisi, L.L. & Willett, J.B. (1995). What high school biology teachers say about their textbook use: A descriptive study. Journal of Research in Science Teaching, 32(2), 123-142.

Fang, Z. (2006). The language demands of science reading in middle school. International Journal of Science Education, 28(5), 491-520.

Fang, Z. & Schleppegrell, M.J. (2010). Disciplinary literacies across content areas: Supporting secondary reading through functional language analysis. Journal of Adolescent & Adult Literacy, 53(7), 587-597.

Groenke, S. (2010). Missed Opportunities, Misunderstandings, and Misgivings: A Case Study Analysis of Three Beginning English Teachers' Attempts at Authentic Discussion With Adolescents in a Synchronous CMC Environment. Journal of Technology and Teacher Education, 18(3), 387-414.

Guthrie, J.T., Wigfield, A., Barbosa, P., Perencevich, K.C., Taboada, A., Davis, M.H., ... Tonks, S. (2004). Increasing reading comprehension and engagement through conceptoriented reading instruction. Journal of educational psychology, 96(3), 403.

Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York: Routledge.

Jacobs, V.A. (2008). Adolescent literacy: Putting the crisis in context. Harvard Educational Review, 78(1), 7-39.

Janks, H. (2013). Critical Literacy. Oxford: Blackwell Publishing.

Jesson, R., McNaughton, S. & Wilson, A. (2015). Raising literacy levels using digital learning: a design-based approach in New Zealand. Curriculum Journal, 26(2), 198-223.

Kamil, M.L., Borman, G.D., Dole, J., Kral, C.C., Salinger, T., and Torgesen, J. (2008). Improving adolescent literacy: Effective classroom and intervention practices: A Practice Guide (NCEE #2008-4027). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc.

Lee, C.D. & Spratley, A. (2010). Reading in the disciplines: The challenges of adolescent literacy. New York, NY: Carnegie Corporation of New York.

Madjar, I., McKinley, E., Jensen, S. & van der Merwe, A. (2009). Towards university: Navigating NCEA course choices in low-mid decile schools. Auckland: Starpath Project, the University of Auckland. Retrieved from the Starpath Project website, http://www.education.auckland. ac.nz/en/about/research/starpath-home/starpath-research/ towards-university.html

McNaughton, S. & Lai, M.K. (2009). A model of school change for culturally and linguistically diverse students in New Zealand: a summary and evidence from systematic replication. Teaching Education, 20(1), 55-75.

May, S. & Wright, N. (2007). Secondary literacy across the curriculum: Challenges and possibilities. Language and Education, 21(5), 370-376.

Mehta, C.R. & Patel, N.R. (1983). A Network Algorithm for Performing Fisher's Exact Test in r x c Contingency Tables. Journal of the American Statistical Association, 78(382), 427-434. http://doi.org/10.2307/2288652

Ministry of Education. (2007). The New Zealand curriculum: Learning Media, Wellington.

Ministry of Education (n.d.). New Zealand Education: Deciles. Retrieved from: http://www.minedu.govt.nz/ NZEducation/EducationPolicies/Schools/SchoolOperations/Resourcing/OperationalFunding/Deciles.aspx.

Moje, E. (2008). Foregrounding the disciplines in secondary literacy teaching and learning: A call for change. Journal of Adolescent & Adult Literacy, 52(2), 96-107.

Moje, E., Overby, M., Tysvaer, N. & Morris, K. (2008). The complex world of adolescent literacy: Myths, motivations, and mysteries. Harvard Educational Review, 78(1), 107-154.

Moje, E., Stockdill, D., Kim, K. & Kim, H. (2011). The role of text in disciplinary learning. In Kamil, M.L., Pearson, P.D., Moje, E.B. & Afflerbach, P. (Eds.). (2011)..Handbook of reading research (Vol. 4). New York: Routledge.

Moll, L.C., Amanti, C., Neff, D. & Gonzalez, N. (1992). Funds of knowledge for teaching: Using a qualitative approach to connect homes and classrooms. Theory Into Practice, 132-141.

Moore, D.W., Bean, T.W., Birdyshaw, D. & Rycik, J.A. (1999). Adolescent literacy: A position statement. Journal of Adolescent & Adult Literacy, 43(1), 97-112.

Neuman, S.B. & McCormick, S. (Eds.). (1995). Single-subject experimental research: Applications for literacy. Newark, DE: International Reading Association.

New Zealand Qualifications Authority (n.d., a). NCEA. Retrieved from the NZQA website, http://www.nzqa.govt. nz/qualifications-standards/qualifications/ncea/.

New Zealand Qualifications Authority (n.d., b). NCEA. Fact Sheet 4. Retrieved from the NZQA website, http://www. nzqa.govt.nz/qualifications-standards/qualifications/ncea/ understanding-ncea/the-facts/factsheet-4/.

O'Brien, D.G., Stewart, R.A. & Moje, E.A. (1995). Why content literacy is difficult to infuse into the secondary school: Complexities of curriculum, pedagogy, and school culture. Reading Research Quarterly, 30(3), 442-463.

OECD (2010). PISA 2009 Results: Overcoming social background: Equity in learning opportunities and outcomes (Vol. II). OECD Publishing. doi.org/10.1787/9789264091504-en.

OECD (2014). PISA 2012 Results: What students know and can do--Student performance in mathematics, reading and science (Vol. I, Revised edition). OECD Publishing. doi.org/10.1787/9789264201118-en

Pearson, P.D. (2013). Research foundations of the Common Core State Standards in English language arts. In S. Neuman and L. Gambrell (Eds.), Quality reading instruction in the age of Common Core State Standards (pp. 237-262). Newark, DE: International Reading Association.

Pressley, M. (2004). The need for research on secondary literacy education. In T.L. Jetton & J.A. Dole (Eds.), Adolescent Literacy Research and Practice (pp. 415). New York: The Guilford Press.

Pressley, M. (2006). Reading instruction that works: The case for balanced teaching. New York: The Guilford Press.

Schoenbach, R., Greenleaf, C., Hurwitz, L. & Cziko, C. (1999). Reading for understanding: A guide to improving reading in middle and high school classrooms. San Francisco: Jossey-Bass.

Schoenbach, R., Greenleaf, C. & Murphy, L. (2012). Reading for understanding: How Reading Apprenticeship improves disciplinary learning in secondary and college classrooms (Second ed.). San Francisco: Jossey-Bass.

Shanahan, T. & Shanahan, C. (2008). Teaching disciplinary literacy to adolescents: Rethinking content-area literacy. Harvard Educational Review, 78(1), 40-59.

Shanahan, T. & Shanahan, C. (2012). What Is Disciplinary Literacy and Why Does it Matter? Topics in Language Disorders, 32(1), 7. doi: 10.1097/TLD.0b013e318244557a

State Services Commission (2013). Better Public Services: Boosting skills and employment. Wellington: State Services Commission. Retrieved from: http://www.ssc.govt.nz/ bps-boosting-skills-employment#result5.

Tatum, A.W. (2008). Toward a more anatomically complete model of literacy instruction: A focus on African American male adolescents and texts. Harvard Educational Review, 78(1), 155-180.

Telford, M. & May, S. (2010). PISA 2009: Our 21st century learners at age 15. Wellington, New Zealand: Comparative Education Research Unit, Research Division, Ministry of Education.

Vannier, D.M. (2012). Primary and secondary school science education in New Zealand (Aotearoa)--Policies and practices for a better future. Wellington: Fulbright New Zealand.

Wallis, S.A. (2013). Binomial confidence intervals and contingency tests: mathematical fundamentals and the evaluation of alternative methods. Journal of Quantitative Linguistics 20 (3): 178-208. doi:10.10 8 0/092 96174.2013.799918.

White, S. (2012). Mining the Text: 34 Text Features That Can Ease or Obstruct Text Comprehension and Use. Literacy Research and Instruction, 51(2), 143-164.

Wilkinson, I.A. & Son, E.H. (2011). A Dialogic Turn in Research on Learning and Teaching to Comprehend Handbook of reading research (Vol. iv, pp. 359-387). New York: Routledge.

Wilson, A., Jesson, R., Rosedale, N. & Cockle, V. (2012). Literacy and language pedagogy within subject areas in years 7-11. Wellington: Ministry of Education. Retrieved from www.educationcounts.govt.nz/publications/series/Secondary_Literacy/Literacy_and_Language_Pedagogy

Wilson, A., Madjar, I. & McNaughton, S. (2016). Opportunity to learn about disciplinary literacy in senior secondary English classrooms in New Zealand. The Curriculum Journal, 1-25.

Aaron Wilson, Stuart McNaughton, & Tong Zhu

Woolf Fisher Research Centre, Faculty of Education, The University of Auckland, Auckland, New Zealand

Aaron Wilson is a Lecturer in the School of Curriculum and Pedagogy, an Associate Director of the Woolf Fisher Research Centre, and the program leader of the literacy strand of the Starpath Project, Faculty of Education at the University of Auckland.

Stuart McNaughton is Professor in the School of Curriculum and Pedagogy, Faculty of Education and Social Work at the University of Auckland and Director of the Woolf Fisher Research Centre. In 2014 he was appointed as the Chief Education Scientific Advisor for the Ministry of Education.

Tong Zhu is a doctoral student in the School of Curriculum and Pedagogy, Faculty of Education and Social Work at the University of Auckland and Researcher Analyst of the Woolf Fisher Research Centre. Table 1. A Selection of Literacy Tasks from Examination Papers for NCEA Level 21 Biology, English and Mathematics Achievement standard Sample question from 2013 external assessment task Biology AS91157: Demonstrate In the sweet pea plant, Lathyrus understanding of genetic odoratus, the allele for purple (P) variation and change flower colour is dominant over the allele for red (p) flower colour. A second gene determines the shape of the pollen. Long (L) pollen is dominant over round (l) pollen. (a) A purple, long/pollen plant, PPLL is crossed with a red, round/ pollen, ppll plant. Give the genotype of the F1 generation: (b) Two F1 generation plants were crossed to produce the F2 generation plants. Use the Punnett square to show the gametes and the expected genotypes of all the possible F2 offspring from these two F1 plants. (http://www/nzqa/ govt/nz/nqfdocs/ncea-resource/ exams/2013/91157-exm-2013.pdf) English AS91100: Analyse Task: 'Analyse how the writer significant aspects of describes his experiences of unfamiliar written text(s) science, and his understanding that through close reading, the boundaries of scientific supported by evidence discovery are expanding. In your answer you should: identify and give examples of techniques used in the text, and explain their effects (techniques might include syntax, similes, and metaphors), show understanding of the key ideas that the writer is presenting, show understanding of the writer's overall purpose'. Text sample:(Describing the author's observation of the transit of Venus): 'First a tiny sliver, hinted at rather than seen; then an obvious bite out of the Sun, like a small chip in a dinner plate; finally a tangible midnight disc, kissing the sun from within and then detaching'. Mathematics and Statistics Systolic blood pressure measures AS91267: Apply probability the pressure of blood in the methods in solving problems arteries as the heart beats and is measured in mm Hg (millimetres of mercury). In this question 'blood pressure' refers to 'systolic blood pressure'. The blood pressure of all the students in a school where Alice is the nurse, is approximately normally distributed, with mean 113 mm Hg, and standard deviation 10.3 mm Hg. (a) (i) What proportion of the students, chosen at random from Alice's school, would be expected to have blood pressure between 113 mm Hg and 120 mm Hg? (http://www.nzqa.govt.nz/ nqfdocs/ncea-resource/exams/2013/ 91267-exm-2013.pdf) (1) Level 2 is typically sat in Year 12 (age 17 years) Table 2 Aspects Biology English Maths (n = 28) (n = 39) (n = 37) M SE M SE M SE Text Usage (a) 62.9 9.1 51.2 8.0 56.3 8.2 Length (b) 3.0 * 0.2 3.5 * 0.2 2.3 * 0.2 Dialogic (a) Extended discussion 0.0 0.0 1.1 1.7 0.3 0.9 Conferencing 20.4 7.6 32.0 7.5 16.3 6.1 Activating prior knowledge 23.0 7.9 20.1 6.4 30.9 7.6 Focus of literacy teaching (a) All 58.7 9.3 54.7 8.0 39.7 8.0 Vocabulary 52.7 * 9.4 16.7 * 6.0 29.3 7.5 Structure 6.0 4.5 14.8 5.7 5.3 3.7 Language 0.7 1.6 13.9 5.5 0.6 1.3 Depth of literacy teaching (c) Item 37.6 9.2 44.7 8.0 38.8 8.0 Strategy 18.3 7.3 10.4 4.9 10.3 5.0 Critical 0.2 0.9 5.2 3.6 2.8 2.7 Note. * indicates statistical significance at 5% level. a. As of percentage of observed blocks b. Text length: 1 = less than 10 words, 2 = between 11 to 50 words; 3 = between 51 to 100 words; 4 = between 101 to 300 words, 5 = between 301 to 600 words, 6 = more than 600 words. c. As of percentage of observed blocks with any focus of literacy teaching Table 3 Biology English Maths Text Length < 10 words 11.0 4.5 28.8 11 to 50 34.4 20.8 44.1 51 to 100 18.4 11.2 10.7 101 to 300 12.3 19.1 6.8 301 to 600 18.4 32.6 6.2 600 + 5.5 11.8 3.4 Text Audience Adult 1.8 29.8 0.0 Youth 98.2 70.2 100.0 Text Form Whiteboard/projected 46.6 32.0 60.5 Photocopied 30.1 48.3 27.7 Published print 23.3 16.3 11.3 Onscreen 0.0 3.4 0.6 Text Source Teacher designed 74.2 65.7 86.4 Published print 24.0 30.4 11.9 Electronic 1.8 3.9 1.7