Literacy assessment of students from poor and diverse communities: changing the programs, changing the outcomes.

Badger, Lynne ; Wilkinson, Lyn

This article reports on a study carried out with teachers working in disadvantaged schools who are grappling with the issues of assessment and equity. The authors conclude that changing literacy outcomes for students from diverse and poor communities is less about finding technically efficient assessment tools and more about changing classroom literacy programs.

Introduction

This article reports on the concerns and issues about literacy assessment that a small group of selected teachers who work in disadvantaged primary schools are grappling with in the everyday world of their classrooms.

One of the fundamental issues which confronts them is that cohorts of students from poor and disadvantaged communities perform less well on school literacy tasks than do students from more affluent families (Freebody & Ludwig, 1995; WRAP, 1992; Connell, 1992; Williams, 1987). A major function of schools, it is often argued, is to sort and classify students, to discriminate among them, and to determine what kinds of socio-economic opportunities will be open to them. Assessment is one way in which schools do this. Because it acts as a gatekeeping mechanism, assessment is therefore heavily implicated in the production and maintenance of socio-cultural privilege (Connell, 1992: p. 20).

This constructs a dilemma for teachers in disadvantaged schools who are mediating between the diverse values and literacy practices of the groups of students they teach and the particular values and literacy practices which are privileged by mainstream curricula and assessment.

About the research project

This project focussed on the questions and concerns that the teachers in the research schools raised about the interrelationship of school literacy programs and assessment outcomes for students from poor and diverse communities. It was felt that these questions and concerns could provide insight for other practitioners in disadvantaged schools. Thus our aim as literacy educators was to explore and document the ways this group of teachers conceptualised their role in promoting students' literacy performance and achieving more equitable literacy outcomes.

The documentation was part of a larger literacy research project conducted in a number of disadvantaged schools across metropolitan Adelaide, South Australia. These schools are designated as disadvantaged because they have large numbers of students whose families are receiving government assistance. The research team had won a grant from the Committee for the Advancement of University Teaching (CAUT) to produce three videos and accompanying written materials which explored the relationship between literacy, poverty and schooling. It was our belief that the teacher development materials could be used to trigger conversations between other practitioners as they in turn explored this relationship in their own schools and classrooms. Thus we assumed that these materials would generate more genuine dialogue and have greater credibility for teachers and student teachers if they were grounded the actual practices and concerns of other practitioners.

The research for the video which dealt specifically with literacy assessment was carried out with teachers working in classes from Reception (Kindergarten) to Year 7, the final year of primary schooling in South Australia. Early in the project the teachers simply wanted to know what we wished to see in their classrooms and what we would film. We explained the kinds of ideas we had and some of the issues that concerned us. As we worked with the teachers we observed and heard the interesting and challenging assessment practices that they talked about and that they were putting into practice. But rather than provide exemplars of `good assessment practice' we wanted the materials to foreground what was problematic from the viewpoint of teachers and other educators with a commitment to achieving more equitable literacy outcomes for students in disadvantaged schools. We also wanted to situate the classroom footage within a framework which explored the politics of advantage (Eveline, 1994) and structural inequality (Connell, 1992) within our society.

As we discussed this framing with teachers they not only began to respond to the questions we asked about their literacy assessment practices but also to the issues of equity we raised. In doing so they began to see themselves as co-enquirers rather than simply informants. Moreover, as we spent time in classrooms observing and interacting with students and sometimes their parents, as well as giving a 'fee' for each classroom community, our discussion with the teachers became increasingly dialogic (Shor, 1980: pp. 95-6). That is, we acted as equals engaged in joint research rather than doing research on teachers and teaching, and this in turn allowed the teachers to have a significant measure of control. While we, the teacher educators, still formulated the actual framework for the research, in other respects there was genuine participation by our classroom colleagues. In particular, we worked in ways that incorporated their experiences, classroom practice, values and beliefs, and which encouraged joint decision making about what aspects of their practical and intellectual work should be documented.

The interrelatedness of the classroom literacy program and assessment

Together we explored issues such as how students at risk are defined; how each classroom literacy practice privileges some students' knowledge and experiences and marginalises others; the cultural values associated with particular kinds of literacy and certain categories of texts; and how unexamined values and beliefs can unwittingly influence teaching practice in ways that contribute to students' literacy success or failure.

These issues may appear to go beyond the usual discussions of literacy assessment which tend to focus more on changing assessment techniques or developing better, different, or more technically advanced instruments. However, we believed that assessment and assessment techniques cannot be understood outside of what counts in the literacy program, and how this is shaped by what is valued and what counts as mainstream literacy practices and competencies. Such practices and competencies are socially and culturally constructed, which means that assessment of students' literacy competencies is an act of social judgement which has social consequences (Connell, 1993).

Because it is an act of social judgment, assessment of itself never provides a level playing field. No assessment tool is free of bias (Connell, 1992: p. 22). It is a myth that any assessment tool, and this includes standardised tests, teacher-devised assessments and new approaches such as portfolios, can be objective in the sense that it is unbiassed or value-free. The kinds of tasks and questions that are set, the knowledge that is called on, the processes which students are required to undertake, all privilege some students' knowledge, experience and practices over others. Teachers in disadvantaged schools have to 'consider the extent to which assessment methods distort or reflect the literacy development of students from diverse backgrounds' (Garcia & Pearson, 1991: p. 254).

Thus a recurring feature of the dialogue we had with teachers was their focus on the literacy program. We found it impossible to talk about assessment without constantly coming back to issues of programming. It seemed that the literacy program was the most significant factor in making a difference to students' achievement. The two -- the classroom literacy program and assessment of students' achievements -- were inextricably linked. In other words, we found that teachers' literacy programs construct the limits and possibilities for students' school literacy performance, which is then the focus for assessment.

The teachers with whom we worked were grappling with the need to radically rethink the literacy programs they had offered to students, and were asking themselves the kinds of questions below.

* What does my classroom program make possible?

* What competencies are the student's able to display?

* What competencies are excluded by my program?

* Which literacy practices and texts are privileged, and which marginalised by the program I offer?

While part of this rethinking meant reviewing assessment practices, it was dear that making a difference for students in disadvantaged schools is not simply a matter of changing or improving assessment techniques. More equitable outcomes result from literacy programs which reflect, build on and assess a diversity of literacy practices.

The teachers were also grappling with the way in which assessment permeated their literacy programs. Students' literacy competence was assessed moment by moment throughout every school day, as teachers listened to them speak, heard them read aloud, observed them during writing and silent reading sessions, and interacted with them. From the moment that students entered the classroom, the 'production of differences in literate competence begins' (Baker & Freebody, 1993: p. 291). Often subconsciously, teachers begin a mental ledger on each student, entering credits and debits according to how well the student matches up with their expectations about what counts as literate behaviour. This crediting is communicated to students both overtly and covertly, consciously and unconsciously, when teachers respond to their literacy work and behaviour. As Baker and Freebody point out, `[t]eachers do not rely on formal tests to infer how good children are at literacy; they hear this competence minute by minute in exchanges' (1993: p. 287).

The teachers in the project recognised assessment as a pervasive fact of classroom life. One teacher explained her viewpoint as follows.

I guess assessment goes on in a variety of situations in our classrooms. It is not just contained in language sessions where I go around and collect specific information but most of the assessment is actually found in real life experiences where [students] are writing letters of real importance like last year writing to get donations of food for camps ... or class meeting agendas where children fill in the problems and concerns or issues they want to discuss in class meetings. Some children filled in SACON forms [minor works request forms] ... and it really shows their power of language and how they can use it to get things done. So language assessment is done all day in a whole range of ways.

If teachers are to make a difference for students from disadvantaged groups, then they need to examine assessment in all its manifestations. Students' success and failure in literacy is not just measured at 'transition points' such as the end of the term or the year, or the end of secondary school. It is constructed moment by moment as students engage in the literacy events offered by the teacher's program and as the teacher assesses their competence during those events.

Broadening opportunities for assessing students' literacy competencies

During the project both teachers and researchers have been challenged to take apart what has been naturalised, what is unexamined and assumed, to see how both programs and assessment practices are implicated in the reproduction of disadvantage. In interrogating their programs to achieve more equitable outcomes for students from poor and diverse communities, teachers identified three key interrelated aspects that need to be addressed. These aspects are:

* the diversity of literacy practices which are reflected in the program

* the constraints and possibilities of classroom literacy practices, and

* student perspectives.

Diversity of literacy practices reflected in the program

Teachers who are concerned about equity place 'social justice at the foundation of thinking about curriculum and assessment' (Connell, 1993: p. 83). This leads them to consider the ways in which their programs reflect or fail to reflect the diversity of literacy practice in the wider community. The traditional school literacy curriculum has privileged the practices, texts, content and forms (Connell, 1992: p. 22) of some groups in the community over others, giving advantage to the children from these groups. If this is to be redressed then researchers and teachers need to:

look at the different kinds of literacy practices that go on in different subcultures and in different areas and in different workspaces and to look at different kinds of ways in which they are projected back into schooling -- to look at the way in which the activities that go on in the classroom reflect or fail to reflect certain sorts of ways in which reading and writing are routinely practised in the everyday lives of people in different sectors of the society, different work sectors, different domestic sectors, different kinds of communities.

(Peter Freebody, video transcript in Badger et al., 1997)

The teachers in the project were, to varying extents, trying to broaden the range of literacy practices and texts offered in their classrooms. They realised that the diversity of students' experiences means that they bring to school different strengths and competencies in literacy, many of which are unrecognised and unvalued in the traditional literacy curriculum. For example, in one school, nine- and ten-year-old students were extremely competent in using timetables to travel to a seaside suburb where there is considerable weekend entertainment, adeptly making connections between two buses and a tram. When the teachers recognised the students possessed such skills they were able to credit them and build on them in their classroom literacy programs.

In the same school, teachers saw how other areas of the curriculum offered opportunity for different literacy practices. For example, the students grew vegetables, and as part of that endeavour they read seed packets, brochures, and Instructions about planting and caring for the plants as well as for dealing with weeds and pests. They wrote labels for different vegetables, and kept descriptive logs recording the growth (or death!) of their plants, Again, many of the students sang in the school choir, where there were opportunities for them to share their interpretations of lyrics, as well as to discuss the meanings of obscure or unfamiliar words.

In another school, two of the junior primary teachers had a particular interest in understanding the nexus between the practices which counted in their literacy programs, the range of outcomes which their programs made possible and the ways they assessed their students' literacy competencies. These teachers too were involved in an on-going process of modifying the curriculum they offered so as to include opportunities for students to use literacies beyond the usual school literacies. They had a particular focus on using literacy for social action. For instance, the students were involved in writing letters to cereal manufacturing companies to request their support in providing breakfast foods for the school camp because the school community did not have the material resources to meet all the costs required to send students to the camp.

In this same classroom students regularly filled out the forms through which schools request minor works and repairs. When students identified that repairs were needed to school property they obtained and completed the form. To do this successfully they had to make a range of decisions about the location of the problem and the category of repair needed, providing accurate information that could be acted on by the maintenance workers when they came to the site. Those who filled out the form also had responsibility for faxing it to the appropriate authority. The students who had been in the school for some time took this responsibility seriously for themselves, and also inducted new students into the processes.

Additionally, students used the literacy practices involved in democratic decision making. Those with designated executive responsibility regularly ran class meetings where the class responded to items that students placed on the agenda, writing minutes to provide a record of decisions that they could refer to later.

These same teachers worked with students to identify a range of classroom responsibilities (e.g., lunch monitors) and developed position descriptions for them. Individual students then selected a position of interest to them and applied for it in writing arguing their suitability against the criteria, and providing a reference from a family member or friend. The application was read by a selection committee of peers who made the decision as to which applicant got the position. All the positions were vacated at the end of each term and the process was repeated.

The kinds of literacy events described above are very different from the traditional school literacy tasks which focus on instruction or evaluation. Through these and many other literacy events the teachers provided a variety of opportunities for students to learn and to demonstrate their literacy competencies, usually through tasks that were real and meaningful for them.

Constraints and possibilities of classroom literacy practices

There is more to constructing an inclusive curriculum, however, than simply providing a diversity of literacy events. Literacy events in themselves are not socially or culturally neutral, but may be enacted in a variety of ways depending on the rules for the discourse within particular communities. Therefore literacy events themselves have to provide spaces for students to participate in ways that take cognisance of students' known ways of behaving and which allow them to build on and extend their understandings. Haas Dyson highlights this point when she says, `If a curriculum is to be truly responsive to diversity, truly child-centred, it must be permeable enough to allow for children's ways of participating in school literacy events' (Haas Dyson, 1992: p. 41).

Teachers in the project understood the need to attend to the ways in which different groups of students participated in and were positioned by specific literacy events. This led them to reflect on and try to understand different students' perspectives on and participation in the events.

One teacher explained how this presented her with a major challenge:

One of the biggest challenges I face in teaching in a disadvantaged school is ... constantly questioning what I present to children. So when I'm presenting something to children I examine the impact that it's going to have on them. For example, when I'm presenting a book to children I try to look at it through each individual child's eyes and ask what messages will this give Aboriginal children in my class. What messages will it give the non-English-speaking background children?

The quote demonstrates one of the ways in which the teachers tried to make their teaching more inclusive of a diverse range of students' experiences, serving to render the 'socio-educational barriers more permeable' (Connell, Johnston & White, 1994: p. 211). They sought to understand how students might interpret the classroom literacy events in which they participated (Garcia & Pearson, 1991) and how these events have the potential to both constrain and to enable different groups of students, in the class. Two examples from the classroom demonstrate how teachers analysed literacy practices.

One Year 7 teacher came to understand the difficulties and hurdles she had unwittingly constructed when she demanded that during silent reading time students read only novels. She saw that many of the students were selecting familiar formula novel series such as Sweet Valley High, Goosebumps, Hair Raisers, and Babysitters Club, or they were only `pretending' to read the book in front of them. She realised that she had been privileging the reading of novels and negating a body of texts which students found interesting and enjoyable, thus limiting the range of texts on which students could develop and demonstrate their reading competencies. She has now changed her practice and permits the reading of magazines and comics during silent reading time.

A junior primary teacher questioned the valorisation of class meetings as a means of allowing students to experience democratic decision making. Her analysis of these meetings helped her to understand that the usual procedures are based on a 'white middle class model of operating' which serves to marginalise Aboriginal students in particular.

Sometimes we provide situations in schools where we are actually limiting the information we obtain about students. For example, we have class meetings ... which are set up on a white middle class model. We have one person talking at a time, people put their hands up and people vote. This can actually limit the involvement of some of the cultural groups we have in our school. For example, Aboriginal children often don't work well on that model, don't work well in that kind of a situation. So even though it is important they are exposed to different situations we really need to incorporate some of the ways that they learn and talk in our class meetings. So we may set up lots of situations where children can come up with ideas in small groups and then have the opportunity to share those back with the class in a class meeting rather than just use the one model.

Rather than assess the students as unable to participate, as inadequate or as lacking in confidence, this teacher analysed the practice from students' perspectives and modified it to provide a means for all the students to participate. In this way she avoided ,reading' the limitations of the practice as limitations of the students. Students cannot receive credit for the literacy understandings and skills they have if the program fails to give them opportunities to show what they know and can do.

Students' perspectives

If a literacy program is to be equitable it needs to reflect the diversity of students' literacy practices, world views and experiences. In addition, teachers need to find out how students are responding to the program that is being offered to them. It is argued that '[t]o understand ... children's perspectives in school is to gain some insight into how they make sense of and interpret instructional experiences' (Dahl, 1995: p. 1). Understanding students' perspectives -- what they are taking from the literacy program, their sense of success and failure, their goals and expectations -- is important for teachers modifying and adapting the program.

In traditional forms of assessment, the knowledge that 'counts' about students' achievement has been predominantly based on the teacher's perceptions. Student knowledge of who they are as literacy users and their literacy achievements has been completely discounted. The teachers in the research project were actively working to redress this, and to include students' perspectives as part of the data which informed their assessment processes and their program evaluation.

The change from student as object of assessment (Edelsky, 1991: p. 87) to student as participant in assessment takes account of the constructed nature of knowledge. If we accept that knowledge is constructed, then the knower is an intimate part of the known (Johnston, 1989), and their perspective should be made to count.

In the same way that classroom programs need to accommodate the multiplicity of literacy experiences, assessment practices have to explore the multiplicity of perceptions (Taylor, 1990) which are reflected in classrooms serving diverse communities.

To assert privilege for one type of voice among all others in a classroom promotes and maintains a hierarchy based on nationality, gender, race, economic class, and ethnicity. Unless teachers and students are allowed and willing to listen to each other ... to use their multiplicity of voices in any classroom, there is little hope for democratic development in our society.

(Shannon, 1993: p. 92)

Teachers in the project schools were clear that this sentiment must extend to assessment, given its predominance in shaping the curriculum. Student self-assessment can serve to change the power dimension in classrooms, affirming students as one of the major stakeholders in the assessment process.

While self-assessment is seen to be empowering for students, it is not unproblematic. It cannot be accepted uncritically as a quick route to equity-based assessment. It can be as subject to bias and constraint as other kinds of assessment, with students 'locked into seeing things only through the single set of lenses provided for them by their cultural guardians' (O'Loughlin, 1995: p. 107). When students wrote comments such as:

* I can write lots of genres

* I can write neatly with finger spaces between my words

* I'm a good reader because I borrow lots of fiction from the library

project teachers reviewed their programs in the light of the emphases and values on which students were focusing.

Interviews with students revealed that some of the kinds of self-assessment required were difficult. They said that parents and caregivers were likely to ask more questions than previously about their literacy achievement, and that, they were expected to be able to give reasons and explanations in a way that they hadn't in the past. They had to take responsibility for their learning, and be able to justify their judgements about their work, and they told us that this was not easy to do.

I think that the hardest part on the sheet is 'How did I go?' because it's really hard to write about yourself because when you show your teacher or your parents at the interview they read it and then you get home and they talk to you about it and it's really hard to talk about yourself and write about yourself.

(Kate, Year 7)

When students' perceptions count as a source of information, and teachers understand the complexity of how these perceptions are constructed in the classroom, then they are in a better position to understand the relationship between their programs and students' literacy outcomes.

Conclusion

It can be argued that the whole enterprise of changing literacy outcomes for students from diverse and poor communities is less about finding more technically efficient assessment tools and more about changing classroom literacy programs. As teaching and assessment are inextricably linked, teachers' programs have to provide the spaces and opportunities through which students can demonstrate the diversity of their literacy competencies and build on those to use literacy as a powerful tool for 'shaping identity, knowledge and power' (Luke, 1993: p. 48).

REFERENCES

Badger, L., Wilkinson, L., Comber, B., Nixon, H. & Hill, S. (1997). Literacy Assessment in Disadvantaged Schools, Video module 6. Melbourne: Eleanor Curtain.

Baker, C. & Freebody, P. (1993). The crediting of literate competence in classroom talk. The Australian Journal of Language and Literacy, 16,4. pp. 279-94.

Connell, R. (1992). Measuring up: Assessment, evaluation and educational disadvantage. ACSA Teaching Resource No. 2, Belconnen, ACT: Australian Curriculum Studies Association.

Connell, R. (1993). Schools and Social justice, Toronto, Canada: Our Schools/Our Selves Production.

Connell, R., Johnston, K. & White, V. (1994). The issue of poverty and educational measurement, in Hatton, E. (ed.), Understanding Teaching Curriculum and the social context of schooling. Sydney: Harcourt Brace.

Dahl, K. (1995). Challenges in understanding the learner's perspective. Theory Into Practice, 34, 2. pp. 124-30,

Haas Dyson, A. (1992). The case of the singing scientist: A performance perspective on the 'stages' of school literacy. Written Communication, 9, 1. pp. 3-47.

Edelsky, C. (1991). With Literacy and Social Justice for All: Rethinking the social in language and education. London: The Falmer Press.

Education Department of South Australia. (1992). Writing Reading Assessment Programme. Final report. Adelaide: Education Department of South Australia.

Eveline, J. (1994). The politics of advantage. Australian Feminist Studies, 19, Autumn, pp. 129-54.

Freebody, P. & Ludwig, C. (1995). Everyday literate practices in and out Of schools in low socio-economic urban communities: Executive summary. Canberra: Department of Education, Employment and Training; Melbourne: Curriculum Corporation.

Garcia, G. & Pearson, P. (1991). The role of assessment in a diverse society. In Hiebert, E. (ed.), Literacy for a Diverse Society. New York: Teachers College Press.

Johnston, P. (1989). Constructive evaluation and the improvement of teaching and learning. Teachers College Record, 90, 4. pp. 509-28.

Luke, A. (1993). The social construction of literacy in the primary school. In L. Unsworth (ed). Literacy Learning and Teaching: Language as social practice in the primary school. Melbourne: Macmillan.

O'Loughlin, M. (1995). Daring the imagination: Unlocking voices of dissent and possibility in teaching. Theory Into Practice, 34, 2. pp. 107-16.

Shannon, P. (1993). Developing democratic voices. The Reading Teacher, 47,2. pp. 86-94.

Shor, I. (1980). Critical Teaching and Everyday Life. Boston: South End Press.

Taylor, D. (1990). Teaching without testing: Assessing the complexity of children's literacy learning. English Education, 22, 1. pp. 4-74.

Williams, T. (1987). Participation in Education. Research Monograph No. 30. Hawthorn, Victoria: Australian Council for Education Research.

Addendum

The correct reference for the project Early Literacy: Practices and possibilities, cited in the November 1997 issue of the AJLL in Susan Hill's article, is:

Commonwealth of Australia. (1997). Early Literacy: Practices and possibilities. Adelaide: Department of Education and Children's Services.