An investigation of discourses of literacy assessment in two first year high school English classrooms.

Moni, Karen ; van Kraayenoord, Christina E. ; Baker, Carolyn D. 等

The aim of this study was to develop an understanding of the discourses of classroom-based literacy assessment that may be established, negotiated and maintained by teachers and students during assessment-related classroom activities in two Year 8 English classrooms during one school year. There are four major questions that guided the data collection.

1. What do Year 8 English teachers think that students' previous experiences of literacy assessment in Year 7 have been?

2. What do Year 8 English teachers think that their students need to know and understand about literacy assessment in Year 8?

3. What do Year 8 English teachers teach their students about literacy assessment in the high school?

4. How do the literacy assessment methods, tasks, and practices teachers use at Year 8 level contribute to socialising students into the assessment culture of the school?

A brief review of research literature

The study drew on the following research developments. First, there has been a paradigm shift in the field of literacy assessment away from standardised testing to alternative assessment methods and practices (Calfee, 1993; Coles, 1998; Johnston, 1992; Murray, 1994). These methods and practices, which involve students and teachers undertaking collaborative assessment activities, acknowledge the subjective nature of assessment and emphasise learning-integrated assessment to inform instruction (Broadfoot, 1995; Harrison, Bailey, & Dewar, 1998; Johnston, 1992; Nisbet, 1993; Theobald & Mills, 1995). This transition from a testing culture to an assessment culture (Wolf, Bixby, Glenn, & Gardner, 1991; Kleinsasser, 1995; Kleinsasser, Horsch, & Tastad, 1993) has stimulated the adoption of classroom-based literacy assessment practices, which have increasingly become the focus of research (Broadfoot, 1995; Graue, 1993; van Kraayenoord, 1996).

At the same time, there has been a developing understanding of how knowledge is socially constructed in classrooms through interactions among teachers and students (Edwards & Mercer, 1987; Mercer, 1995; Nuthall, 1997). This has led to research that has investigated such interactions from the perspectives of all the participants. Findings from research undertaken in the United States of America and Australia based on this understanding have indicated that literacy is socially constructed through interactions among teachers and students during talk in daily classroom activities. Such activities include the sharing and composition of texts (Baker, 1992; Baker & Freebody, 1989; Erickson, 1984; Fisher & Hiebert, 1990; Gee, 1990; J.L. Green, 1990; J.L. Green & Meyer, 1990; McCarthey, 1994; Myers, 1992; Prentiss, 1995; Turner & Paris, 1995).

Correspondingly, there has been a growing focus on the role of talk, tasks, and texts in the construction of knowledge with researchers investigating the interplay of these factors in a range of classroom settings (Baker, 1992, 1994, 1997; McCarthey, 1994; Myers, 1992; Turner & Paris, 1995). Investigations of talk in classrooms in the area of literacy have shown that literacy is constructed through teacher-student and student-student interactions around texts (J.L. Green & Meyer, 1990; Heap, 1995; Myers, 1992; Prentiss, 1995; Talty, 1995). In particular, Baker (1992), and Baker and Freebody (1989) revealed how interactions around texts constructed knowledge about how to refer to, respond to, and interpret storybooks. McCarthey (1994) showed how student writers in a primary school setting learned about appropriate topics and ways of writing through the writing conferences held with their teacher. In these studies, teachers have been identified as mediators, activators and interpreters of texts, and the students as active participants in the construction of classroom knowledge during teaching and learning activities.

Based on the findings from these various research developments, the present study adopts a view of literacy assessment as socially constructed by classroom members in the course of interactions around assessment-related activities. These interactions are considered in terms of the production and negotiation of discourses related to assessment. Discourses (with an upper case D) have been defined by Gee (1990) as:

 ways of behaving, interacting, valuing, thinking, believing,
 speaking, and often reading and writing that are accepted as
 instantiations of particular roles by specific groups of people.
 They are always and everywhere social. Language as well as
 literacy, is always and everywhere integrated with and relative
 to social practices constituting particular Discourses. (p. xix)

Baker (1991) has suggested that discourse involves 'collections of ideas, knowledges, vocabularies, and material practices (p. 5)'. It is possible to hear and witness such 'collections' in participants' expressions of perspectives and in their social interactions. Discourses have not been made central in previous research on literacy assessment. This study combines elements of the definitions of Discourse provided by Gee (1990) and discourses as described by Baker (1991) to propose that there will be discourses operative in classroom-based literacy assessment which include ways of valuing, speaking and behaving as well as practices, knowledges, ideas and vocabulary related to assessment. The current study therefore extends research into literacy assessment events by focusing on how teachers and students construct such discourses related to literacy assessment during their daily classroom activities.

Data sources

The study was undertaken in two Year 8 English classrooms (the first year of high school) in separate and different state high schools in Queensland, Australia. School I was located in a dormitory city close to the state capital. Fiona Jackson (pseudonym) who participated in the study was the Head of English at the school. She had a wide range of experience in rural and urban schools, and a strong personal commitment to teaching and professional development. The three groups of four students from her class who participated in the study have been referred to as the Gang, the Pragmatists, and the Enthusiasts. School 2 was a suburban high school in the state capital. Elizabeth Lewis (pseudonym) and three groups from her class participated in the study. Elizabeth was a reflective teacher who was committed to developing effective practice. The student groups were referred to as the Friends, the Quiet Achievers, and the Rebels.

Methods

A range of qualitative approaches was used to systematically collect data related to literacy assessment activities from multiple sources, numerous times throughout one school year. Ninety-three lesson observations took place in School 1 and 134 in School 2. Semi-structured interviews about literacy assessment were conducted with the teachers regularly throughout the year (n = 23 for School 1 and n = 18 for School 2). Semi-structured interviews were also conducted with the groups of students (n = 27 for School 1 and n = 27 for School 2). Lessons in which the teachers introduced and returned assessment tasks were also videotaped and transcribed (n = 28).

During the fieldwork phase the data were organised according to type and origin. For example, all of the transcripts from the teacher interviews were stored in date order in separate folders for each teacher, while the videotapes were stored in chronological order with both schools together. The first step in the data analysis process was therefore to organise the data in meaningful ways in order to answer the research questions.

Appendix A shows how the research questions were the starting point for all data analysis procedures. Using these questions, the data were organised into three overlapping and interrelated sets. Each set was developed both to address the research questions and to focus on different aspects of classroom life related to assessment. Appendix A also provides information about the type of data collected and the procedures used for analysing each set of data.

Using this framework, the data were analysed multiple times, in multiple ways and for different purposes. Transcripts from interviews involved systematic and recurring questioning and analysis in clearly defined stages comprising open, axial and selective coding (Bogdan & Biklen, 1992; Strauss & Corbin, 1990). Transcripts from videos were analysed using applied discourse analysis approaches following the work of Gee, Michaels and O'Connor (1997), Potter (1997), and Potter and Wetherell (1994).

Results

This section of the article synthesises two sets of findings related to discourses of assessment. These are first, results from analysing transcripts of interviews with the teachers and the students. The second set of findings relate to discourses of literacy assessment arising during literacy assessment events. These discourses were produced by the teachers, the teachers and students together, and the groups of students during interactions related to assessment.

Teachers' discourses of assessment

Both teachers valued assessment highly. Throughout the year they described assessment as 'serious', 'important' and 'valued'. They perceived that students should consider assessment to be 'rewarding' and as something requiring 'effort'. Assessment was the 'work' of the classroom.

For both Fiona and Elizabeth, discourses related to assessment were as much about teaching students about sharing responsibility and teamwork as about completing tasks. In this respect assessment was used as a means of establishing appropriate relationships and the work ethic of the classroom. For example, Elizabeth constructed assessment as a team effort involving students, the teacher and their parents collaborating to ensure that students completed tasks:

[E.sup.1] I hope that they will have understood by now that it's important to me because ... I've put it on paper, typed and given it to them and spent time on it.... I think that they should know that I feel that their assessment is very important. I also want them to understand that I think that their assessment should be shown to their parents, that their work should be shown to their parents, that the parents and me and they are all together in expecting them to get their work done. (S2: 7/2)

For both teachers, their classroom activities and teaching were constructed as the teacher helping the students to do well with the assignments. For Fiona, sharing responsibility for assessment was about building trust:

F I think it's important with Year 8s to establish a level of trust, you know, with assessment and the autobiography, you know, in a way, they've been really happy to tell me all sorts of interesting things, so I think that does need to be handed out and handled somewhat tactfully, sensitively, and usually, I mean because they have been so open, it's a sign that, yes, they do feel like they can tell me things and so I find that fairly encouraging in terms of our relationship as a class and teacher for the first year. That's important with that assessment. (S1: 5/3)

The most consistent thread in these teachers' assessment discourses was that written assessment tasks were more important than oral tasks for developing language, and for providing the teacher with useful assessment information. This is evident in the following excerpt from an interview with Fiona at the start of Semester 2:

F I felt after last semester and when I was aggregating their results for their report cards that some of them were getting somewhat inflated results because of the amount of umm assignments or pieces of assessment that allowed the students to work hard and do pretty pictures but not really have a lot of language content and so I wanted to umm start off this semester with a piece of solid writing so we could really look into their writing a little bit more again. When I think about first semester perhaps the most extensive piece of writing they did on their own was their autobiography back near the beginning of the year and so, you know, we're almost six months down the track before doing another solid piece of writing.

The teacher's version of each task was also consistently privileged more than the students' version. For example, in the following excerpt where Fiona was introducing a picture book assignment her version of an appropriate children's storybook was foregrounded by her repeated references to illustrations, here named 'pictures' (lines 12-17):

[12.sup.2] OK, most of you like books with pictures, certainly plenty of 13 you read books without pictures but it's nice to have a look at 14 pictures too, isn't it. OK so the little kids love books with 15 pictures, so pictures are an important part of this story, and 16 therefore you're focusing on visual language there, what you 17 can see in the illustration as well as the words.

While these discourses remained stable, other discourses changed. For example, what counted in assessment changed with each task because each task focused on a different genre and was completed in a different medium and under different conditions. Teachers' perceptions of student knowledge and their behaviours related to assessment also changed from the first semester to the second semester. In the first semester both teachers expected that students were beginning to understand assessment. At the beginning of the second semester both teachers perceived that their students' understandings had altered. For Fiona, students were more task oriented and more like older students which implies that they were being enculturated into high school assessment practices. In a similar way, Elizabeth perceived that students were more settled:

E I think generally over the year the students have just become more settled, you know, ... I've noticed that there's a lot more sense of purpose and focus with people like Stuart, and Adam, and David, umm Jett, a lot of those boys who were really scatty and unable to focus on things for very long, they've really improved in--in, you know, their stickability, their attention to things. (S2: 12/10)

During introductions to assessment tasks, mediating the task sheet, and when giving feedback, both teachers constructed assessment as the responsibility of the teacher. This was achieved through the dominance of teacher talk, teachers' explicit instructions to students to listen, and their accounts of their roles in marking assessment tasks. An example of this is in the opening phase of a feedback session where Elizabeth positioned herself as the authority regarding assessment, as having privileges related to talking about assessment (line 5), as determining how students should respond in feedback sessions (lines 6-7):

1 E OK, now I told you today, that for today I would give you back your 2 assignments and that's what I want to do so it's very important that 3 you pay attention to what I have to say. I don't want to spend lots of 4 time talking to you today about this so this is what I propose that first 5 of all we'll talk--I'll talk a little bit about the assignments--give them 6 back to you, you can spend some time looking at the comments I've 7 made

Teachers used feedback to account for their practices of literacy instruction and assessment to the students, establishing themselves as responsible for reading, marking, and suggesting improvements in their students' work. Fiona, for example in the excerpt below, positions herself as writing comments on the students' work (line 59), giving advice (line 60), suggesting improvements (line 61), as the main audience for their work (line 62), and as the expert who can correct their work (line 65):

59 On the front, the comment on the front, that's just a general 60 comment. It might be some advice for next time, how you 61 could improve for next time or it might be just something that I 62 especially liked in what I saw or just some overall general 63 comment. OK? (4.0) Are there any questions about anything 64 that I've got there? You should look through your stories so 65 you can see if I've made some corrections. (S-1: 16/5)

These discourses of teacher responsibilities were conveyed by teachers' references to their completion of the task sheets through the writing of comments, the recording of grades, and the highlighting of statements of standards. For example:

7 F OK, lets have a look at the feedback sheet, so you can see, you can see 8 on the front, that I've given you a general comment about your work 9 but if you turn to the umm criteria--standards sheet inside, you can 10 see that I've highlighted with a highlighter pen the descriptions of 11 your work. I think that many of you, most of you in fact should be 12 very proud of your work, there were some beautiful pieces of work 13 handed in.

Students' discourses of literacy assessment

The study found that student discourses related to assessment as enacted in group interviews, differed across the groups during the year. The study also showed that students' discourses both within and across the groups also changed as the year progressed. At the beginning of the year, despite initial misgivings, most of the students constructed assessment as easy and fun. For example, the Friends enjoyed putting the poetry anthology together because they liked the topic:

C It's fun though

J It's good

B I love putting them together!

I How does it compare to others that you've done, the previous ones?

B It's so easy

J It's easy (Friends, 28/5)

Students did not find tasks challenging if they were writing about themselves or if they had had previous experience with the genre:

I Do you think it's challenging Alison?

A No

I You don't, so what's easy about it?

R The story

A Everything, the story, everything!

T I mean well like if you like reading stories the older kids read it puts you at an advantage because younger kids like they're more easy to amuse like I mean for a younger kid you just have to bring a dog in the room and they go Ooohoooh, for an older kid they see a dog and they say so what. (Pragmatists, 30/4)

There were differences in the values students placed on assessment. For example, the students' reactions to the first assessment task varied. There was initial panic from students in School 1. Tim, for example said: 'First of all I thought "assessment ugh (vomit sounds)" but I read it and I came out and I found out--ah, this is not too bad, I can do this' (Pragmatists, 27/2). The Enthusiasts felt very strongly that assessment was starting too soon:

I What's the first reaction to getting the task sheet that assignment sheet?

E We're starting=

C =Too early

E No, we're starting

I Starting? You were happy to think that you were starting?

E Yeah

I You were or are you just saying that?

E Nooo, it-it I hated the thought of starting something

I Oh right and Carl you figured that it was too early

C Yeah

M Yeah they should have it about

C [We haven't even been here

M [Yeah in term two or term three or something (Enthusiasts, 27/2)

The Rebels (School 2) came to high school with negative views of assessment, which they transferred to the high school context:

A You've got to keep to them (deadlines) and you need to make more time

B And there's no way I'm giving up my free time, not all of it

K I'm not, I'm not not going to go to the dances and that, like

A You need your social life, it's hard to sit in every night and just do everything (Rebels, 13/3)

The Friends (School 2) were positive about their initial assessment experiences in high school. Specifically they were enthusiastic about task sheets and the teacher's detailed explanations.

B And she also--I liked how they did this 'cos it shows you more

I You mean the criteria sheet?

B Because like you never had them last year either so it gives you a better--more understanding of what you did and what you didn't do (Friends, 29/3)

One unexpected finding from the study was that while students' accounts were different in many respects, what did emerge across all groups as the year progressed were increasingly negative discourses related to assessment. For students 'work' became the word they most commonly associated with assessment tasks. Assessment was the work of English. It was work because it involved completing tasks. Students constructed assessment as too much work because of the number of tasks they had to complete:

Si You get lots of assignments

J Yeah, there's just too much

I There's too much?

Su Yeah like in the first term in the first part she gave us like, you know, like one and then there was a break but now it's like two at a time and no breaks or anything (Quiet Achievers, 3/12)

In second semester, assessment was work because it was no longer fun. It was also work because the students stated that completing tasks entailed effort:

B First semester was fun

J Yeah I thought it was fun and I thought I can't wait to come back on Monday and for the week I wanted to come back all the time, everyday

B But now ...

J But now it's serious and you have to study and like we get a lot of homework (Friends, 5/8)

By focusing talk of assessment on length, how the finished product should be structured, and what it should contain, the teachers re-inforced the nature of assessment as work. In addition to these discourses of work, the teachers and students produced and maintained discourses of effort. These were different for teachers and students. For teachers, effort was valued and applauded but only in their personal responses and written comments about students' work. Students, however, believed that working hard on an assessment task should be considered when grades were awarded. For example, the Friends were puzzled about where effort fitted into the assessment process. Jeremy wanted recognition for the amount of work involved in completing continuous assignments and what he saw as the repetitive nature of the assessment tasks:

J And also what's the story about effort, as soon as you do an assignment and like you finish and like all the ink in your pen is like running out, they take it up and then the next lesson they give you another one and say oh we want this one in about two weeks time, son. Thanks and you do that one so you've got to go about and buy a new pen and they give you another one (mock crying). It's just continual! (Friends, 12/6)

Bernie, too recognised that the amount of work and effort was not part of the assessment criteria in high school and as with so many of the accounts of this group linked this omission to the teacher at a personal level:

B I just wish that Mrs--because I put a lot of effort--even though it doesn't not look like it-but I put a real lot of effort into making it neat and everything and like--she doesn't know that and she just marks you however she sees it, but she doesn't realise how much work you've put into it

I OK, so how could she acknowledge that in your assignments then?

B Umm (2.00) I don't know umm she could be a bit more lenient as to what we do:o like she could be pretty soft, like she's not very soft as in--she doesn't care, she just sees what--she's looked at that but she doesn't realise how much work you've put in. (Rebels, 12/6)

During the year the students' discourses of assessment became more institutionalised. Groups talked about the importance of task sheets, following instructions, doing what the teacher wanted, and meeting deadlines. Literacy assessment information was constructed as being important to their teachers for monitoring their performance and writing reports.

I Why do you do assessment in the classroom, why do you do these things these tasks?

C Becau::se we have to!

I Because you have to

Z So you can get marked on your report things

I So it builds up towards your reports?

T Oh, yes, makes it easier for your teacher to give you your final marks (Gang, 11/6)

Literacy assessment became serious:

B This has got more serious

J More business-like

B And it counts

T More work

I OK, Belinda first and then Julie

B It counts because it's more serious. Like in primary school it wasn't a major, it didn't like really affect your marks if it wasn't good or anything like you didn't have to put like heaps of work into it but this one you do. (Friends, 5/8)

Doing well in assessment became a matter of compliance with the teacher's expectations. When asked how they would advise incoming students about how to do well in assessment, the Quiet Achievers responded as follows

J Listen

M Follow the task sheets

I Mhmm and listen, any other ideas? (5.00) OK

Su Do the work

I Do the work

Si Umm, umm revise at night and stuff- that kind of thing (Quiet Achievers 3/12)

After an assessment piece had been submitted and marked by the teachers it was returned to the students in whole class feedback sessions. These sessions were dominated by teacher talk in which teachers constructed feedback in several ways. First, feedback was constructed as less important than introducing the task, or the processes and activities involved in completing the task. An example of talk from Fiona's classroom demonstrates this:

9 OK we're just about ready, normally I'd give back the umm the 10 assignments back towards the ends of lessons but I'm going to 11 give them back at the beginning today so that you can have a 12 quick look at those and think about those for a minute or two 13 before we go on to what you're going to be doing tomorrow 14 when you start your assessment. (S1: 20/5)

Opportunities to share work with other students were seen as an important part of the feedback process, although these were only brief, and not as important as the sharing of problem areas as in this excerpt from Elizabeth's classroom:

2 I know that you haven't had as much chance to sha:re your work 3 with each other so I'd like you to spend a little bit of time today 4 after I've gone through the--after I've handed them out and 5 gone through the problem areas with you I would like you to 6 umm spend a little bit of time just sharing your work with other 7 people because I'm sure that's something that you will enjoy 8 doing that and seeing what other people have done (S2: 16/4)

Second, feedback was an opportunity for the teachers to comment on their students' work. For these teachers what counted was their personal response to the work, the amount of effort put in by the students, and the final presentation of the students' assignments, for example when returning photo-essays to the students, Elizabeth commented:

31 First of all they were as I thought they would be, very interesting. 32 It was wonderful to see the sort of work that you're capable of 33 doing, the photographs were mostly very well composed, 34 very interesting photographs and it was great for me to be able 35 to see something of you and your life and your interests. And 36 a lot of work had been put into those assignments, I could tell that.

Third, feedback was public and general. During feedback sessions, the students had to locate their individual performances within the comments directed to the whole group. Finally, feedback was an account of the students' performance which summarised the students' efforts.

Fiona and Elizabeth also used feedback to familiarise students with acceptable and appropriate behaviours for receiving feedback and responding to the teacher's comments, for example:

8 F OK shssh. Sshssh OK when I give you your assessment out, 9 you're supposed to look--you're supposed to look at the 10 comment on the front and you're supposed to look in detail at 11 the underlined parts on the back, the criteria OK to find out how 12 you went. (S1: 16/4)

In this further example from Fiona's classroom these behaviours included listening quietly, sharing their work quietly and asking questions:

1 F Shu:sh I'll give these back for you to have a look at, now there won't

2 need to be a lot of umm discussion about them, that is loud discussion.

3 You may show your neighbours your results and if you've got any

4 questions about what you've got, we'll also have a look at that. (S1: 20/5)

In the next example, taken from a lesson where Elizabeth returned poetry anthologies to the students, Elizabeth positioned the students as needing to be responsible, quiet, and mature. The process of going through the results was constructed as a classroom collaboration between the teacher and the students:

17 In order for this to work properly you have to be really 18 responsible for yourself and your behaviour (2.0) OK so I don't 19 want anybody making a lot of noise or being silly while we go 20 through this process. It's an important process--you did it well 21 last time and I think that you can do it well again this time (S2: 14/6)

Student discourses about feedback were harder to discern than teachers'. This was mainly because students' talk was not privileged in feedback sessions, nor was student participation encouraged. Their limited topic initiations during feedback sessions focused on grades:

18 S I got a B

19 F Yes a B's good

20 S What about A--?

21 F A--is very good, you should be very pleased

22 S What does B--mean?

23 S Not quite a B

24 S Is C+ bad?

25 F Sorry B--? Oh, C+ (with a shrug) is a good mark

26 S Really?

27 F Shu:sh-sh! Just check the comments and you can just have 28 one or two minutes more to show your friends.

29 S How many As were there?

30 F A couple of As and one A--

31 S Who got them? (S1: 20/5)

The focus on grades evident in students' discourses reveals that these students did not share the same construction of feedback and appropriate questions about their work as their teacher and actively disrupted their teacher's intentions in giving them feedback (lines 27-28). For them, feedback was about establishing their place in the class and not about responses to their language performance.

In summary, the study found that there were multiple discourses related to literacy assessment in these two classrooms. It was also evident that these discourses were not always shared and were understood in different ways by the teachers and students. In the next section of this paper the findings are discussed.

Discussion

This study has shown that discourses related to assessment are complex and are influenced by many factors. In the classrooms the students were required to learn about and use multiple discourses. The study showed that these multiple discourses were introduced, maintained and reinforced during assessment events. However, the study also revealed that the discourses were not stable, making it difficult for teachers and students alike to establish a shared understanding of what was important in assessment.

In addition, the study revealed that assessment is more than using certain methods, tasks, and practices to monitor students' performance, it is also about defining teaching and learning activities, and building and maintaining relationships in the classroom. For example, when teachers (and systems) adopt new approaches to assessment or change their assessment methods, the impact of those changes on relationships must be considered.

The findings reveal that the teachers positioned themselves as audience, marker, commentator, corrector, all-round expert and guardian of the students' work. The students in turn were positioned by the teachers as receivers of information and as beneficiaries of their teachers' comments. This was particularly evident during feedback sessions. These sessions were potentially negative and limited learning experiences for many students in these classrooms. Students' participation was not encouraged and their talk was seldom acknowledged as important and/or salient. Overall, the findings reveal that teachers may underestimate the complexity of the feedback process and underutilise the opportunities feedback provides for enhancing learning. This raises issues both about the organisation of feedback and the focus of the talk during these sessions that need to be addressed if feedback is to become more effective.

The findings about students' increasingly negative discourses related to assessment are important and disturbing. Previous research has suggested that over the long-term, students' experiences with assessment, particularly if they are negative, affect students' attitudes towards assessment and consequently their academic performance (Paris, Lawnton, Turner, & Roth, 1992). This study has shown, first, that students may develop negative attitudes towards assessment within one year, and second, that the first year of high school may be crucial in establishing students' attitudes towards assessment that may affect their attitudes in subsequent year levels. The findings suggest strongly that it is not the content of the tasks or the processes involved in completing the tasks that led to disillusionment but the frequency and continuous nature of assessment, and the perceived need to comply with the teacher's expectations for the task.

Conclusion

The study revealed the importance of developing an understanding of literacy-related assessment from the perspectives of all classroom participants. This and future investigations of classroom-based literacy assessment practices will lead to a better understanding of how assessment can be utilised more effectively as a teaching and learning tool.

An earlier version of this paper was presented at the American Educational Research Association 2001 Annual Meeting, 10-14 April, Seattle, WA.

Appendix A

Research questions Data organisation Data

1) What do Year 8 1. Chronological and * Preliminary
 English teachers interwoven: Data study-survey
 think that students' arranged by date, responses
 previous experiences lesson, and time * Transcripts of
 of literacy order for the interviews
 assessment in school year * Transcripts of
 Year 7 have been? videotapes of
 classroom events
 * Fieldnotes from
 observations

2) What do Year 8 1. Chronological and * Preliminary
 English teachers interwoven: Data study-survey
 think that their arranged by date, responses
 students need to lesson, and time * Transcripts of
 know and understand order for the interviews
 about literacy school year * Transcripts of
 assessment in 2. Cluster: Data from videotapes of
 Year 8? the first unit of classroom events
 the school year, a * Fieldnotes from
 mid-year unit and observations
 the last unit of * Documents
 English for the
 year
 3. Talk about literacy
 assessment: Data
 arranged
 chronologically
 for the school year

3) What do Year 8 1. Chronological and * Preliminary
 English teachers interwoven: Data study-survey
 teach their students arranged by date, responses
 about literacy lesson, and time * Transcripts of
 assessment in high order for the interviews
 school. school year * Transcripts of
 2. Cluster: Data from videotapes of
 the first unit of classroom events
 the school year, a * Fieldnotes from
 mid-year unit and observations
 the last unit of * Documents
 English for the
 year
 3. Talk about literacy
 assessment: Data
 arranged
 chronologically
 for the school year

4) How do the literacy 1. Chronological and * Preliminary
 assessment practices, interwoven: Data study-survey
 methods and tasks arranged by date, responses
 teachers use at lesson, and time * Transcripts of
 the Year 8 level order for the interviews
 contribute to school year * Transcripts of
 socializing students 2. Cluster: Data from videotapes of
 into the assessment the first unit of classroom events
 culture of the the school year, a * Fieldnotes from
 school? mid-year unit and observations
 the last unit of * Documents
 English for the year
 3. Talk about literacy
 assessment: Data
 arranged
 chronologically
 for the school year

Research questions Analysis procedures

1) What do Year 8 * Coding of survey
 English teachers responses,
 think that students' generation of
 previous experiences frequency
 of literacy distributions
 assessment in * Development of
 Year 7 have been? categories and
 themes
 * Open, axial and
 selective coding
 Bogdan & Biklen,
 1992; Strauss &
 Corbin, 1990)
 * Transcript analysis
 (Gunnarsson, 1997;
 Potter & Wetherell.
 1994)

2) What do Year 8 * Coding of survey
 English teachers responses,
 think that their generation of
 students need to frequency
 know and understand distributions
 about literacy * Transcript analysis
 assessment in (Gunnarsson, 1997;
 Year 8? Potter & Wetherell.
 1994)
 * Development of
 categories and
 themes
 * Open, axial and
 selective coding
 Bogdan & Biklen,
 1992; Strauss &
 Corbin, 1990)
 * Triangulation and
 member checking
 * Text analysis
 (Fairclough, 1992;
 Gee, 1990)

3) What do Year 8 * Coding of survey
 English teachers responses,
 teach their students generation of
 about literacy frequency
 assessment in high distributions
 school. * Development of
 categories and
 themes
 * Open, axial and
 selective coding
 Bogdan & Biklen,
 1992; Strauss &
 Corbin, 1990)
 * Transcript analysis
 (Gunnarsson, 1997;
 Potter & Wetherell.
 1994)
 * Triangulation and
 member checking

4) How do the literacy * Coding of survey
 assessment practices, responses,
 methods and tasks generation of
 teachers use at frequency
 the Year 8 level distributions
 contribute to * Development of
 socializing students categories and
 into the assessment themes
 culture of the * Open, axial and
 school? selective coding
 Bogdan & Biklen,
 1992; Strauss &
 Corbin, 1990)
 * Transcript analysis
 (Gunnarsson, 1997;
 Potter & Wetherell.
 1994)
 * Text analysis
 (Fairclough, 1992;
 Gee, 1990)
 * Triangulation and
 member checking

References

Baker, C.D. (1991) Reading the text of reading lessons. Australian Journal of Reading, 14(1), 5-20.

Baker, C.D. (1992) Description and analysis in classroom talk and interaction. Journal of Classroom Interaction, 27(2), 9-14.

Baker, C.D. (1994) Constructing classroom literacies: A situated perspective. Education Views, 3(19), 13.

Baker, C.D. (1997) Literacy practices and classroom order. In S. Muspratt, A. Luke, & P. Freebody (eds.), Constructing critical literacies: Teaching and learning textual practice (pp. 243-263) St Leonards, NSW: Allen and Unwin.

Baker, C.D., & Freebody, P. (1989) Talk around text: Constructions of textual and teacher authority in classroom discourse. In S. DeCastell, A. Luke, & C. Luke (eds.), Language, authority and criticism: Readings on the school textbook (pp. 263-283). London: Falmer Press.

Bogdan, R.C., & Biklen, S.K. (1992) Qualitative research for education: An introduction to theory and methods (2nd edn) Boston MA: Allyn and Bacon.

Broadfoot, P. (1995) Performance assessment in perspective: International trends and current English experience. In H. Torrance (ed.), Evaluating authentic assessment: Problems and possibilities in new approaches to assessment (pp. 10-43). Buckingham: Open University Press.

Calfee, R. (1993) Assessment, testing, measurement: What's the difference? Educational Assessment, 1(1), 1-7.

Coles, M. (1998) Assessing reading: Changing practice. In M. Coles, & R. Jenkins, (eds.), Assessing reading 2: Changing practice in the classroom (pp. 1-8). London: Routledge.

Edwards, D. & Mercer, N. (1987) Common knowledge: The development of understanding in the classroom. London: Routledge.

Erickson, F. (1984) School literacy, reasoning and civility: An anthropologist's perspective. Review of Educational Research, 54(4), 525-546.

Fairclough, N (1989) Language and power. London: Longman.

Fisher, C.W., & Hiebert, E.H. (1990) Characteristics of tasks in two approaches to literacy instruction. Elementary School Journal, 91(1), 3-18.

Gee, J. (1990) Social linguistics and literacies: Ideology in discourses. London: Falmer Press.

Gee, J.P., Michaels, S., & O'Connor, M.C. (1992) Discourse analysis. In M. LeCompte, W.L. Millroy, & J. Preissle (eds.), The handbook of qualitative research in education (pp. 228-287) San Diego, CA: Academic Press.

Graue, M. E. (1993) Integrating theory and practice through instructional assessment. Educational Assessment, 1(4), 283-309.

Green, J. L. (1990) Reading is a social process. In J. Howell, A. McNamara & M. Clough (eds), Social context of literacy: Selected papers from the 15th Australian Reading Association Conference (pp. 104-123). South Carlton, Vic: Australian Reading Association.

Green, J. L., & Meyer, L.A. (1991) The embeddedness of reading in classroom life: Reading as a situated process. In C.D. Baker & A. Luke (eds), Towards a critical sociology of reading pedagogy. Papers of the XII world congress on reading. Amsterdam: John Benjamins.

Harrison, C., Bailey, M., & Dewar, A. (1998) Responsive reading assessment: Is postmodern assessment of reading possible? In C. Harrison, & T. Salinger (eds.), Assessing reading 1: Theory and practice (pp. 1-20). London: Routledge.

Johnston, P. H. (1992) Constructive evaluation of literate activity. New York: Longman.

Kleinsasser, A.M. (1995) Assessment culture and national testing. The Clearing House, 68(4), 205-211.

Kleinsasser, A., Horsch, E., & Tastad, S. (1993, April) Walking the talk: Moving from a testing culture to an assessment culture. Paper presented at the annual meeting of the American Educational Research Association, Atlanta.

McCarthey, S.J. (1994) Authors, text, and talk: The internalization of dialogue from social interaction during writing. Reading Research Quarterly, 29(3), 201-230. Mercer, N. (1995) The guided construction of knowledge: Talk amongst teachers and learners. Clevedon, Avon: Multilingual Matters.

Murray, J. (1994) Judge and jury or guide and co-traveller? The changing face of assessment. Literacy learning: Secondary thoughts, 1(1), 44-54.

Myers, J. (1992) The social contexts of school and personal literacy. Reading Research Quarterly, 27, 296-333.

Nisbet, J. (1993) The United States of America. In Curriculum reform: Assessment in question. Paris: OCDE.

Nuthall, G. (1997, July) Learning how to learn: The social construction of knowledge acquisition in the classroom. Paper presented at the European Association for Research in Learning and Instruction (7th Biennial Conference), Athens.

Paris, S.G., Lawton, T.A., Turner, J.C., & Roth, J.L. (1992) A developmental perspective on standardised achievement testing. Educational Researcher, 20(5), 12-20.

Potter, J. (1997) Discourse analysis as a way of analysing naturally occurring talk. In D. Silverman (ed.), Qualitative research: Theory, method, and practice (pp. 144-161). London: Sage.

Potter, J., & Wetherell, M. (1994) Analyzing discourse. In A. Bryman, & R.G Burgess (eds), Analyzing qualitative data (pp. 47-67). London: Routledge.

Prentiss, T.M. (1995) Constructing literacy practices: An analysis of student lived and perceived experiences in high school English. Journal of Classroom Interaction, 30(2), 27-39.

Strauss, A., & Corbin, J. (1990) Basics of qualitative research: Grounded theory procedures and techniques. Newbery Park, CA: Sage.

Talty, F. (1995) Small talk around big books: Interaction or conversationalism? The Australian Journal of Language and Literacy, 18(1), 5-18

Theobald, P,. & Mills, E. (1995) Accountability and the struggle over what counts. Phi Delta Kappan, 76(6), 462-467.

Turner, J.C., & Paris, S.G. (1995) How literacy tasks influence children's motivation for literacy. The Reading Teacher, 48(8), 662-675.

van Kraayenoord, C.E. (1996) Literacy assessment. In G. Bull & M. Anstey (eds), The literacy lexicon (pp. 235-247). Sydney: Prentice Hall.

Wolf, D., Bixby, J., Glenn, J., & Gardner, H. (1991) To use their minds well: Investigating new forms of student assessment. In G. Grant (ed.), Review of Research in Education (Vol. 17, pp. 31-73). Washington, DC: American Educational Research Association.

Karen Moni, Christina E. van Kraayenoord and Carolyn D. Baker

SCHOOL OF EDUCATION, UNIVERSITY OF QUEENSLAND

(1) Excerpts in paragraph format have been taken from transcripts of interviews

(2) Excerpts with line numbers have been taken from transcripts of videotaped classroom interactions