The Fantastic Four of mathematics assessment items: Jane Greenlees makes reference to four comic book characters to make the point that together they are a formidable team, but on their own they are vulnerable. Read on to find out the Fantastic Four of Assessment and how they work together.

Greenlees, Jane

Standardised tests are becoming an ever increasingly important presence in schools today. In fact, national measures of academic performance have the potential to be the driving force behind teaching practice (Lowrie & Diezmann, 2009; Pedulla, Abrams, Madaus, Russell, Ramos & Miao, 2003). In 2008, the introduction of the National Assessment Program (NAPLAN) in Australia increased the emphasis on national standards and benchmarks and created greater forms of accountability. Student performance became a reflection of the teacher, staff, principal, school and State. With such high stakes involved, it can be anticipated--based on international evidence--that teachers will abandon personal teaching beliefs and focus on "teaching to the test". However the Ministerial Council for Education, Early Childhood Development and Youth Affairs (MCEECDYA) argues that success is found in teaching numeracy basics as "the tests reflect core elements of state and territory curricula".

What it fails to acknowledge is that the entire nature of test design has changed dramatically in recent years and has moved beyond simply mathematical content. As Lowrie and Diezmann (2009) maintain, "substantial data is obtained (and reported) on student performance on mathematics, but rarely do we consider whether the tasks actually assess student knowledge and numeracy understandings" (p. 8). In this paper, I examine the four components of assessment items and the need for implicit instruction within the classroom for student success. Just like the "Fantastic Four" of Marvel comics fame, each component has its own unique 'power' that makes them strong enough to stand alone but as a group can be a force to be reckoned with. These are mathematical content, literacy demand, contextual understanding, and graphics (see Figure 1).

[FIGURE 1 OMITTED]

1. Mathematical content

Mathematical content is at the heart of all assessment, examining what the students know. They are the fundamental basics children are taught throughout their school career and are often the standards used by parents to ascertain how well their child is doing. While a national curriculum is still being developed in Australia, its foundation has been laid in the release of the National Statements of Learning for Mathematics (NSL) (MCEETYA, 2006) in conjunction with the NAPLAN tests. While most curriculum documents of Australian education jurisdictions are organised into stages or levels, the NSLs focus on the four years used within the national assessment. They are Years 3, 5, 7 and 9. Subsequently these year levels are structured around "five broadly defined and inter-related aspects of Mathematics curriculums that are considered essential and common" (MCEETYA, 2006, p. 3).

These are:

* Working mathematically--problem solving, representation and modelling, investigating, conjecturing, reasoning and proof, estimating and checking the reasonableness of results or outcomes (see Figure 2);

* Number--representation and models for number, counting, magnitude, order and exact or approximate calculations (see Figure 3);

* Algebra, function and pattern--working with functions and relations applied to everyday and mathematical objects, patterns in number and space, and general forms (rules, formula, tables, graphs, equations and equivalences) expressed using words, symbols or diagrams (see Figure 4);

* Measurement, chance and data--length, area, volume, angle, mass, time, temperature, probability and statistics (see Figure 5);

* Space--identification, classification and representation of 2D and 3D shapes and objects.

[FIGURE 2 OMITTED]

[FIGURE 3 OMITTED]

[FIGURE 4 OMITTED]

[FIGURE 5 OMITTED]

It is generally assumed that effective teaching of mathematical basics would be reflected in student success in standardised tests. However by failing to acknowledge other components of an assessment item and the need for explicit instruction in the classroom, students are disadvantaged and subsequently results affected. As Perso (2009) notes, "one particular concern that has been raised in relation to the items on the tests relates to the literary demands inherent in understanding the requirements of the tasks" (p. 14). Therefore a student may know the mathematics but lack the ability to coherently read the question.

2. Literacy demand

When reading the words in mathematics, just as within any language, children need to contend with and understand its many conventions. These include developing meaningful and applicable definitions of mathematical terminology, identifying words that may have one meaning in normal discourse and a different meaning in the context of a mathematics problem, and the use of homophones and similar sounding words. For example, in the 2009 Year 3 NAPLAN the word "flipped" was used as shown in Figure 6. It has been argued that this term has an everyday meaning and a different mathematical meaning that students need to identify appropriately (Lowrie & Diezmann, 2009). Therefore children who have never been exposed to this term in a mathematical sense would be disadvantaged in such an assessment.

As such, learning scenarios need to be provided for students to practise and become familiar with mathematical jargon and concepts. This includes utilising strategies similar to literacy tools; e.g., building vocabulary through the use of dictionaries, thesauruses, and discussing, sharing and tabulating meanings of words (Perso, 2009). Teachers also need to provide opportunities for students to unpack the vocabulary embedded in mathematical tasks through specific processes associated with "working mathematically" including questioning, communicating and reasoning.

[FIGURE 6 OMITTED]

Another problem is the superficial attention children give to the text within a task by simply focusing and finding relevant keywords (Lowrie & Diezmann, 2009). As such, "this can hinder students' holistic understanding of the task, and hence, the rationality and correctness of their answers" (Weist, 2003, cited in Lowrie & Diezmann, 2009, p. 9). It is therefore important to reinforce the need for students to read thoroughly and carefully through questions and relevant information, as well as utilising step-by-step strategies before attempting to solve them--in other words, less rush more fuss.

Being numerate, therefore, is more than being able to perform the mathematics: it also requires a certain degree of literacy skill. While many teachers focus on thinking and working mathematically in the classroom, students also need to be taught how to "read mathematically" if they are to be successful in the current assessment environment. In a similar way, teachers also need to acknowledge the role the context of the question plays in a student's understanding of an item.

3. Contextual understanding

The NSL outline in the Year 3 Working Mathematically strand states that students will "actively investigate everyday situations as they identify and explore mathematics" (MCEETYA, 2006, p. 5). The concept of making mathematics 'real' by incorporating everyday contexts has been a growing trend in schools for the past 30 years (Boaler, 1994). It is believed that such an approach would help students realise the relevance of mathematics as it is applied to their world outside the classroom. Consequently, mathematics is no longer presented as a dry, abstract subject of 'absolute truths' but rather one which encourages flexibility and creativity thus enhancing motivation, confidence and enjoyment (Walkerdine, 1989, cited in Boaler, 1994).

As a result, test designers are attempting to make questions more realistic and possibly authentic. For example, the 2010 Year 5 NAPLAN included a question where children needed to calculate the date of the next meeting if they are held on the first Tuesday of each month and the last meeting was on the 6 March. No calendar was used within the question; it only contained the written instructions. As such, students needed to have an understanding of the formation of weeks, months and calendar conventions in order to draw the right conclusion. However it is believed that this can be problematic for some students, particularly low performing students who draw upon their everyday knowledge that is not specifically relevant to the task. On this occasion, many students overly relied on their understanding of the number of days in a month, identifying March as having 31 and April having 30, and subsequently made the meeting date 5 April, one day earlier than the month before. This example highlights the need for various contexts to be utilised in a classroom so students can practise applying appropriate strategies and understandings rather than resorting to routine and familiar concepts. This goes beyond teaching basic mathematical understanding to applying it to real-life situations.

However it is also argued that mathematics in context often requires the student to "suspend reality and ignore their common-sense in order to get a correct answer" (Boaler, 1993, p. 556). In the 2007 NAPLAN practice questions, an atypical graphic (Diezmann, 2008) was included, with a grid being overlaid on a map of Australia to identify large areas. However, in the 'real' world, grids are more typically used on regions with similar characteristics (e.g., geographic terrain, States or Territories). The result of this kind of situation is categorised as "context conditioning" (Boaler, 1994). Therefore student performance is often a result of how well they recall a method that last proved successful in a similar question, not necessarily how well they apply mathematical understanding. Teachers need to acknowledge that while the use of contexts can motivate students, they will only enhance learning transfer if they are able to offer a "realistic and holistic view of mathematics which makes sense to students both in the classroom and in the real world" (Boaler, 1994, p. 559).

What also needs to be recognised is the important role individuals play in creating meaning and imagery as they "adopt, develop and invent sign-uses in the contexts of teaching, learning, doing and reflecting on mathematics" (Ernest, 1997, p. 1). This is particularly relevant when considering mathematical assessment, as it is often assumed that all children will have a similar knowledge and understanding of mathematical signs, symbols and language.

The symbols and language used to describe mathematics concepts are certainly not static and the meaning created and constructed from one word or symbol can be strongly influenced by who the child is. Yet children often have little chance to represent these possibilities in a test situation and are simply given the option of a right or wrong response. Every child is unique and it therefore cannot be assumed that in mathematics there is a generic code by which all students will abide. For example, a child in a metropolitan city and a child in a small rural town are exposed to vastly different cultures. Therefore the inclusion of a train timetable in the 2009 Year 5 NAPLAN (see Figure 7) could be viewed as advantaging one group over another. For one child, the item may seem foreign, included only in textbooks, while for another it is a part of their daily lives. Contexts cannot offer a unique meaning to all students and therefore when used within an assessment must necessitate recognition that different students are likely to interpret the same mathematical situation in many different ways.

However what also needs to be acknowledged is the increasing number of graphical and visual representations embedded within assessment items and the impact on student performance.

4. Graphics

Graphics can be categorised as graphs, maps, diagrams, charts and tables (Diezmann & Lowrie, 2007). Within this technological age they have become increasingly important in representing and organising information (Harris, 1996, cited in Lowrie & Diezmann, 2007). There have been a growing number of graphical and visual representations embedded within assessment items. An analysis of the 2009 Australian National Mathematics Assessment revealed that 23 out of 35 (66%) questions in the Year 3 and 26 out of 40 (65%) in the Year 5 test contained a form of graphic. With such a high prevalence in national tests, there needs to be more explicit teaching practices in reading, producing, understanding and decoding graphical representations. This includes focusing and teaching the three levels of graphical comprehension as noted by Curcio (1987):

1. Reading the data--requires the literal reading of the graph.

2. Reading between the data--includes the interpretation and integration of the data in the graph, e.g., the ability to compare quantities. This is the level of comprehension most often assessed on national tests.

3. Reading beyond the data--requires the reader to predict or infer from the data information that is neither explicitly nor implicitly stated in the graph. These questions seem to be the most challenging; examples are hard to find and therefore are not utilised in the classroom. For many primary-aged students, the comprehension of the graphic may be the most demanding aspect of a mathematics task. If the student is unable to access and interpret the information effectively, the actual mathematics involved is irrelevant. Thus, "students' performance may be a measure of their ability to comprehend the graphical (or linguistic) components of a task rather than a student's knowledge of the mathematics within the task" (Lowrie & Diezmann, 2009, p. 8).

Friel, Curcio and Bright (2001) suggest a guideline for creating a progression of sequential development of graph comprehension in K-8 grade levels:

* Grades K-2: object graphs, picture graphs, line plots, bar graphs (with use of grid lines to facilitate reading; labeling of bars with numbers);

* Grades 3-5: bar graphs (stacked or using multiple sets of data), stem plots, pie graphs (reading primary emphasis);

* Grades 6-8: pie graphs (reading and constructing), histograms, box plots, line graphs.

Friel, Curcio and Bright (2001) recommend that developing mathematics knowledge should progress along a continuum K-8, as well as the complexity of the data. They also note that in order for students to gain a deep knowledge of graphs and to make and use graphs effectively, they need instructional materials that are carefully constructed. This includes providing opportunities for students to read beyond the data by creating their own graphs, working backwards and gaining a true appreciation of the structure and design of the graphic.

Conclusion

In today's technological age, information is represented in visually rich modes, with quite different language, context and graphics demands. Multiple representations in graphs are regularly found in computer games and in the media. It is not surprising that recently introduced national instruments have followed a similar theme. Therefore if students are to be successful they need to be explicitly taught all aspects of these representations since the interrelated nature of the design components can be complex "in their own right". This includes incorporating all aspects of the curriculum in teaching, drawing attention to the use of mathematical terminology, exploring the complexity of graphics and incorporating different contexts within the classroom.

Similarly, test designers need to pay particular attention to making questions accessible, clear and fair. With the federal government seemingly determined to extensively publishing student results, more research needs to be conducted to examine the design of mathematics items and its impact on student performance. If testing is going to be given such a high priority, we need to closely examine what we actually are assessing and whether it is a true indication of students' mathematical knowledge.

References

Boaler, J. (1994). When do girls prefer football to fashion? An analysis of female under achievement in relation to realistic mathematics contexts. British Educational Research Journal, 20(5), 551-564.

Boaler, J. (1993). The role of contexts in the mathematics classroom: Do they make mathematics more real? For the Learning of Mathematics, 13(2), 12-17.

Curcio, F. R. (1987). Comprehension of mathematical relationships expressed in graphs. Journal for Research in Mathematics Education, 18, 382-393.

Diezmann, C. M. (2008). Graphics and the national numeracy tests. In M. Goos, R. Brown & K. Makar (Eds), Proceedings of the 31st Annual Conference of the Mathematics Education Research Group of Australasia (Vol. 2, pp. 659-662). MERGA: Sydney

Ernest, P. (1997). Semiotics, mathematics and mathematics education. The Philosophy of Mathematics Education Journal, 10. Accessed at http://www.ex.ac. uk/~PErnest

Friel, S. N., Curcio, F. R. & Bright, G. W. (2001). Making sense of graphs: Critical factors influencing comprehension and instructional implications. Journal for Research in Mathematics Education, 32(2), 124-158.

Lowrie, T. & Diezmann, C.M. (2009). National numeracy tests: A graphic tells a thousand words. Australian Journal of Education. 53 (2), 141-158.

Lowrie, T. & Diezmann, C. (2007). Solving graphics problems: student performance in junior grades. The Journal of Educational Research, 100(6), 369-377.

Ministerial Council on Education, Employment, Training and Youth Affairs (MCEETYA) (2010). National assessment program: Literacy and numeracy. Year 5 numeracy test. Melbourne: Curriculum Corporation.

Ministerial Council on Education, Employment, Training and Youth Affairs (MCEETYA) (2009a). National assessment program: Literacy and numeracy. Year 3 numeracy test. Melbourne: Curriculum Corporation.

Ministerial Council on Education, Employment, Training and Youth Affairs (2007b). National assessment program: Literacy and numeracy. Numeracy Year 3 sample questions. Retrieved 6 February 2008 from: http://www.naplan.edu.au/verveZ_ resources/NAP08_num_y3_web_v12.pdf

Ministerial Council on Education, Employment, Training and Youth Affairs (MCEETYA) (2006). Statements of Learning for Mathematics. Retrieved 2 November 2009 from: http://www.mceecdya. edu.au/verve/_resources/SOL_Maths_Copyright_update2008.pdf

Pedulla, J. J., Abrams, L. M., Madaus, M. K., Russell, M. K., Ramos, M. A., & Miao, J. (2003). Perceived effects of state-mandated testing programs on teaching and learning: Findings from a national survey of teachers. Boston, MA: National Board on Educational Testing and Public Policy. Retrieved 28 October 2009 from: www.bc.edu/reseacrh/nbetpp/statements/nbr2. pdf

Perso, T. (2009). Cracking the NAPLAN code: Numeracy and literacy demands. Australian Primary Mathematics Classroom. 14(3), 14-18.

Jane Greenlees

Charles Sturt University

<jgreenlees@csu.edu.au>

Figure 7. The role of context (MYCEETYA, 2009, Year 5
Numeracy test, Item 14).

This is a train timetable.

                     DEPARTURE TIMES

Central    6:20 am    9:50 am   2:52 pm   7:00 pm
Rainer     6:31 am   10:01 am   3:03 pm   7:11 pm
Bradford   6:43 am   10:13 am   3:15 pm   7:23 pm
St Marks   6:53 am   10:23 am   3:25 pm   7:33 pm

What time does the last morning train depart from Central?