What you test is what you get: (WYTIWYG for short).

Morony, Will

A version of this paper was presented as one of several 'provocative papers' to the Connections and Continuity conference conducted in December 2014 by the AAMT working in partnership with the Australian Council of Deans of Science (ACDS). The focus of that conference was to explore the interface between school and university mathematics. Whilst the paper therefore takes as its starting point the Year 12 end-of-year examinations that have very high stakes for students' futures, the points it makes about the influence of assessment on what is valued as mathematics apply throughout schooling.

Each year in October and November, the latest group of school leavers completes their final tasks as school students. For many, that means formal examinations conducted by local examination authorities.

A decreasing number of those school leavers will have sat examinations in mathematics, with an even more select group doing a two or three hour 'advanced' mathematics paper. This paper argues that examinations and testing create a distorted view of mathematics among our young people and, indeed, the whole society. We need to do better. There needs to be a mandated 'package' of assessment in the senior years of schooling so that we get more of what we want.

If you are able I would like you to pause and have a look at one or more of those papers. Looking at a paper or two from another state could well be instructive as they tend to be different. But that will hardly be a surprise. The webpage of the Australasian Curriculum, Assessment and Certification Authorities (ACACA; http://acaca.bos.nsw.edu.au/go/ list-of-acaca-authorities/) has links to the organisations responsible for examinations. Copies of papers can be accessed through their websites .... pause while dutiful reader does as requested.

Now that you have refreshed yourself with what these papers are like I would like to explore a few issues that arise for me when I look at them. Before I do, I would like to note that some readers will have been part of setting such papers. They will know far better than me the constraints and pressures that serve to make them what they are. The influences are political, epistemological, social, historical and more. This paper should in no way be taken as critical of the people who set and administer these papers. The context makes the papers what they are--that some are really quite good for their type is a testament to those people's creativity.

What image of mathematics does the paper paint? What aspects of the 'essence' of mathematics does the paper portray? What does the paper miss? For me it is a somewhat bleak picture. And remember that this is what we offer to our best and brightest as the culmination of a successful set of experiences in mathematics throughout their schooling.

Consider what you observe in the examination papers in the light of the following statement about mathematics taken from the Rationale of the Senior Secondary Australian Curriculum: Mathematics.

   Mathematics is the study of order, relation and pattern. From its
   origins in counting and measuring it has evolved in highly
   sophisticated and elegant ways to become the language now used to
   describe much of the modern world. Statistics is concerned with
   collecting, analysing, modelling and interpreting data in order to
   investigate and understand real world phenomena and solve problems
   in context. Together, mathematics and statistics provide a
   framework for thinking and a means of communication that is
   powerful, logical, concise and precise.

The timed written examination is a cornerstone of assessment in mathematics in the senior years of high school. It is a genre with strengths, especially when the stakes are high--as they are in end of school assessments in the context of ATARs and university entrance. Exams guarantee that it is the student's own work. Allocation of marks and adding the student's scores to get a single summary number gives a sense of objectivity. The examiner can control the difficulty of questions that hopefully results in an appropriate spread of student scores.

However, the genre of the timed written examination in mathematics is necessarily limited in what it can assess--all modes of assessment have limitations. Examinations have a focus on skills and techniques. The limited time means that examinations can only assess students' abilities across the range of areas seen as important to mathematics in the quote above in limited ways, if at all. Yet those who are deeply engaged with mathematics know that it is much more than tests and exams, and the picture of mathematics they most often paint. In a very real sense, tests and examinations make important that which is assessable, rather than making that which is important assessable. And WYTIWYG.

The timed written examination is a colossus in its influence on mathematics teaching and learning in schools around the world--witness NAPLAN testing for children as young as 8. Views of what mathematics is about that NAPLAN begins to establish will be validated for these young people a decade later when they sit one of the papers you have just had a look at. But remember that unless things change, and we all hope they do, perhaps 90% of the people with whom they sat their Year 3 NAPLAN won't be sitting the advanced mathematics paper with them. Is this an accident? Is it inevitable? No and no.

Perhaps more telling is the impact of the timed written examination on the assessment of students' mathematics learning throughout their schooling. The 'Maths test' is an abiding negative memory of school mathematics for many adults, and failure often their last experience in the subject. The attrition noted above is undoubtedly a consequence, as is the societal acceptance of comments like "I was never any good at Maths."

I acknowledge Professor Hugh Burkhardt from the Shell Centre in the UK for the title of this paper. He first used the acronym in 1987, at a time when "what you see is what you get" was a goal for word processors. For the more than two decades I have known him, Burkhardt has highlighted the influence of examinations on curriculum and promoted his view that the best, perhaps the only, way to effect real change in the teaching of mathematics is by changing the assessment. He advocates modest, carefully planned changes in examinations as a positive lever for change--the 'assessment tail wagging the curriculum/teaching dog'.

So what change might we make to assessment in mathematics in our schooling systems?

The full assessment package at all year levels consists of all the internal and external assessments undertaken for different purposes --tests, assignments, projects, problem solving tasks, presentations etc. Together, the package must include tools that assess across the range of what mathematics is about. This is easy to say, but very difficult to achieve in practice. However, creative teachers and schools in Australia and overseas will have examples of good quality assessment in mathematics, and more can be developed over time through research and development. Each school could then design their assessment package to meet it needs in knowing their students' learning.

That would change teaching and learning in schools in ways that result in our best and brightest--and all the others--leaving school with a more rounded appreciation of mathematics and its practice than most do today. And a greater likelihood of seeing the point of continuing to study the subject and related fields.

Will Morony

AAMT inc.

<wmorony@gmail.com>