Reaction to Chris Sangwin’s paper John Monaghan University of Leeds

Introduction Chris Sangwin has provided CAME participants with an interesting overview of STACK, a computer aided assessment system (CAA) which includes a computer algebra system (CAS) to perform a variety of tasks including syntax checks on students’ input. Chris is very knowledgeable in his field and is respected in the UK and internationally; indeed, the Mathematics Department of my university recently consulted him when they were considering introducing CAA. Chris’ paper is an informative overview but it does not have a central focus other than description. I thus select some issues Chris addresses and does not address which CAME 5 participants may wish to pursue in the discussion part of our meeting. On CAME and research A major concern of CAME is with research on the use of computer algebra in mathematics education. We rely on development work by mathematicians, computer scientists and teachers but our focus is on education research. I take a broad view of what research is – systematic enquiry – that includes teacher-research and action research. Research is, however, more than just description. I do think there is much which can be researched on the use of CAA and a question I would like to put to Chris and CAME participants is What research agendas are contained in this paper? Chris raises important questions on p.5 but then states “these larger issues are not addressed in this paper”. Perhaps we should address them in the discussion group. On assessment Chris introduces formative assessment and a paper by Wiliam and Black in paragraph 2. Black and Wiliam have written a number of groundbreaking papers on assessment, especially on formative assessment. Black and Wiliam (1998) reports on the work of Butler on teacher feedback to students where they compare grade only, comment only and grade and comment teacher feedback. If you are not aware of this work, then take a minute to think – which of these is likely to be the most effective with regard to assessment to improve learning? The answer requires a number of qualifications (see Black and Wiliam, 1998, for details and further reference) but, simplistically, the answer is ‘comment only’. My question regarding STACK’s feedback to students is How might STACK’s feedback to students be improved and how might we research this? On context In the conclusion of his paper Chris states that CAA/CAS can support a pedagogy of practice and can encourage informal group work and that virtually immediate feedback can provide students with an incentive to reflect upon their answers. Chris does not elaborate on this but it is something we might address in the discussion group. There are at least two aspects to consider: how these ‘can’ statements might be realised, e.g. putting some detail on how CAA/CAS can encourage informal group work; how we might research these issues. To address these aspects I feel we need to consider CAA with regard to the wider student experience: teaching, learning with regard to the module CAA supports, learning with regard to students’ degree (as the centrality of mathematics to their studies undoubtedly matters) and

other forms of assessment. These are things of which Chris does not provide us with details. Without these details I fail to see how we can appreciate or assess the value of STACK to student learning, which leads to the question What do we need to know about students’ experience in order to assess the value of STACK? On tasks and design I think the basic structure of questions given on p.6 has been well thought out. I particularly like the set of stationary point questions given at the bottom of p.5 which are clearly designed to get students thinking deeply about the algebraic form of functions with specific properties. But privileging question types inevitably means that distinct question types are not privileged. This will happen with any assessment (computer aided or not) or, indeed, as Kendal and Stacey (1999) point out, with teaching. With CAA, however, the question type is not the choice of the teacher but of the CAA designer who is likely to be greatly influenced by what the computer can do. With regard to stationary points, when I teach this topic I often: include graphic work, e.g. sketching a function and asking for sketches of the first and second derivative; ask students, in groups, to state what the second derivative tells us about the original function. In recapping the topic I often set a trick question to catch out those who think that if the second derivative at a point is zero, then the function has a point of inflection at that point. Whether I set these tasks depends on my interaction with the group. Question which arise with students working with STACK are How do STACK questions interrelate with other tasks students are set? What are teachers opinions of the STACK questions? Lagrange (2005) describes research on students using Casyopée, software with a CAS kernel. This software is designed for teaching and learning rather than assessment. With Casyopée parameters in the software can be manipulated. A question for Chris in the first instance is Is there scope for user manipulation of parameters in STACK? Lagrange (ibid., 144) notes that “‘Traditional’ software design in mathematics educational research often put a strong emphasis on the analysis of mathematical content, and take teaching practices and curriculum into account only as a second dimension.” Last December saw the 17th ICMI study conference on the theme ‘technology revisted’ (see http://www.math.msu.edu/~nathsinc/ICMI/). One of the four conference themes was Design of Learning Environments and Curricula, and Chris was a member of that group. There are many schools of thought on software design. Something that is not clear to me from Chris’ paper which I would value Chris’ opinion of is Where does the design of STACK feature in the spectrum of design approaches? The anthropological and the instrumental approach CAME 5 is the first CAME symposium not to include input on the anthropological and/or the instrumental approach. I partially remedy this omission by briefly noting areas of the work Chris introduces for which these approaches could provide insight. My CAME 4 paper (Monaghan, 2005) provides an introduction to and critique of both of these approaches. An important construct in the instrumental approach is instrumental genesis, a process over time which links the affordances and constraints of the tool to the user’s prior understandings and activity. A study of students’ STACK instrumental geneses would be of undoubted interest.

A central feature of the anthropological approach is that educational practices are described in terms of Chevallard’s ‘tasks, techniques, technology and theory’ construct. I focus on ‘tasktechnique’ as this relates to the previous section. Researchers in this field note that tasks and techniques are linked, that tasks are artefacts constructed in educational settings and that certain techniques are institutionally privileged. These researchers differentiate between pragmatic and epistemic values of techniques (pragmatic values concern breadth of application of a technique; epistemic values concern the role of techniques in facilitating mathematical understanding). A consideration of Chris’ p.5 stationary point task with regard to these constructs may be illuminating. Computer vs paper assessment Do students do equivalent items in the same way when the items are presented and answered on a computer and on paper? NB Let us assume that, like STACK, the student cannot harness the processing power of the computer, e.g. the student cannot directly use, say, a CAS. This question is addressed in a recent paper (Threlfall et al., 2007). They work with younger students than Chris does but there may be issues for us to consider in their work. I focus on one of the questions they report on, Circles, to illustrate issues arising. Circles states “Here is a grid with eight circles on it”. The paper-based item states “Draw two more circles to make a symmetrical pattern” and the computer-based item states “Move the two extra circles on to the grid to make a symmetrical pattern”.

On paper, the circles cannot actually be drawn until after a decision has been made about where they should go, because of the mess resulting from a change of mind. The pupil needs to decide that it will look right without being able to try it properly, so has either to be able to visualise, or be analytic – for example by matching pairs across possible lines of symmetry. Either of these involves dealing with elements simultaneously, with high demands on working memory. On computer, the pupil can put the two circles on and make a judgement by recognition – does this arrangement look symmetrical? If not, he or she can move them elsewhere (or, if he or she cannot remember which circles were placed and which were already there, can “start again”). The sequential affordance of the computer medium brings a smaller ‘cognitive load’ and enables easier success – by recognition of symmetry rather than through visualisation or by analysis. If pupils are willing to try things out, the question is assessing whether they recognise symmetry when they see it. The implicit aspect of the paper assessment is that a desirable understanding of symmetry is more than just the ability to recognise it when one sees it,

Threlfall et al. (pp. 8-9) comment

but also should incorporate elements of visualisation and/or analysis. If that is accepted, then the activity afforded by the computer is not legitimate for the assessment, and the computer question is less valid as an assessment item.

I think it is very difficult to anticipate differences in how students answer questions in different media and I would not like to speculate about potential differences in the work Chris presents. I do, however, think that research on CAA into this area would be very useful. Endnote CAA is in its infancy but it is clearly one aspect of the future of teaching, learning and assessing mathematics with ICT/CAS. Any criticisms of Chris’ paper are made with this, ‘in its infancy’, comment as his defence. It is, however, important that we step into this future with a critical mind towards CAA. References Black, P. and Wiliam, D. (1998) Assessment and classroom learning. Assessment in Education, 5(1), 7-75. Kendal, M. and Stacey, K. (1999) ‘Varieties of teacher privileging for teaching calculus with computer algebra systems’. The International Journal for Computer Algebra in Mathematics Education, 6(4), 233-247. Lagrange, J-b. (2005c). ‘Curriculum, classroom practices and tool design in the learning of functions through technology-aided experimental approaches’. International Journal of Computers for Mathematical Learning, 10(2), 143-189. Monaghan, J. (2005) ‘Computer algebra, instrumentation and the anthropological approach’. Available from http://www.lonklab.ac.uk/came/events/CAME4/index.html Threlfall, J., Pool, P., Homer, M. and Swinnerton, B. (2007) ‘Implicit aspects of paper and pencil mathematics assessment that come to light through the use of the computer’. Accepted for Educational Studies in Mathematics and available at the time of writing via http://www.springerlink.com/content/t61556153k435553/fulltext.pdf