• Embed Doc
  • Copy Link
  • Readcast
  • Collections
  • CommentGo Back
Download
 
Session M2G
978-1-4244-4714-5/09/$25.00 ©2009 IEEE October 18 - 21, 2009, San Antonio, TX39
th
ASEE/IEEE Frontiers in Education ConferenceM2G
-
1
Formative Teaching: A Conversational Framework for Evaluating the Impact of Response Technologyon Student Experience, Engagement andAchievement
Samuel O. King, and Carol L. Robinson
Loughborough University, S.O.King@lboro.ac.uk, C.L.Robinson@lboro.ac.uk 
 Abstract 
 
-
There are two compelling constraints on thedelivery of effective engineering mathematics educationat university: Decline in the mathematical preparednessi.e. confidence and ability of engineering students, andthe predominance of the passive learning-leaningtraditional lecture mode of instruction. In this paper, wepresent a formative teaching paradigm, consisting of theuse of specific question types and implemented via theuse of Response Technology systems, whose adoptioncould help overcome these constraints. To evaluate theimpact of this paradigm on student experience,engagement and achievement, we adopted a mixed-methods research protocol, consisting of multiplesurveys, interviews, and observations. This evaluationprocess was guided by the application of the Laurillardlearning model, designed for evaluating learningtechnologies. The evaluation results show that theparadigm facilitates
meaningful 
behavioural andcognitive engagement, provides a richer and morelearner-centred atmosphere, and contributes to areduction in the number of students failing. However,the impact on student attendance and overall meanacademic grades is negligible.
 
 Index Terms
- formative assessment, active learning, studentexperience, engagement, learning theory, evaluation,clicker.
1.0
INTRODUCTION
 
The ‘mathematics problem’ - the perceived decline in themathematical confidence and competency of students,together with the reasons for the decline, has been welldocumented [1,2]. Compounding this mathematical‘morass’ is the predominance of the traditional lecture modeof teaching in university classrooms, a mode which researchshows is less than optimal for active learning [3,4). Tofacilitate active learning in lectures, a number of research-informed practices, which are often technology-mediated,have been initiated [5,6]. These platforms are oftenimplemented or mediated through the application of Response Technology (RT) systems, also known as clickers,electronic voting systems, et cetera. The ample literature onRT systems indicates that their use could make lectures,especially in large classes, more interactive, facilitate higher student engagement, and increase participation [7-12].In this paper, we present the results of an empiricalstudy on the impact the use of the RT systems has had onstudent learning at an English university. The specific focusis on the use of the systems for the teaching of engineeringmathematics (i.e. mathematics to engineering students) totwo cohorts of second-year engineering students over the2007/2008 and 2008/2009 academic sessions.
1.1 Background 
The RT systems being used are the TurningPoint [13] radio- based electronic voting systems. These systems enable aninstructor to poll students, and for student responses to besubmitted live and anonymously. Student responses aresubsequently displayed in the form of a suitable chart e.g.histogram, bar chart, et cetera – based on an instructor’s preference. The TurningPoint handsets are minimalist indesign, durable and easy to use. However, only multiple-choice questions (MCQs), requiring alphanumeric responsesmay be used with the handsets (handsets allowing text-entryhave been used in another context, but this is not the focusof this study). The handsets are not used to register or monitor student attendance, and questions are set for purelyformative assessment purposes – assessment is therefore notgraded. Further, students do not have to purchase thehandsets. Students collect the handsets at the beginning of alecture and hand them over at the end.
1.2 Research Questions, Significance and Outline
This study was designed to answer the following researchquestions:
(I)
What is the impact of RT on studentengagement and experience of learning mathematics?
(II)
 What is the impact of RT on student achievement, and therole of feedback in the learning process?
(III)
How may theConversational Framework, Laurillard’s learning model [4] be applied to the evaluation of the impact of RT use?
 
Session M2G
978-1-4244-4714-5/09/$25.00 ©2009 IEEE October 18 - 21, 2009, San Antonio, TX39
th
ASEE/IEEE Frontiers in Education ConferenceM2G
-
2
The specific research contributions of this paper include:
(I)
An evaluation of the impact of RT use onstudent learning in mathematics through the application of the Laurillard’s conversational framework;
(II)
Highlight of the relationship between RT use and student academic performance;
(III)
An evaluation of the impact of RT use onspecific components of student experience and engagement.The outline of this paper is as follows: Section 1 is theIntroduction; Section 2 describes the methodological procedures adopted for this study; and Section 3 describesthe formative teaching paradigm. Section 4 presents theapplication of the Laurillard theoretical framework to theevaluation of RT usage impact, and this framework helps totheoretically contextualise the results presented in Sections5, 6 and 7. Section 8 is the concluding section.
2.0
METHODOLOGY
 
A mixed-methods research protocol was adopted for thisstudy. Reasons for this approach [14] include a need toensure the validity and viability of results by designing theresearch around the use of multiple research methods,incorporating both quantitative and qualitative approaches.Further, a triangulation approach, defined as the use of morethan one method to investigate a research question, wasadopted. Third, a review of RT literature indicated that amajor downside to the research conducted in this domain isthe tendency to base findings on data collected from the useof one research method [8,9]. To make a meaningfulcontribution to the field, we felt that it was important thatwe anchor our data collection on a more rigorous researchmethodological platform.
2.1 Sample
This consisted of engineering students on a mathematicsmodule. The students are second-year engineering studentsi.e. the penultimate year in the English system, and aretypically adjudged to be, overall, above averageacademically, with a significant number having obtained anA grade in mathematics prior to admission to the course.This sample was selected for the study for two reasons [14].First, one of the authors has been implementing theformative teaching approach with two cohorts of theseengineering students over a two-year period. Second,another instructor who regularly uses RT had also taughtstudents on this module in a first year class. The students aretherefore not new to the use of RT, and it may therefore beexpected that the Hawthorne Effect would have less of animpact on their perceptions of RT usage.
2.2 Methods
These consisted of a 13-question survey, consisting of a mixof open-ended and closed questions; a one minutequestionnaire, based on the one minute paper concept [6];semi-structured interviews with individual students;classroom observations of RT in use; a staff blog fromwhich we gleaned insights related to student learning, whichsubsequently informed the 13-question survey; and informalfeedback from students and staff. The discussions inSections 4 and 5 are based on research data from thesemethods, with student comments (in quotation marks)inserted where appropriate.
3.0
FORMATIVE TEACHING
 
This refers to the intentional design and creation of alearning environment where active student questioning andfeedback is an integral part of the lecture experience for students in a class. Our approach is a variant of the QuestionDriven Instruction (QDI) paradigm [15], a variant because peer discussion is not a major goal – although students haveoften been asked to discuss their answers with their peers. Atypical lecture might feature four to eight multiple-choicequestions (Table 1), depending on the topic and the learningoutcomes envisioned for that particular lecture. A criticalcomponent of this teaching paradigm is the readiness of theinstructor to adapt the teaching, both in a particular lecture,and if required, for subsequent lectures, in response tofeedback from student submissions to questions.
TABLE 1
A DESCIPTION OF THE RT
-
BASED QUESTIONS THAT HAVE BEEN USED FOR FORMATIVE TEACHING IN UNIVERSITY MATHEMATICS
’
CLASSES ATLOUGHBOROUGH UNIVERSITY
 
MCQ Type Purpose
T
EASERS
 
To create a ‘light’ /relaxed atmospherein class; For classroom management
R 
EVISIONS
 
To identify student prior knowledge e.g.from first year mathematics course and;
 
To highlight common student
 
misconceptions e.g. about specifictopics from previous instruction
I
NTRODUCERS
To introduce a topic - such that it getsstudent thinking in a different way thanthey are accustomed to about aparticular topic
A
PPLICATIONS
 
To assess understanding of recentlycovered material
C
ONCEP
T
ESTS
 
To assess conceptual understanding of asub-topic deemed fundamental or onethere is evidence students usually havedifficulty understanding
R 
EPEATERS
 
Repeat practice – the repeating of aquestion, which students found difficultat the first attempt, in a slightly variedmode to assess improvement in student
 
understanding of the concept assessed
 
in the initial question
In general, implementing the formative teaching paradigmrequires:
•
 
Redesign of the curriculum to identify thelearning outcomes – this would, for example,include identification of the topics/sub-topics that
 
Session M2G
978-1-4244-4714-5/09/$25.00 ©2009 IEEE October 18 - 21, 2009, San Antonio, TX39
th
ASEE/IEEE Frontiers in Education ConferenceM2G
-
3
students routinely find difficult, common studentmisconceptions, incorporation of feedback from(previous) RT questions, etc. This is an iterative process within the formative teaching paradigm
•
 
Creation and design of (good) questions – thisshould lead to the attainment of the learningoutcomes by students. However, this is the mostdifficult part as creating really good questionstakes considerable skill, time and (voluminous) practice [12,15]
•
 
Adoption of an effective approach, based onteaching style preference(s) and theoreticalideology. It is possible to have two lecturersapplying formative teaching, but with one moredemonstrably effective in engaging students thanthe other, based on differences in delivery i.e. pedagogical content knowledge [16].
4.0
CONVERSATIONAL FRAMEWORK 
 
The motivation for applying Laurillard’s ConversationalFramework is to explain and illuminate, from a theoretical perspective, the factors that account for the observed impactof RT use on student learning [7,9]. Laurillard basicallyadvocated that for learning to occur, there must beconversation or dialogue between the instructor andstudents. Further the theory has been specificallyformulated for the evaluation of learning technologies[4,10,17]. The best way to illustrate the Framework is byshowing how the different elements encapsulated in Figure1 are achieved via the RT-enabled formative teaching platform. We would illustrate this with an example of a task set in class.
4.1 Application Context 
Task:
The task is the question, ‘Without integrating,evaluate
3421
4
 y x xy
dydx
=== =
∫ ∫ 
 This task was given, via RT, in the first lecture of amathematics module for second-year automotive/aeronautical engineering students;
Context:
 Skill with relating double integrals to real world objects for conceptual understanding of the topics (volumes of solids,centres of mass, moments of inertia, et cetera) that thestudents, trainee engineers, would encounter on the module;
Task goals
:
(I)
To get students thinking of real worldapplications when they work with integrals;
(II)
To getstudents thinking throughout the whole module on how theycould apply seemingly abstract mathematical constructs tosolve problems in the discrete world;
(III)
To see how manystudents found the double integral difficult to evaluate because it had been observed, from previous teachingexperience, that some students often struggled with relatingit to the volume of a cuboid.The
‘Teacher’s ideas’ 
represents instructor’s currentContent Knowledge (CK) [16] and Pedagogical ContentKnowledge (PCK) of topic i.e. integration, while
‘Learner’sideas’ 
represent current student understanding and prior knowledge of subject. The use of RT for polling studentsdepicts the ‘practice environment’ while students’engagement with answering the questions in class represents‘
 Learner’s practice’ 
.
FIGURE 1
A MODIFIED VERSION OF LAURILLARD
’
S CONVERSATIONAL FRAMEWORK FOR EVALUATING LEARNING TECHNOLOGIES
[17]
4.2 Applying the Framework: Stage 1
The instructor creates and presents the question (task) basedon current CK/PCK (Teacher’s ideas) to achieve the task goals identified above. The learner receives the instructionand thinks about working out the solution. This first step(especially if the task is changed from a question to beattempted in class to a worked example) occurs in mostmathematics lectures at university.
4.3 Applying the Framework: Stage 2
(I) Adaptation:
Before and whilst a learner is actuallyattempting the question - instead of making a mental note asis the case in the provision of worked example by instructor - s/he has to
adapt 
the previous ideas held about the subject(in this case, always solving integration in a linear,decontextualised manner) in the light of the instructions provided, and the question's demands. Hence thisadaptation utility is absent in a typical lecture because thereis no opportunity for students to practice on their own.
(II) Practice:
This is when a student attempts the question by mentally processing the question, and this would requireactive and structured drawing on (learner’s) ideas andmaterial presented in the lecture and/or instructions provided in order to arrive at a solution. This deliberate practice [3] helps reinforce targeted mathematical skills andidentify gaps in knowledge and practice because it serves adiagnostic, self-assessment function. This principle is particularly poignant in mathematics because students learnlargely by (thinking and) doing.
of 00

Leave a Comment

You must be to leave a comment.
Submit
Characters: ...
You must be to leave a comment.
Submit
Characters: ...