See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/309073543

Evaluation of Nokia Mobile Maths (Imfundo yami/yethu) Project Executive

Summary

Technical Report · January 2009

CITATIONS READS

6 71

2 authors, including:

Nicky Roberts
University of Johannesburg
63 PUBLICATIONS   103 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Mobile learning in resource constrained contexts View project

ChildConnect View project

All content following this page was uploaded by Nicky Roberts on 02 March 2018.

The user has requested enhancement of the downloaded file.

 Evaluation of the Imfundo yami/yethu
Project:
Executive Summary

August 2009
Nicky Roberts (Kelello Consulting) on behalf of
Neil Butcher and Associates

1
Contents

Contents ................................................................................................................................................................................2
Introduction ..........................................................................................................................................................................3
Project Context ...............................................................................................................................................................3
Evaluation Research Design .........................................................................................................................................4
Summary of Findings...........................................................................................................................................................5
How Can a ‘MXit-Based’ Model for Mobile Technology be Used to Support For mal Mathematics
Education in South African Grade 10 Classes?..........................................................................................................7
How Can an ‘SMS-Based’ Mobile Technology Solution be Used to Support Formal Mathematics Education
in South African Grade 10 Classes?.......................................................................................................................... 10
How Do These Uses of Mobile Technologies in Grade 10 Mathematics Classes Impact on Teachers’ Work
Load? ............................................................................................................................................................................. 14
How do These Uses of Mobile Technologies in Grade 10 Mathematics Classes Impact on Learners’
Attitudes to Mathematics?........................................................................................................................................ 15
How Do These Uses of Mobile Technologies in Grade 10 Mathematics Classes Impact on Learners’
Competence in Mathematics? .................................................................................................................................. 20
Reco mmendations ............................................................................................................................................................ 23
Conclusion .......................................................................................................................................................................... 28

2
Introduction
In this executive summary, the evaluation design is briefly described and then the main
evaluation findings are summarized in relation to the primary evaluation questions:
• How can a ‘browser–based’ mobile technology solution be used to support formal
Mathematics Education in South African grade 10 classes?
• How can an ‘sms–based’ mobile technology solution be used to support formal
Mathematics Education in South African grade 10 classes?
• How do these uses of mobile technologies in grade 10 mathematics classes impact on:
– Teacher’s work load;
– Learners’ attitude to mathematics; and
– Learners’ competence in mathematics?
The answers to these questions draw together the data presented in Chapter Four of the full
evaluation report, to provide our analytical interpretation of the evaluation findings. Where
appropriate, implications of the findings are identified. We provide our recommendations on
what we think ought to be considered when planning and implementing such interventions in
the future.

Project Context
In 2008, the South Africa’s Deputy President’s office requested a project exploring the use of
mobile technologies in supporting formal education in South Africa. This was in support of
work being done by the Presidential International Advisory Commission on Information
Society and Development (PIAC on ISAD), and the recommendation of the e-Skills Council
for government action to raise the level of mathematical competencies among its learners.

The basic rationale for such a project was that, as mobile telephone penetration among South
African learners is very high, it seemed sensible to explore and present a ‘proof of concept’
for how those devices could be harnessed for learning purposes in a formal education setting.
In addition, as mathematics is a priority learning area in formal education, such exploration
should be done within this learning domain. It was recognized that most trials that have
sought to harness mobile devices for formal education have been short-term pilots testing
different technical possibilities. It was felt that they had lacked proof of sustainability and
affordability issues for wider usage in formal education in South Africa.

As such, Nokia, with the support of Nokia Siemens Networks (NSN), led a project team in
conceptualizing a pilot project which tested and measured the efficacy of using mobile
technologies to support learning of grade 10 mathematics in South African schools.

The aim of the learning concept of the project was for the mobile technology intervention to
help teachers and learners, and to increase collaboration amongst learners and between
learners and teachers. In addition, it was felt that there would be value in considering two
different scenarios. The first uses low-level technologies (a ‘short message service (SMS)
model’ solution) which have low bandwidth and handset requirements. The other uses
higher-end technologies (a ‘MXit model’ solution). The former was expected to use SMS,
Unstructured Supplementary Service Data (USSD), and MobilEdu technologies, while the
latter would combine Mobile Browser, MXit, TC3 Mobile and Mobile Edu technologies. The
following were identified as potential learning components of the two envisaged solutions:

3
competition, exercises, 1 gaming, video clips, audio clips, collaboration, tutoring, SMS
reminders, and test features.

In terms of the pilot project scope, six schools (two in eac h in the North West, and the
Western Cape, one in Gauteng and one in Mpumalanga) were selected for inclusion in the
interventions. A grade 10 mathematics class in each school was selected to participate and
their mathematics teachers would be trained and supported in use of the browser-based
solutions. One grade 9 class in a Western Cape school was used to pilot the SMS- model.

Evaluation Research Design

The Imfundo yami/yethu project evaluation was intended to:
• Describe the project intervention conducted at each school, collating data pertaining to
technologies used, training interventions and the project process at each school.
• Measure changes in:
– Nature and volume of teachers’ work;
– Learners’ attitude towards mathematics; and
– Levels of mathematical competence amongst learners.

The evaluation team gained a detailed understanding of the project by conducting a review of
project planning documentation and interviews with the project leader and consortium
partners on the envisaged project and its intended outcomes.

The above was collated into a detailed description of the project and its intended outcomes
and was used to inform the activities that followed.

The evaluation team felt that, in measuring efficacy of an intervention, it was important to
locate the planned approaches within an understanding of the educational paradigm being
adopted. Therefore, using the detailed description of the project, a mathematics education
expert reviewed the planned pedagogical approaches. This review was intended to provide
the project team a basis for reflecting on the initial project design of the project from a
pedagogical rather than technical perspective, in order to inform future plans. It was also
intended to provide the evaluation team a theoretical framework to guide analysis of data
emerging from the project evaluation process.

The selection of schools was made by Nokia and the project team, and they also secured
permissions from the relevant Departments of Education to undertake the interventions and
conduct evaluation research at those school.

Once such permission was obtained, evaluation research school visits were conducted at each
of the six schools participating in the project. For the MXit model classes, these visits were
conducted on two occasions: an initial school visit was conducted near the outset of the
project, and an endpoint school visit was conducted towards the end of the project. As the
grade 9 SMS-model class was only introduced into the project when the grade 10 classes had
already started the project, it was agreed that there would only be one school visit for this
school. The instruments used for the grade 10 classes to considering initial and end-point

1 Note on terminology. ‘Exercise’ refers to a set of 5-10 questions on a similar mathematics topic, to which learn ers
respond.

4
reactions to the project, were adapted to allow for a single administration of one set o f
instruments. During these visits, the evaluation researcher:
• Interviewed the school leader;
• Administered a teacher questionnaire with the grade 10 mathematics teacher;
• Interviewed the grade 10 mathematics teacher;
• Administered the learner questionnaire with the grade 10 mathematics learners; and
• Conducted a focus group with a selection of the grade 10 mathematics learners.

The learner assessment of mathematics competence was administered by the teacher and
researcher at the school a few days after the sc hool visit. This assessment took the form of a
timed, written test, similar to those required for regular school assessment. An exemplar
standardized assessment was designed by Lorraine Burgess, of the Western Cape Department
of Education. The NBA evaluation team then revised the exemplar assessment to ensure that
it was independently developed from the project team.

Analysis of this data was conducted by first developing detailed case studies for each of the
six MXit model classes (presented in Annexure Two of the main report). Following this, the
trends and weighted average 2 responses, when looking across the six ‘MXit model’ case
study classes, were identified and described. This was followed by a detailed description of
the grade 9 SMS-model class, where the approach to using only SMS in the absence of a
browser-based interface was piloted. The data gathered for this pilot class, was compared
wherever feasible to the trends that emerged across the six MXit model classes. These
evaluation findings are described in detailed in Chapter Four of the main report.

Finally the evaluation questions were answered. These represent a summary or the main
evaluation findings, as well as an outline of their implications. These are presented in Chapter
Five of the report and repeated in this executive summary for ease of reference.

Summary of Findings

The Imfundo yami/yethu Project Described

The Imfundo yami/yethu project was a proof-of-concept project requested by high- level South
African government structures and invo lving a form of public-private partnership to
implement it. It hoped to evaluate how almost ubiquitous ownership of mobile telephones by
learners could enhance mathematics teaching and learning. The main aim of the initiative was
to support educators and learners in their endeavours to teach and learn mathematics.

The project comprised three groups of partners: local, global, and Finnish. Local partners
were organized through a project steering group, which comprised the President’s Office,
members from government Departments of Education, Communication, and Science and
Technology, the Government Communication and Information System (GCIS),
representatives from the Meraka Institute, the Midrand Graduate Institute, and South Africa -
Finland knowledge partnership on ICT (SAFIPA), a project sponsor, and Neil Butcher and

2 Weighted average has been used for learn er data through out this chapter. For each ‘MXit model’ case study class,
the average learner response was calculated. Looking across the six class, an average o f the average class responses
was then calculated. This is the w eighted average. This was used as the unit of analysis w as the case study class. It
was felt that th e absolute average of learner responses, would be less helpful, as he differen ced betw een case study
classes was of greater relevan ce.

5
Associates. Nokia functioned as the project leader, and provided expertise on pedagogical
and technical issues. The Finnish partners, Bitville Oy and Mobiletools International Ltd,
provided technical and pedagogical expertise.

Selection of potential content vendors was conducted through the steering group, and
involved ‘benchmarking’ the potential vendors against the envisaged ‘framework for learning
and technology’. MXit, Dr Maths, and WebAlt were selected as content and delivery
partners. The learning context and process for the project were envisaged as having four
components: digital citizenship; training for teachers; information for and involvement of
parents; and curriculum development (including m- learning support).

Two models of mobile learning intervention were considered: the ‘MXit-based’ model and
the SMS-model. In the ‘MXit-based’ model learners had, or were given, mobile telephones
which could access the Internet. They used MXit Instant Messaging to access and respond to
specially designed mathematical content on identified topics, through the WebAlt application.
They also had access to Dr Maths, a tutorial support service making use of SMS via MXit.
The SMS-model was a much simpler technical offering, and could be conducted on any type
of mobile telephone. Interaction with the WebAlt mathematics content was via SMS, which
was more expensive per interaction than was the case for the ‘MXit-based’ model. The types
of content with which learner could interact were limited, and they did not have access to Dr
Maths. The SMS- model was also used for administration of mathematics homework
exercises.

Six schools participated in the project, one each from Gauteng and Mpumalanga and two
each from North West and the Western Cape. Four schools were selected to participate in the
project because they had been adopted by Nokia and NSN. Two schools were selected
because of their high ICT adoption levels, learners who have consistently produced good
results, and the functionality of the schools, which is very good and supported by good
leadership. All six schools were involved in the pilot of the ‘MXit-based’ model. For the
SMS-model only one of the six schools was involved. While the ‘MXit-based’ model pilot
ran for fifteen school weeks and involved seven grade 10 mathematics classes 3 across six
schools, the SMS- model ran for seven school weeks and involved one grade 9 mathematics
class at one of the six schools.

The Finnish WebAlt content was adapted for the South African grade 10 mathematics
curriculum by a mathematics curriculum advisor of the Western Cape Education Department.

Both teachers and principals from the participating schools attended a two day training
session to orient them to the programme and familiarize them with the technology to be used.
After the formal training, technical support and further teacher support was facilitated by the
project team for each school.

The project partners identified strengths of the project at the end of the conceptualization
phase, immediately prior to implementation. These included that the project:
• Combined strong partners in an innovative process, for which the time was right;
• Made use of mobile technology which was familiar to, and liked by, learners;
• Allowed for learner and teacher interaction with the WebAlt and Dr Maths application
that was quick and easy;

3 School A h ad two classes particip ating in the MXit-based model, only one of which w as in cluded in this evaluation.

6
• Enabled learners to work at their own pace, anytime, and anywhere and have access to
support;
• Was valid in a South African context; and
• Would ‘hopefully result in a high quality of evaluation of the results’. 4

When reflecting on project weaknesses, partners commented on the tight implementation

timeframes. This was particularly with regard to there not being enough time for teacher
training, or to test the technology and concepts properly. There was also concern that the
project duration was too short to measure results effectively. Other weaknesses identified
were that the project relied on a network and that the mobile device required for t he MXit-
based Model was more expensive than other available handsets. One of the partners
considered planning and project management to be a project weakness.

Prior to implementation, the WCED curriculum advisor raised concerns about the
pedagogical approach being adopted in the project:
The content at present is mainly ‘drill and practice’ type. The content [should] include
investigations and explorations as well. The type of question is mostly multiple-type
questions. Thus a wider variety of question type would be great.5
This is particularly significant, as it supports critique of the project provided in the
independent pedagogical review, as well as critical analysis of the learner attitude to, and
learner competence in, mathematics, that is presented below.

How Can a ‘MXit-Based’ Model for Mobile Technology be Used to

Support Formal Mathematics Education in South African Grade 10
Classes?

What was the ‘MXit-Based’ Model?

The ‘MXit-based’ model required use of mobile telephones that could access the Internet. It
involved a unique integration of WebAlt mathematics content with the Instant Messaging
(IM) client, as well as with mathematics experts through Dr Maths. It allowed for use of the
following learning components: competitions, exercises, gaming, video clips, audio clips,
collaboration, tutoring, SMS reminders, test features, and self assessment. It was expected
that the learner or the teacher would log into MXit and then access the WebAlt content which
was transmitted from WebAlt to MXit. MoMaths, a programme in the WebAlt application,
was used to provide content. The project aimed to cover identified topics in the national
curriculum for grade 10 mathematics: number patterns, products, factorizing, algebraic
fractions, and equations and inequalities.

For each topic, learners were expected to complete tasks by responding to SMS prompts from
their teachers, as well as determine which exercises they wanted to do by logging on to MXit
and selecting quizzes they wanted to do. Teachers could use the SMS manager function on
their laptop computers to send group messages to tell learners which activities to complete.
Question options provided for both multiple-choice and written answers. Feedback on
whether or not the answer was correct was provided immediately. It was originally intended
that there would be video and audio explanations of some of the mathematics problems, if
learners chose to get the explanation. Activities/quizzes were designed to range in complexity

4 Project p artner questionnaire, SAFIPA, March 2009.

5 Project p artner questionnaire, WCED, January 2009

7
as learners completed them. As part of the model, teachers could access the results of
exercises completed by learners, seeing how long each learner took to complete tasks and
how many attempts were made on each question. This information was intended to be used
for remediation and to determine the course of action in the teaching and learning. In addition
to using WebAlt, learners making use of the MXit-based model were encouraged to use the
services of Dr Maths, a tutoring service offered by volunteer students using text messaging
via MXit. This model included the possibility to link to multiple content vendors in the
future.

How Did Schools Respond to the ‘MXit-Based Model’?

Principals, teachers, and learners were asked to explain their understanding of the purpose of
the project. Responses from both principals and teachers reflected their overall understanding
that the project aimed to make use of mobile technologies to support mathematics learning. In
addition, some teachers also emphasized that the technology being used was familiar to the
learners. In a similar manner, from the learners’ perspective, the project purpose was to use
mobile telephones to improve their mathematics. However, several learners made the way in
which this was to be done more explicit, with some mentioning Dr Maths and MoMaths or
referring to the focus on homework and supplementary support.

The teachers, principals, and learners were asked in general terms to explain how their school
had responded to the project. Most principals and teachers were positive about the project,
indicating that they thought it had been a success. There was generally consistent agreement
from teachers and principals that the project had had a positive effect on learners. This was
mentioned in relation to changes in learner attitudes toward mathematics, their working
harder or spending more time on mathematics homework, and the project impacting on their
mathematics results. In particular, there were specific comments about how the project had
motivated and excited learners, and ensured that they did more mathematics homework.

Most teachers and principals did not seem to know for certain how their school was selected
to become part of the project. Some put forward their assumptions on this, and all described
having been contacted by the Department of Education to request their involvement.

Some teachers and principals were deliberate in distinguishing different types of learner
responses, indicating that, while some learners had been very engaged with the project, others
had not or had lost interest in it over time.

At two schools – School D and School E – technical problems seem to have dogged
implementation and uptake of the project. This was reported by both principals and teachers.

According to teachers’ and principals’ reports, parents seem, in general, to have been
supportive of the project. Those who were initially sceptical became supportive of the project
when they saw the benefits to learning mathematics. It was positive that , in some schools, a
meeting was held to inform parents about the project.

In relation to teacher training offered to teachers and principals, it seems that the logistical
arrangements were appropriate and that the training was well received. However, there
remained a sense that some teachers did not know exactly what the project aimed to achieve
and did not feel confident in their roles when back at school. Some teachers indicated that
they would have liked a greater focus on mathematics teaching, and less focus on technology.
Most felt that they needed more support beyond the initial training session.

8
Most learners indicated that they had found the project very useful and interesting. Of course,
there, were learners who were not as happy about the project. These learners either were not
happy because they did not receive mobile telephones or they reported that they were
experiencing too many technical problems.

What we re the Identified Stre ngths of the ‘MXit-Based’ Model?

Project partners identified the following as strengths of the MXit-based model, just prior to
project implementation. They reported that the Imfundo yami/yethu project:
• Provided an opportunity for learners to collaborate, communicate, and compete in a
positive manner;
• Made use of MXit technology which was familiar to, and liked by, learners;
• Provided access to good content through the web resources and exercises in the WebAlt
application, as well as the possibility of using new media, such as videos, to explain
content;
• Provided access to Dr Maths support;
• Was scalable in relation to other subject areas, as well as with regard to large-scale
implementation; and
• Had small and cheaper costs in comparison to the SMS-model.

‘MXit-based’ model teachers and principals identified several strengths of the project. They
reported that the Imfundo yami/yethu project:
• Improved learners’ attitudes to mathematics, making mathematics fun to learn;
• Enabled learners to work at their own pace, anytime and anywhere;
• Made use of the WebAlt application, which provided automatic marking of learners’
work, giving immediate feedback and reducing the teachers’ marking workload ;
• Allowed for quick and easy learner and teacher interaction with the WebAlt application;
• Improved learner performance in mathematics;
• Made use of mobile technology which was familiar to, and liked by, learners;
• Supported active learning, encouraging involvement or interactive engagement in the
learning process
• Provided access to Dr Maths tutoring service;
• Provided access to technology by donating mobile telephones to learners who did not
have their own one, or one with Internet access; and
• Had several pedagogical strengths, such as that it improved homework submission and
completion, encouraged learners to think deeply, and enhanced mental calculation skills
and computation speeds.
The above list is presented in order of most frequent mention. That the project improved
learner attitude to mathematics was the most frequently mentioned strength, while each
pedagogical strength that appears in the final bullet was only mentioned by one person. It is a
positive reflection on the project that some participants identified particular pedagogical
strengths of the project. However, each one was an isolated comment, reflecting that such
shifts were not consistently observed or planned for.

What Were the Identified Weaknesses of the ‘MXit-Based Model’?

When project partners were asked to identify weaknesses of the ‘MXit-based’ model, they
expressed concern that this model required more expensive mobile telephones and was
dependent on a mobile network.

9
The following list presents ‘MXit-based’ model project weaknesses as identified by teachers
and principals, again listed from most frequently to least frequently mentioned. The
weaknesses of the Imfundo yami/yethu project were that it:
• Experienced technical problems and provided inadequate technical support;
• Suffered from network connectivity problems;
• Required more problems of greater complexity in the MXit/WebAlt application;
• Experienced difficulties with the speed and efficiency of Dr Maths;
• Suffered from insufficient airtime, as learners could not make use of the applications
without this;
• Experienced some difficulties with the speed, efficiency and accuracy of MXit/WebAlt
applications;
• Experienced technical problems at the initial set up phase;
• Did not provide equal access to mobile technology, as not all the learners received
donated mobile telephones; and
• Had project timelines which were too short or inappropriate.

How Can an ‘SMS-Based’ Mobile Technology Solution be Used to Support

Formal Mathematics Education in South African Grade 10 Classes?

What was the SMS-Model?

In the SMS-model, it was expected that WebAlt content would be transmitted to the learner’s
mobile telephone via SMS or Unstructured Supplementary Services Data (USSD) and that
the learner would complete the quiz and send it back to the teacher by SMS. Any type of
mobile telephone could be used to implement this model. The SMS-based model was far
more limited than the MXit-based model. Learners did not have access to competitions,
gaming, video and audio clips, collaboration, tutoring, or self assessment. It was expected
that there would be greater ongoing cost to this model per learner, as SMSs were used and not
MXit. However, this had to be weighed up against initial costs, as less expensive handsets are
required for this model.

How Did the Grade 9 Pilot Class Respond to the SMS-Model?

The teacher was enthusiastic about the SMS- model, noting that she had been able to identify
which learners were having serious problems with mathematics. She made it clear, however,
that only some of the learners had benefitted from the project, as others remained reluctant to
do their homework even when it was provided to them on their mobile telephones.

The SMS- model teacher appreciated her use of a laptop and data projector in the project, and
that 16 of her learners had received mobile telephones for use in the project. She thought that
the technology used was appropriate. Although she did not attend the initial teacher training
sessions, she was happy with the training and support provided to her during the project by
the curriculum advisor and her colleagues in School F taking part in the ‘MXit-based’ model.

The teacher collaborated with the curriculum advisor on the WebAlt content to be used, and
felt satisfied that it matched her grade 9 curriculum requirements.

The SMS-model learners seemed less enthusiastic about the Imfundo yami-yethu project than
their grade 10 counterparts. This difference in learner attitude to the project may be a result of
various factors. First, as this was a grade 9 class, learners had not yet made an individual

10
choice to study mathematics rather than mathematical literacy. Their overall attitude to
mathematics as a subject could therefore be expected to be less positive than grade 10
learners, all of whom had chosen to study mathematics. Second, as these learners were in
School F where there were two grade 10 classes engaged in the ‘MXit-based’ model, they
may have felt that their SMS-model was less sophisticated than the ‘MXit-based’ model.
They were aware that they had a reduced technology offering, and complained that they
would have liked to use MXit and Dr Maths. Third, the SMS- model itself seems to have been
less effective. It was planned fairly late into the project, and acknowledged by partners not to
have been as well considered as the ‘MXit-based’ model. It also seemed to strive to replicate
the ‘MXit-based’ model using less sophisticated technology, which has limited its potential.

What Were the Identified Strengths of the SMS-Model?

When the project partners identified strengths of the SMS-model, they reported that the
Imfundo yami/yethu project could make use of more accessible, basic technologies which
could be used in the rural areas where mobile networks may not be strong.

The strengths of the SMS- model as identified by the teacher were similar to many of the
strengths identified by teachers for the ‘MXit-based’ model. The SMS- model teacher
reported that the Imfundo yami/yethu project:
• Made use of the WebAlt application which provided automatic marking of learners’
work, giving immediate feedback and reducing the teachers’ marking workload ;
• Made use of mobile technology which was familiar to, and liked by, learners.
She also felt that it was a strength that the project provided learners with mathematics
exercises. It was, however, notable that she did not mention the two strengths that were most
commonly cited by the MXit- model respondents, those being that the project:
• Improved learner attitude to mathematics, making mathematics fun to learn; and
• Enabled learners to work at their own pace, anytime and anywhere.
This supports the observation that the grade 9 learners were less enthusiastic about the
project, in relation to their attitude towards mathematics, as well as how much they used the
project offerings (for the reasons outlined above).

What Were the Identified Weaknesses of the SMS-Model?

Some project partners reported that the SMS-model was not really an option for the future.
This was because it was not scalable as it was too expensive, and did not allow for complex
content or long questions. It only provided limited support and services and was not very
user-friendly.

In contrast to the weaknesses identified for the ‘MXit-based’ model, weaknesses identified by
the SMS- model teacher did not include technical problems, lack of technical support, or
network connectivity problems. This may be because the technical offering was less
sophisticated, resulting in fewer problems. It may, however, also have been a result of
technical problems with the ‘MXit-based’ model having already been addressed by the time
the SMS- model was piloted. The absence of reported technical problems is therefore an
important strength of this model, although it should be noted that it was tested in Cape Town
where network connectivity is more extensive than in other areas.

However, the SMS- model only made use of the Vodacom network. As such, problems
experienced in relation to connectivity for other networks (as experienced in the ‘MXit-
based’ model) were not of concern. On this issue, the SMS-model teacher thought that

11
limiting airtime to Vodacom as the only network option was a weakness of the SMS- model.
She also felt that the SMS- model was expensive. This concern covered two distinct issues.
First, ongoing airtime costs of the SMS- model were thought to be expensive in comparison to
the MXit-based model, where each SMS was significantly cheaper. Second, the SMS- model
teacher was concerned that each individual quiz item in the WebAlt application required a
new SMS response as this made it more expensive to answer a question set.

How Much did Learners and Teachers Use the WebAlt/MXit Application During the
Project?
The ‘MXit-based’ model learners made consistent and regular use of the WebAlt/MXit
application throughout the project. For most classes, the extent of this use declined during
this time. Over the 15 weeks, learners were posting on average four times per week. 6 The
classes in Schools B and C embraced the technology more enthusiastically than other
classes. 7 Learners at Schools D, E, and F made the least use of the application. 8 However, all
classes made regular use of the application. Learners’ positive response to the application was
evident in the fact that, in all classes, learners continued to use the application after the
project officially closed, with School C continuing to make use of it extensively in the week
after closure.

SMS-model learners used the WebAlt/MXit application less than their ‘MXit-based’ model
counterparts, posting on average 1.69 times per week. This was comparable to average use by
School E, which, of the ‘MXit-based’ model classes, made least frequent use of the
application. Learners in the SMS- model class continued to use the application for three weeks
after the official project close, during the period for which usage data was still collected.

‘MXit-based’ model teachers also made regular and consistent use of the application. For
these classes, the average teacher occurrences 9 in the MXit/WebAlt application log per week
was more than 53, which shows that there was relatively consistent and frequent use of the
application by all participating teachers. 10

As mentioned above, principals and teachers at School D and E reported that technical
problems had hampered project implementation. This was reflected in learner and teacher
usage data. Learner usage was lowest at School E. 11 Following intervention from the district,
it appears that there was a concerted effort from the teacher at School E to improve learner
engagement. In fact, the School E teacher made the most teacher use of the application. This
may have been a result of her attempts to improve learner engagement with the project. It is,
however, notable that, by the time of the endpoint visit to this school, the teacher reported
that none of the applications were being used due to technical problems.

6 Weighted average across the six ‘MXit-based’ model classes of ‘average posts per learner per week’ = 4.00 (SD =
3.15).
7 School C ‘average posts per learner per w eek’ = 8.75; School B ‘average posts per learn er p er week’ = 7.25.
8 School D ‘average posts per learner p er w eek’ = 1.5; School F ‘average posts per learner per w eek’ = 1.85 ; and

School E ‘average posts per learner per w eek’ = 2.08.

9 Teach er use was measured b y counting the number of o ccu rren ces of the teach er’s user name in th e MXit/WebAlt

usage logs over the project period, and dividing this by the number of active project weeks for each class (14 or 15
weeks).
10 Average number of teach er o ccu rren ces in the MXit/WebAlt log across the six MXit-based model schools = 53.7

(SD = 14.6)
11 Average number of posts per learn er p er week for School E was 1.5.

12
In School D, the teacher made use of the application more than the teachers in School B and
School C (where there was such enthusiastic uptake from learners). The teacher in School B
made least use of the application, but this use was still frequent and consistent, as on average
there were more than 30 occurrences of user activity per week from this teacher. From this,
we can infer that all teachers played their part in trying to make effective use of the
application.

For the SMS- model teacher, there was an average of 39 teacher occurrences in the WebAlt
application log per week. Again, this was less than the weighted average for the ‘MXit-based’
model teachers, but comparable to School B which had the lowest average teacher use. The
SMS-model teacher also made frequent use of the application, and played her part in
encouraging learners to use the application.

How Much Did Learners Use the Dr Maths Individual Tutoring Service?
There was varied uptake of the percentage of learners per ‘MXit-based’ model class
registering with Dr Maths. In some classes, such as School B and School A, fewer than one in
four learners registered with Dr Maths. In contrast, almost 60% of the School F Class 10A
learners registered for this service. The weighted average of the percentage of learners per
class registering with Dr Maths was 37%. It is not clear why there was such a variety in the
proportions of learners registering with Dr Maths.

Only certain learners made regular use of Dr Maths after they had registered. In total, 30
learners (13% of the ‘MXit-based model’ learners) used Dr Maths more than ten times over
the project duration. Average learner use of the Dr Maths service was 4.5 times per week, for
these ‘active users’. This is quite high and reflects regular use of the service. Those learners
who registered and used the service seemed to use it often. They also spent a lot of time
online making use of the service, as the weighted average of time online using Dr Maths by
active users, across the six ‘MXit model’ classes, was approximately 24 minutes.

Learners at School A, which is a specialist mathematics, science and technology school,

made least use of the Dr Maths service. 12 As Dr Maths was a tutoring service which provided
access to a mathematics expert, it is perhaps to be expected that learners at this school felt
that they had adequate support from their own teacher and peers and so did not require the
service. Particular learners at School C and School D made extensive use of the Dr Maths
programme, 13 while a small core of learners at School F also frequently used it. 14 For learners
at School C, this is consistent with their enthusiastic use of the WebAlt/MXit progra mme. For
School D, however, this is contrary to the pattern of WebAlt/MXit use. In School D, where
technical problems were reported, learner use of WebAlt/MXit was comparatively low, yet
some made frequent use of Dr Maths.

Although learners were encouraged to use Dr Maths by their teachers, whether or not they
actually did use it was not monitored by the teachers. This was not the case for WebAlt/MXit.

12 School A ‘percentage of learners registering with Dr Maths’ = 21.74%; School A ‘average frequen cy of use of Dr
Maths (interactions per day by ‘active users’ LB) = 1.58; School A ‘Average time online (minutes per interaction of
‘active users’ LB) = 5.
13 Six learners at each of School C and School D made use of Dr Maths more than 51 times in the two month

period, And there were a further seven learn ers at School D and four at School C that made use of it b etween 31
and 50 times.
14 Three learners at School F Class 10 A used Dr Maths more than 51 times and two learn ers at this school used it

between 31 and 50 times.

13
Given this voluntary factor, the result that a core of learners used Dr Maths frequently is a
positive indication of what the service offers. However, it should also be noted that there
were consistent complaints about speed and efficiency of the Dr Maths from both teachers
and learners across the six schools.

No analysis was conducted on teacher use of Dr Maths. As this was a tutoring service to
support learners outside of school hours and on their own initiative, it was not expected that
teachers would use it. Similarly, Dr Maths was not part of the SMS- model and as such no
analysis was conducted on its use with this class.

How Do These Uses of Mobile Technologies in Grade 10 Mathematics

Classes Impact on Teachers’ Work Load?
This question is the most difficult to answer, as expectation and opinions both differ from
respondent to respondent. Teachers and principals had mixed views about how these
technologies impacted on the mathematics teacher’s workload.

In general, ‘MXit-based’ model mathematics teachers expected that they would experience a
reduction in their workload as a result of the project. However, by the endpoint, they had
found that this has not been the case and had revised their expectations. One of the ‘MXit-
based’ model teachers described this in detail in terms of the increased administration and
paperwork required for a project of this nature, her role in monitoring learners’ use of the
applications, and the fact that she was giving extra classes to the project class to ensure that
they remained engaged with it. These additional time commitments for teachers should be
reduced if the technology is deployed as a system-wide service, and not as part of a targeted
proof of concept intervention, such as this one.

The SMS-model teacher expected that the project would reduce her workload and confirmed
at the end that she thought that it had.

‘MXit-based’ model’ principals, in general, had not expected to see reduction in workload for
their mathematics teachers, and only one principal revised this view to indicate that there had
been a reduction in workload by the endpoint.

The impact that one would expect to notice on the teacher’s workload depended on what was
envisaged to be the teacher’s role in the project. For this reason, teachers were asked to
describe typical lessons and to reflect on their role as mathematics teacher during the project.
Data on the issue of teacher role indicates that the project did not seek to shift how
mathematics classroom teaching took place. The project sought primarily to provide a
supplementary resource, for learners to use in their own time, which would facilitate the
administration and marking of homework.

This was confirmed when ‘MXit-based’ model teachers described what they thought were
‘typical’ learner experiences of the Imfundo yami/yethu project. Typical descriptions where
that the mobile telephone would be used in supplementary activities to consolidate and
administer homework. Several teachers described that their classroom teaching would be ‘as
normal’. When they described their typical mathematics lessons during the project, it was
clear that, in general, they had a very conventional or instructivist view of their lessons. In

14
relation to the theoretical framework developed in the pedagogical review, their style seemed
to be more in the injection quadrant of the Cronjé model. His description of teaching style in
the injection quadrant is that it has ‘a high level of direct instruction of relatively simple
content’. 15 For example, the teacher described how, in a typical mathematics lesson, they
would explain the concepts, then give learners exercises to do from the textbook, and finally
set them homework using MXit/WebAlt. During the project, teachers clearly expected to act
as the expert and drive teaching and learning processes. As has been described in the
pedagogical review of the initial project design, this teaching style was not challenged during
the original project conceptualization.

So, the Imfundo yami/yethu project appears to have reduced teachers’ workload in relation to
setting, administering, and marking homework. However this reduction in teacher workload
was offset by the time and effort required to receive training and support, as well as to
administer the project. For the SMS-model teacher, this reduction of workload seemed to
make a difference to her, although the project was shorter and less complex, so her
administrative involvement with the project process was likely to have been reduced.

At the same time, the project seems to have had little impact on mathematics teachers’ role in
the mathematics classroom. The intervention was seen primarily as a means for learners to
consolidate independently work done in class. It was seen as a homework administration
process, and did not directly impact on mathematics lessons. There was some evidence (in
two classes) of the WebAlt application being used as a diagnostic tool to influence the lesson
focus. As was raised in the pedagogic review and is evident in the generally poor
performance of learners in their mathematics tests and June examinations, it seems
appropriate for an intervention of this nature to focus on the potential it has to improve
mathematics teaching practice and shift mathematics teachers’ roles.

How do These Uses of Mobile Technologies in Grade 10 Mathematics

Classes Impact on Learners’ Attitudes to Mathematics?
In general, learner attitude data indicates that learners reacted very positively to the project
and its use of technologies to support mathematics learning.

What Were the Grade 10 MXit-Based Model Learners’ Attitudes to the Project?
On average, learners:
• Very strongly or strongly agreed with the statement that they really enjoyed using their
mobile telephone to help with mathematics;
• Very strongly or strongly agreed that the MXit/WebAlt programme had helped them to
improve their mathematics;
• Strongly agreed or agreed that Dr Maths had helped them to improve their mathematics.

When comparing learner responses at the initial stage of the project with their views near the
end of the project, it was clear that most learners expressed their agreement with the
statements less strongly by the end of the project. The biggest change in learner attitude
recorded was towards Dr Maths, which also received the biggest variation in responses.
Where learners were optimistic about Dr Maths helping them at the start of the project, this
optimism was tempered by the endpoint, with several learners changing their opinions on this

15 See Chapter Three, p68.

15
quite considerably. This suggests that learners were at first very excited about using
technologies, but, as might be expected, this novelty wore off over time. As noted above, a
core of enthusiastic Dr Maths users did emerge from the project, but dissatisfaction was
expressed by several learners and teachers.

Learners reported that the project had inspired them to want to study something that requires
mathematics at university, and that it had helped them to understand their mathematics better.
They also reported that, after the project, they now felt that they could do, or at least try,
harder mathematics problems. They anticipated that they may not get better grades for
mathematics after the project, but that they now wanted to work harder in mathematics. These
are indications of the positive way in which learners viewed their experience of this project
and reflected on its impact on their attitude towards mathematics. These are only learners’
reported views, so actual shifts in behaviour resulting from these are not known. However,
learners’ reported opinions of the project are important indicators of their experience of it.
They thought that the project had influenced them in several ways regarding their future
study aspirations, their understanding of mathematics, their motivation to try hard problems,
and their desire to work hard.

What Were the Grade 9 SMS-Model Learners’ Attitudes to the Project?

On average, learners in the SMS model class agreed or strongly agreed with the statement
that they had enjoyed using their mobile telephone to help with mathematics. This was a less
enthusiastic response than their ‘MXit-based’ model counterparts. Like the ‘MXit-based’
Model learners, grade 9 SMS-model learners also reported that the project had helped their
mathematics and they felt able to try harder mathematics problems. They also said that the
wanted to work harder in mathematics (even after project completion). However, they
expressed this less strongly than the ‘MXit-based’ Model learners. Unlike the MXit-based
Model learners, the SMS- model learners did not report that they project had inspired them to
study something that required mathematics at university.

The average response for SMS-model learners indicated less strong agreement with all
statements of project impact, when compared to the weighted average response of the ‘MXit-
based model’ learners across the six schools. As noted above, these differences may simply
be a result of the one group constituting grade 9 learners who had not yet made their own
decision to study mathematics. However, given that the greatest difference in responses was
for statements that refer directly to project impact, it seems fair to conclude that the grade 9
SMS-model learners felt that the project had had less of an impact in terms of motivating
them to want to:
• Study something that requires mathematics at university;
• Work hard and do well in grade 10, 11 and 12 mathematics at school;
• Do well in mathematics so that they can get into the degree they want to study at
university; and
• Enjoy mathematics more than they did before the project.

Given the limited scale of the SMS-model pilot and that it was conducted with a different
grade but in the same school as the ‘MXit-based’ model, it does not seem fair to ascribe this
difference in learner attitude to the SMS-model itself. This possibility should not be
discounted, however, as learners’ lack of enthusiasm may have been a result of the lesser
offering. To establish this for certain, however, it would be necessary to undertake a more
thorough pilot study of the SMS- model. However, we think that any further investment in

16
alternative models should focus on the teacher use of the applications, rather than attempting
to replicate the MXit-based model.

What Were the Grade 10 ‘MXit-Based’ Model Learners’ Attitudes to Mathematics?

Questions asked about learners’ attitude to mathematics in general gave useful insights into
the average learner’s beliefs about and attitudes towards learning of mathematics in relation
to: teacher role; learners’ views on their competence in mathematics; relevance of
mathematics; and style of learning and teaching. For these statements about attitude to
mathematics in general, the instrument used had been trialled with other learner populations
in Europe and found to be reliable. This means that, when administered at different times,
learners still gave the same responses. The project intervention was only over 15 weeks, so
significant shifts in learner attitude to mathematics were considered unlikely. As such, it was
expected that most learners’ responses to statements at the initial and endpoint of the Imfundo
yami/yethu project would remain the same. However, analysis of the learners’ responses to
statements for each of the four factors reveals much about the teachers’ pedagogical styles
and general attitudes and belief about mathematics across the participating classes.

By the end of the project, the average ‘Mxit-based’ model learner was less satisfied than
initially with how their mathematics teacher engages with them. Their perception of their
teacher’s desire for them to understand, to make their lessons interesting, and to listen to
them had declined. The average learner agreed more that their mathematics teacher was too
absorbed in mathematics to notice them and that their teachers did not really care about how
they felt in class. It seems that, on average, their relationships with their teacher had declined
over the course of the project. It may be that learners were now more familiar with the
research process, and felt more able to reflect critically on their teachers. Given that these
shifts were slight, this is more likely than there having been a actual deterioration in their
relationship with their teachers. This also may reflect that, by the end of the project, learners
generally had a better sense of what was expected for them in grade 10 mathematics, and
what to expect from their teachers.

It does seem remarkable that the average ‘MXit-based’ model learner reported that their
mathematics teachers were less caring, listening less, and noticing them less by the end of
project. Perhaps, teacher enthusiasm for the project had waned by the end o f the project, and
learners noticed this change. It may also be possible the project exposed the deficiencies of
their teachers to them. Their teachers were no longer the only source of information and
explanation, and as a result they now viewed them more critically.

There were several positive indications of project success in relation to shifts in learner
attitudes towards mathematics learning.

First, by the end of the project, the ‘MXit-based’ model learners seemed to be less concerned
about competing with other students and showing their teachers their competence in
mathematics. This may be interpreted as positive if the learners are then intrinsically
motivated to perform well, and are not requiring external rewards for this. It is however
unclear, whether this was the case for these learners.

Second, the average learner felt more strongly that they can tackle problems that take a longer
time to complete and that discussing different solutions to a mathematics problem is
constructive. This may reflect that learners are motivated to spend longer working on
problems that may not be familiar and take longer to complete than routine exercises which

17
they recognize. They agreed that discussing different solutions to a problem was valuable,
although this was not a project focus, and learner discussion of alternative solutions was not
evident in lesson observations or descriptions of typical lessons for the project.

Third, learners seemed to be more aware that they did not understand everything that they
have done this year in mathematics. This can be interpreted as a positive outcome as learners’
very high expectations of their results in mathematics (which contrasted strongly with their
actual performance) were clearly not realistic. So, reporting that they were subsequently more
aware of what they do not know is probably a positive development. However, extracting
value from this positive development requires that their teachers are then in a position to
guide them through their areas of difficulty, so that they come to understand sections of
mathematics that are of concern. One might also interpret this shift as a normal function of
time, and not a result of the project. They were further into the grade 10 mathematics
programme, and starting to realize that they do not follow everything that has been covered or
is expected of them. Again, whether this results in them finding ways to address their areas of
concern is another matter.

There were, however, several negative shifts in attitude. For example, by the end of the
project, on average, learners felt less strongly that mathematics is worthwhile and necessary
as a subject, and were less certain that time used to understand why a solution works is
worthwhile. There was a greater sense that mathematics was difficult and understood only by
intelligent students. That mathematics is difficult and understood only by intelligent students
is certainly not the conclusion that one would want learners to draw from their state of ‘not
knowing’. One would want them to feel that mathematics is something that can be done by
all students and is not reserved for certain learners only.

Possibly the biggest area of concern is that the most notable shifts in learner attitude were
about valuing memorization more and focusing on the correct answer rather than on
understanding. Memorizing procedures and emphasizing correct answers over conceptual
understanding of processes, is not an attitude towards mathematics education which should
be encouraged. It is unlikely to bring long term success in mathematics, and is not valued in
the national curriculum statements. Content presented in grade 10 to 12 mathematics covers a
wide range of topics, in considerable depth. Understanding and making conceptual
connections between sections is key to learners’ success in mathematics. If learners are
memorizing solution strategies in grade 10, then by grade 12 they are likely to run into
significant challenges. Recognizing problem types and then applying a set of learnt rules may
work when there is a small volume of content, but is not viable when the entire FET
mathematics curriculum is the scope. In fact, considering these learners’ poor results in the
standardized assessment, this strategy is likely not viable even for the small amount for
content covered during the project (which focused predominantly on manipulation of
algebraic expressions). This is not to suggest that memorization is being actively promoted by
the teachers or through the project. However, the project designers may not have been aware
that this seems to be a pervasive approach to teaching and learning mathematics in South
Africa, and thus did not explicitly focus on this as something which should be shifted by the
intervention.

As is argued in the pedagogical review of the project design, the approach adopted in the
project did not focus sufficiently on trying to change dominant styles of mathematics learning
and teaching. By encouraging practice of routine exercises through administering homework,
the project may have inadvertently reinforced these approaches.

18
What Were the Grade 9 SMS-Model Learners’ Attitudes to Mathematics?
Like the ‘MXit-based’ Model learners, the SMS-model learners most strongly agreed that
their mathematics teacher always showed them, step by step, how to solve a mathematical
problem before giving them exercises. They also mostly strongly agreed that their
mathematics teacher appreciates it when they try hard, even if their results are not so good.
This provides insight into the dominant learning and teaching style, which seems to be
conventional or instructivist (located in the ‘injection’ quadrant, as defined in pedagogical
review).

In contrast to the MXit model learners, SMS-model learners agreed less strongly that their
mathematics teacher tried to make mathematics lessons interesting. They seemed generally to
have a less positive attitude about their mathematics teacher’s role than their grade 10
counterparts. Again, this is to be expected, given that, for grade 9 learners, mathematics is
compulsory, whereas by grade 10 learners have chosen to take the subject. However, that the
SMS-model class did not have as good a rapport with their mathematics teacher as the MXit-
based Model learners did may also be an influencing factor on their views.

Overall, on average, SMS-model learners were less confident in their mathematical

competence than the ‘MXit-based’ Model learners. As a group, they were less certain that
they liked what they were learning in mathematics class.

Like the ‘MXit-based’ Model learners, the SMS-model learners most strongly agreed that
everyone can learn mathematics and that they thought mathematics was an important subject.
However, overall, SMS- model learners were less convinced of the relevance and usefulness
of mathematics, than the ‘MXit-based’ Model learners. As may be expected from a grade 9
class, where learners have not yet made a choice to continue to study mathematics, learners in
the School F grade 9 class seemed to think that mathematics has less relevance to them
(particularly in terms of its use in people’s daily life, and their future ability to earn a living).

SMS-model learners more commonly thought that certain people can do mathematics without
having to think hard. In relation to themselves, they thought that they could do mathematics if
they tried hard enough (although again, on average, as a group, they seemed less convinced
of this than the average grade 10 learner).

As the SMS- model intervention was short, there was only a single measure of learner attitude
and no notable shifts in attitude were reported by the SMS-model learners.

In general, the SMS- model class were less supportive of their teacher’s role, seemed less
confident in their mathematical ability, and thought that mathematics is less relevant to them
than the ‘MXit-based’ Model learners. They seemed to have views about the inaccessible rote
statements more in common with the ‘MXit-based’ Model learners. However, as mentioned
above, as the scale of the SMS-pilot pilot was small, constrained by being in the same school
as the MXit model, and was implemented at a different grade level, it is unwise to read too
much into these consistent differences in attitude. We can observe that the se learners were
generally less positive about mathematics in most elements relating to mathematics, but do
not know if this is because they are grade 9 learners or due to the specific experience in their
school, with their particular teacher. Further investigation would be required to establish if
these less positive attitudes were typical of grade 9 learners when compared to grade 10
learners of mathematics.

19
How Do These Uses of Mobile Technologies in Grade 10 Mathematics
Classes Impact on Learners’ Competence in Mathematics?
From the outset, it was cautiously hoped that the Imfundo yami/yethu project would impact
on learners’ competence in mathematics. However, as described in Chapter Two of the full
evaluation report, several project partners expressed concern that the timeframes for the
project were too short to expect significant shifts in this regard.

Findings reveal that there was not improvement in the learner’s mathematical competence as
measured in standardized tests and examinations. In fact, the average mathematics
performance data for after the project was lower than the ‘best available baseline’ data for all
six classes. This was the case irrespective of which measure was used for the post project
assessment, as both the average performance in the standardized test, and the average
performance in the June examination were lower than the ‘best available baseline’ data.

On average, the learner group performed poorly in the standardized assessment. The
weighted average across the six ‘MXit-based model’ classes was 29% in the standardized
test. The School A class average of 58% was substantially above this, and the School C class
average, at 32%, was also above the weighted average across the six classes. The average for
all other classes was below the weighted average, with Schools E and F having the lowest
average of 21% for the standardized test.

Learners, on average, did much better in the multiple-choice section than in Section B, where
they were required to show their working. As when considering the total percentages, both
School A and School C classes scored above the weighted average for the multiple-choice
section across the six ‘MXit-based model’ classes and all other classes had average
percentages of below this.

The investigative question was of particular interest, as it required learners to apply their
knowledge of algebraic expressions to a specific situation. This question reflects the type of
more complex question being requested by teachers, while the investigative element was
something that the WCED curriculum advisor thought should be included in future. As such,
responses to this question give further insight into teaching and learning style. About a third
of learners in each class attempted this question, but were not able to gain one mark for it.
This shows that they were unable to follow the instructions, generate a few numeric
examples, and observe a pattern. Approximately one in five learners in each class was able to
do this, and so gained one mark for the question. None of the learners was able to use their
knowledge of algebra to generate an expression and explain their findings. This is a
demonstration of weak conceptual understanding of purpose of use of algebra in solving
problems. The main focus of the intervention was not on this type of problem, but was
restricted to abstract manipulation of algebraic expressions and equations. This focus is
reflected in the learners’ performance in the investigate question.

We do not think that the generally poor performance in the standardized assessment and June
examinations (where these have been available) can be attributed to the Imfundo yam/yethi
project. The poor performance of learners in the mathematics assessments is consistent with,
and reflection of, the serious challenges facing mathematics education in South Africa. As
was cautioned at the outset, a 15-week intervention is unlikely to address these problems. The
poor performance highlights, however, that, in more than half of the schools, most grade 10
learners are simply not attaining the level of performance required for them to continue in

20
mathematics. This is the reality of the South African situation, and as such reiterates the
importance of ensuring that ongoing teacher professional development and deep pedagogical
reflection on how to solve this problem is a key driver of any mathematics intervention.
Providing learners exercises to complete for homework seems to have motivated them, but
has not addressed the underlying serious pedagogical issues.

The generally poor performance of learners in the assessments after the project stood in stark
contrast to views expressed by teachers and principals. Both participant groups had agreed or
strongly agreed that the project significant improved learners’ mathematics results. In School
A, there was the strong claim that the Imfundo yami/yethu class had performed better in
relevant questions in the June examination than the three other grade 10 mathematics classes.
As a result, this claim was analysed in detail (as presented in detail in Chapter Four of the
main report and summarized below).

Did the Imfundo yami/yethi class at School A perform better in that the other grade 10
classes in School A as a result of their involvement in the project?
The claim by the School A principal and teachers that the Imfundo yami/ yethu class
performed better than other grade 10 mathematics classes at School A on test questions
related to the project content in the June examinations was true. Class 10B (the Imfundo
yami/yethu class) had a higher average percentage for relevant questions in the June
examinations than the other three grade 10 mathematics classes in the school. In addition, the
standard deviation for the average results for these questions was smaller for 10B than for the
other classes, showing that the 10B results were in a smaller range and more t ightly clustered
around this higher mean result.

This does not, however confirm that this difference in examination performance is
necessarily, or only, a result of the Imfundo yami/yethu project intervention. That Class 10B
had a different teacher is a significant additional variable, and their better performance in the
June examination may simply be a consequence of that difference. As a result, we
investigated comparative class performance in examination questions not related to the
project content. It was found that, while the Imfundo yami/ yethu class performed better than
the other grade 10 mathematics classes at School A on test questions that were relevant to the
project content, it also performed better than the other classes on test questions in the June
examinations that were not relevant to the project content. This may indicate that the learners
benefitted from the skills and approaches to mathematics developed through the project, and
this then impacted on other areas of mathematics. Nevertheless that differences in results
cannot solely be attributed to the project content. This is not to imply that the Imfundo
yami/yethu project content and approach did not have a positive impact on the learners’
mathematics results or on their teacher (both of whom claim that it did). However, the above
results must be seen in relation to two interrelated variables which impacted on the results:
their teacher and their involvement in the project.

Further, if the Imfundo yami/yethu intervention was primarily responsible for the better
results for 10B, one would expect that this class would have fared better on the Imfundo
yami/yethu relevant questions than on those that were not relevant. However, 10B had better
average results on questions that were not relevant to the project. As such, the better results
for the 10B mathematics class cannot be attributed to the Imfundo yami/yethi project alone.

21
Why Did Some Schools Seem to Make a Success of the Project While Othe rs Did Not?
Given the generally poor average performance by learners in the standardized assessment, we
felt it important to reflect on differences between the schools, as some schools had far greater
success with their learners than others did. Why was this so? We considered ‘making a
success of the project’ to be a reflection of the extent to which:
• Learners and teachers responded positively the project;
• Learners expressed a positive attitude towards mathematics and made positive shifts in
their attitude after the project; and
• Learners performed relatively well in mathematics in the standardized test and there was
a smaller decline in their performance as measured before and after the project, in
comparison to other schools.

To answer this we contrasted two extreme cases, comparing a school that seemed to have
made a success of the project intervention with one where there was less success.

These two schools seemed comparable in terms of their contexts and prior performance, but
responded to the opportunities provided by the project in very differe nt ways. These
differences were evident in relation to learner attitudes, the teachers’ role and learner
performance in the standardised assessment.

In undertaking this comparison, we aimed to reflect on how conditions in the schools differed
and speculate why the project was well received in one context, but not the other. We also
provided a critical review of both schools to yield lessons which may inform design of future
projects. This is presented in Chapter Four of the main report.

This comparison revealed that in both schools, the ‘conventional’ or instructivist approach’ to
teaching mathematics was evident. The school case where the project was less of a success
seemed to present an extreme case of the ‘injection’ quadrant of Cronje’s four quadrants in
the pedagogical review. The school case where the project was more of success showed
elements of constructivist approaches, and would lie more in the ‘integration’ and
‘construction’ quadrants (particularly when considering the lessons observations and learner
attitudes towards mathematics learning). Both teachers and schools could benefit from
reflecting on their pedagogical styles and learner attitudes to mathematics learning.

It was also clear that technical problems were experienced in both contrasting schools.
Technical problems cannot be allowed to hamper project implementation and addressing
these will be essential to the success of such projects in any future.

Comparison of the two schools revealed how the culture of teaching and learning in the
school, the role, pedagogical style, and attitude of the mathematics teacher, and learner
beliefs and attitudes about learning mathematics are all interrelated factors that impact on the
success of a project intervention of this nature. These factors also obviously impact on each
other and on the overall learner attitude to mathematics and learner performance in
mathematics assessments. The comparison also shows that shifting mathematics performance
in South African schools is a complex process which needs to take all of these into account,
and can only be achieved over a sustained period.

22
Recommendations
The evaluation findings and analysis reveal the depth and complexity of the challenges facing
mathematics education in South Africa. They also clearly show that the culture of teaching
and learning in the school, the role, pedagogical style, and attitude of the mathematics
teacher, and learner beliefs and attitudes about learning mathematics are all interrelated
factors that impact on the success of a project intervention of this nature.

As a result of this, we think that future projects of this nature have two options regarding
working to improve mathematics performance in schools, and choosing which schools to
focus on:
1) Select very carefully which schools, teachers, and mathematics classes participate in the
project for it to be successful; or
2) Expect to work with certain schools and teachers in an intensive intervention over a long
period to improve learner performance in mathematics by supporting shifts in:
a) The culture of teaching and learning the school;
b) The role, pedagogical style, and attitude of the mathematics teacher;
c) The learner beliefs and attitudes about learning mathematics.

If the former approach is adopted, schools which already have a culture of teaching and
learning that would support improved performance in, and attitude towards, mathematics
need to be identified, or invited to nominate themselves, to participate. In this scenario,
school involvement can likely not be imposed, but should be invited, with teachers and
principals demonstrating a personal commitment and motivation to be involved. Criteria for
selection may include that certain profiles of schools are given priority access to the
intervention to ensure sufficient focus on previously disadvantaged communities. The
‘project success’ case demonstrates that there are former Department of Education Training
schools where they have the necessary teaching and learning culture and capacity to make a
success of the project offerings.

If the latter approach is adopted, deliberate emphasis on shifting mathematical pedagogical

practice and learner attitudes towards mathematics learning is required. This would entail
more detailed reflection on the pedagogical approaches which are intended. It would also
demand that detailed attention be paid to shifts required in learner attitudes and beliefs
towards mathematics. Many of the desirable and positive attitudes which should be the
intended focus of such shifts are articulated in the findings of the main report, and restated in
the recommendations below. Again we stress that such shifts can only take place over a long
period of time (two to five years). We estimate that such a process could have an intensive
initial focus, of an estimated one to two years. The project may reduce the investment and
time spent with the teacher over time, but should continue to support and monitoring the
teacher for about three years after the intensive period.

This scenario is one where teachers participate in deep pedagogical reflection and need
support and time to plan new approaches to mathematics lessons, facilitate them and reflect
on them. They need to be supported to shift their own attitudes and beliefs about
mathematics, as well as that of their learners. It is difficult to see how such an approach could
be attained in the absence of a mathematics teacher education partner being a core part of the
project team. Such professional development programmes already exist in the form of
Advanced Certificates in Education (that focus on mathematics education or the integration
of ICTs into education) as well as in Masters in Education programmes. We think that this

23
type of long term, and intensive intervention, which uses technology as catalyst to support
shifts in teaching and learning, should be done in partnership with a mathematics teacher
education agency.

We think that the ‘MXit-based’ model was sufficiently successful in the Imfundo yami/yethu
project to develop it further. However there are many ways in which this model could be
improved, which we detail in the set of recommendations presented in this section.

The SMS-model seems to have been a technical success, but was less well received by the
teacher and learners, than the ‘MXit based’ model. Within the context of only having one
school case, the shorter project duration, and the influence of the MXit-based model being
conducted at the same school, we cannot be sure that this not just a result of the particular
school and context. However the SMS-model also does not appear to have been a core focus
for the project team. Therefore, with our judgements on this, based primarily on the
pedagogical review of the model, as well as the limited findings from the pilot study, we do
not think it would be worthwhile investing further in this model as it stands.

Nevertheless, we do think it important to investigate and, as necessary, develop a model that

does not require learners to have constant online access. This was a major weakness in the
‘MXit-based’ model. Requiring Internet access and stable and reliable network coverage for
all learners when at school and when at home potentially limits reach of the model in future.

We therefore recommend that the focus be on developing a n alternative model that does not
aim to mirror the ‘MXit-based’ model. Some possibilities of such a model include, designing
a model that seeks to:
• Explore what tools can be provided to learners that they can use on their mobile handsets
without requiring online access. This may include, for example, supporting learners to use
the calculator functions in their handsets or providing graphic calculator or dynamic
geometry applications, or ‘mathlets’ as described in the pedagogical review.
• Design a model which assumes that the teacher ha s access to the Internet via a laptop
and/or mobile telephone, but that her learners do not. This may lead to a stronger model,
which draws on the ‘MXit-based’ model but lowers technology and network coverage
requirements.
As was seen in some of the classes in the ‘MXit-based’ model, the teacher may use the
application, for example, to source questions which are then written on the chalk board, or
use her laptop and data-projector to display explanations, or problem sets to the class. This
would potentially remove the advantage of having automated marking, although there could
be ways in which learners might be expected to input their homework results during class
time if homework administration and automated marking remain a priority. Such a model
should have a strong mathematical pedagogy focus and seek to use mobile connectivity to
enable teachers to reflect on their practice and be part of a mathematics education community
of practice.

In relation to either model – the improved ‘MXit-based’ model, or the proposed ‘learner
handset and teacher connectivity’ model – we think that there are a range of lessons that have
emerged from the proof of concept experience of the Imfundo yami/yethu project. We group
each by key focus area:

1) Project planning and implementation:

24
 a) Allow sufficient time prior to the project implementation to ensure that the right
partners are able to work collaboratively under reasonable timeframes and within
clear contractual frameworks.
b) Do not underestimate the serious challenges facing mathematics education in South
Africa. Ongoing teacher professional development and deep pedagogical reflection on
how to overcome these challenges and improve learner performance in mathematics
must be a key driver of any mathematics interventions.
c) Decide whether to work with schools and individual teachers which already have a
culture of teaching and learning mathematics that would support improved
performance in, and attitude towards, mathematics; or whether to work with certain
schools and their teachers in an intensive intervention over a long period to improve
learner performance in mathematics.

2) Project conceptualisation and purpose in relation to teacher role and workload:

a) As indicated in the pedagogical review, deeper reflection on precisely what
pedagogical shifts in mathematics teaching and learning are intended by the project
intervention is required.
b) Ensure that projects such as this actively seek to improve mathematics teaching
practice and to shift mathematics teachers’ roles away from the ‘injection quadrant’ or
conventional teaching style, as defined in the pedagogical review, towards a more
constructivist, and connectionist approach.
c) Actively seek out and provide tools that reduce the workload of teachers. These
should be focused on reducing routine administration and preparation time.
Automated marking of learner homework is a good example of this type of tool which
was effectively used in the Imfundo yami/yethu project. Another is providing a source
of well considered learning resources and mathematics problems. Such reductions are
essential to off-set the additional time and effort required by participating in such
projects. This also creates more time for reflection and professional growth for
teachers.

3) Project conceptualisation and purpose in relation to learner attitudes to mathematics:

a) Be aware that, although technologies, can have a motivating effect on learners, the
novelty factor of introducing new technologies should not be underestimated. Initial
excitement will be tempered over the duration of the intervention, and so actively seek
ways to integrate the possibility of mobile use into daily mathematics learning, rather
than being only a supplementary activity.
b) The improvement in learner’s attitudes to mathematics and excitement created by
such interventions should remain a project focus. When considering the findings on
what shifts in learners attitude emerged, it is necessary to reflect in a more detailed
manner on what specific beliefs and attitudes about learning mathematics are
considered important.
c) In future projects, strategies to break down the perceived value of memorization in
learning mathematics and the notion that the correct answer is more important than
the strategy used to obtain it should be a driving part of the teacher support and the
pedagogical approach of the intervention.
d) In relation to learner attitudes to mathematics learning, future projects should build on
learners’ apparent willingness to:
i) Reflect on what they do not know;

25
 ii) Discuss alternative solution strategies to mathematics problems (with teachers and
peers), noting that this is distinct from the teacher instructing different solution
strategies to the learners; and
iii) Tackle problems that take a longer time to complete.
e) Also with regard to specific attitudes towards mathematics, it is important to:
i) Focus attention on getting learners to be realistic about their current performance
and aware of their personal goals for attainment in mathematics;
ii) Focus attention (of both the teacher and learner) on what learners should do when
there are sections of work that are not understood ; and
iii) Place particular emphasis on mathematics as something for all students (and not
reserved for only intelligent students).

4) Project conceptualisation and purpose in relation to learner performance in

mathematics:
a) Ensure that improving learner performance in mathematics remains central to any
project intervention of this nature.
b) Even though improvement in performance takes time, ensure that standardised
assessments are included in the project. This is an essential way of monitoring the
project and supporting teachers to improve learner performance. That improving
mathematics performance takes time, and is difficult, should not be a reason to avoid
measuring it or aiming for it.
c) From the experience of the Imfundo yami/yethu project, attention should be paid to:
i) Focusing learners’ attention on solving mathematics problems that require them to
show their working, and not just on multiple-choice type questions.
ii) Providing far more exposure to investigative problems which require the use of
algebra within context, and opportunities to understand the purpose of being able
to manipulate algebraic expressions, and solve equations.

5) Selection of schools and teachers to participate in the project:

a) Ensure that selection processes and criteria for participation are transparent and
communicated clearly to schools.
b) Ensure that teachers have a degree of choice about whether or not they partake in a
project of this nature, rather than having it imposed on them.
c) Involve schools and teachers in the selection process, and consider making their
selection competitive. Having to market the intervention to attract teachers to
participate in it, will ensure that the specific benefits to teachers will have to be well
thought through and communicated. For example, project involvement may be driven
through linking it to an accredited mathematics education professional development
programme.
d) Involve schools that have an enthusiastic teacher who is motivated to be part of the
intervention, and interested in reflecting on teaching and learning styles.
e) A criterion for school selection should be that network coverage is sufficient and
stable enough for learners to have connectivity at least on a daily basis while at school
and at home for the MXit-based model to be viable. Where this is not the case the
proposed ‘learner handset teacher connectivity’ model may be an alternative option.

6) Technology access:
a) Ensure that all learners participating in the project have access to t he technology
required to take part in it. Although this was the intention in this project, learner and

26
 teacher reports suggest that a minority of learners in some classes did not have
suitable handsets.
b) Spend time orienting learners to use of the technolo gy and attending to any technical
problems that arise, before the official start of the project.

7) ‘MXit-based’ model technical platform:

a) Invest more time up front in checking the technology platform and ensuring that it is
robust before using it in schools.
b) It seems to be a significant limitation in the ‘MXit-based’ model that learners have to
have an online connection to interact with the system. If feasible, invest in a way in
which learners can receive batches of questions, save their responses, and submit
them as a set.
c) Continue to monitor teacher and learner usage of the application and produce monthly
or weekly reports of this.

8) Content applications in relation to WebAlt/MXit:

a) Increase the number of mathematics topics covered.
b) Include more challenging questions, as well as more examples and explanations.
c) Consider granting limited administrative rights to teachers so that they can author
their own content and add in more exercises.
d) Consider having a portal for teachers to author content under a Creative Commons
License (this may be distinct from the content provided by WebAlt, where quality and
accuracy is assured).
e) Check all the WebAlt content for accuracy as there were some comments from a few
teachers that there were some errors in the content. One such error (the incorrect
answer was selected as correct in one of the quizzes) was evident in one of the lesson
observations.
f) Consider developing multilingual content, particular in Afrikaans as it is a medium of
instruction for mathematics in some schools (although this may be addressed by
involving teachers in the content creation process through c or d above).
g) Aim to further improve the WebAlt/MXit application in term of ease and speed of
use.

9) Content applications in relation to Dr Maths:

a) Investigate why there were consistent reports of no or very slow responses from Dr
Maths.
b) Aim to improve the Dr Maths application in term of ease and speed of use.
c) No qualitative data was obtained from learners who were ‘active users’ of the Dr
Maths service. With voluntary and supplementary services of this kind, it would be
useful to plan to include soliciting feedback on the service from those learners who
used it regularly. This was not done in this evaluation as the clear distinction between
‘active learners’ and other learners was not evident at the design phase. We conducted
focus groups with learners and received their feedback in questionnaires, on Dr
Maths, but we did not actively target learners who were making regular use of Dr
Maths to ask them more detailed questions about it.

10) Content applications in general:

a) Allow teachers and learners access to a wider range of content options (as was
planned from the outset).

27
 b) Aim to include not just closed content applications where content delivery and
explanation is the focus. Options that include more open-ended and investigative
problems for learners to work, publish, and share their solutions would be valuable.
This should deliberately incorporate problems that are more challenging and varied
than routine exercises
c) Include open-ended mathematical tools such as spreadsheets, graphing calculators,
scientific calculator application, and dynamic geometry. There is a wide range of such
specialist software or applications available that are specific to mathematics
education.

11) Teacher training and support to teachers:

a) Include a specialist mathematics education professional development partner as part
of the project team.
b) As suggested in the pedagogical review, and by some teachers, include a greater focus
on pedagogy and teaching and learning of mathematics in the initial teacher training.
c) Anticipate that teachers will need support immediately after initial training, when they
are back in their school environments. This support might include technical support in
ensuring that the technology offering works, as well as pedagogical reflection on
teaching practice and style.
d) Ensure that strategies are in place to intervene quickly to investigate lack of use at
particular schools. In this way technical support should be proactively provided, and
technical problems efficiently solved when low levels of interaction are evident.
Where the low levels of interaction are not technically related, the teacher support
team should establish the reasons for lack of use and devise strategies to improve use.
e) Anticipate that there will be varied uptake in the use of mobile technology by learners
and support teachers to encourage inactive learners to take part.
f) Where automated marking is being used as a mechanism to reduce teachers workload,
ensure that teachers are adequately supported to be able to use this information as a
diagnostic tool, informing their classroom practice.
g) Ensure that opportunities to explore ways of identifying, and learning from, common
errors and misconceptions arising from learners work are actively built into the
pedagogical support in which teachers engage.
h) It may be necessary to include professional development for teachers which focuses
on building and maintaining a trusting and caring relationship with their learners.

Conclusion
The project team and steering committee are to be congratulated on their achievements
initiating and implementing the Imfundo yami/yethi project.

The overwhelming response from learners, teachers, and schools has been positive and most
of them have enthusiastically embraced the opportunities it provided. Teacher workload in
relation to administration and marking of homework has been reduced. Project participants
have said that it has improved learner attitude and performance.

Detailed analysis demonstrates that, on close inspection, there have been mixed successes in
relation to teachers’ workload and learner attitude, as might have been expected from the
outset. It clearly reveals that the project has not yet improved learner performance in
mathematics.

28
 However we have shown that improving learner performance in mathematics was never
likely to be possible from the outset. Striving to improve performance, and examining learner
performance as part of the project, has revealed the complexities this entails and the serious
problems facing mathematics education in the country. It is clear that some schools have
made more of a success of the project than others have. Our comparison of two contrasting
schools within the project gives some insight into this complexity, while also revealing
desirable shifts in teacher role and learner attitude and beliefs. We trust that the
recommendations provided will provide a basis for detailed reflection and influence planning
for projects of this nature.

29

View publication stats