https://blogs.brighton.ac.

uk/materialsdip/category/materials-evaluation/

Materials Evaluation

Before this session myself and my colleagues on the course got into some interesting
discussions over readings we did. Though part of the pre-task was also to come up
with an evaluation framework we ended up with a rather petty and unfinished
consensus.

Literature offers an array of various approaches, checklists and guidelines.

McDonough and Shaw (2013) suggest two stages of evaluation: external and
internal. The first one briefly looks at what the coursebook claims to offer in terms of
context, level, principles as well as type of audience is it aimed at. This phase is to
identify titles we would like to investigate in more detail. During internal evaluation
author’s claims are put to a test. This is achieved through use of questions about
presentation, grading, appropriacy, suitability and authenticity. Though the evaluation
framework seems rather universal McDonough et al. (2013) also point out that
‘criteria often are very local’. In other words, questions for evaluation are best
identified by the evaluators themselves in order to cater for a specific audience and
context. Tomlison (2012) also stresses the importance of not only the context but
also ones own beliefs when generating criteria for the evaluative process. In addition,
he highlights the need for the framework to be flexible and monitored throughout.
Given the subjective nature of each evaluation it is essential not to focus on its
limitations but on its original purpose, which would always be context-specific, and
therefore context-relevant. As McGrath (2013) reminds us that evaluators ‘use the
lenses of (their) experience and context to evaluate’ coursebooks, which might
suggest that all evaluation criteria are partly shaped by ones experience. Given the
size of the EFL/ELT industry and so many contexts English is now being taught and
learnt it surely seems close to impossible to have ‘one size fits all’ evaluation
framework. An interesting study by Johnson et al. (2008) also highlighted differences
in teacher’s expectations evolving with years of experience. This suggests that the
same book would score differently depending on its evaluator’s expertise. Both
McGrath (2013) and Mishan and Timmins (2015) talk of the evaluation model as a
cyclical one. In other words evaluation is an ongoing process. We evaluate before we
teach, while we teach and again after we finish teaching. This approach will give us
most accurate picture of a coursebook and its effectiveness in achieving aims and
objectives. What’s more McGrath (2013) also stresses the importance of learner

1/6
https://blogs.brighton.ac.uk/materialsdip/category/materials-evaluation/

input in this process. Students’ opinions can shine a different light on teacher’s
perceptions and add valuable point to the process.

In class presentations evaluating same coursebook followed by a talk with its author
Theresa Clementson were very interesting as there were differences between both
groups which confirmed how subjective the framework can be. Theresa Clementson,
the author of English Unlimited admitted she wrote her coursebook to have
something she enjoyed teaching from. She has also reassured us all and given
validity to experience and intuition as teacher’s tool to facilitate learning with
materials available to them. The principles we were all expecting to be backed up by
research and theory turned out to be very similar to ones which we discussed during
our brainstorming session. Her talk has definitely instilled some confidence in me as
a fairly experienced teacher capable of making judgement calls on utilising materials
as and when needed.

***

Having gone back to my post about Principles and revisiting my notes from SLA I
have finally come up with a framework I would like to use to evaluate some of the
published materials I use at my school and also ones I aim to create as part of this
module.

2/6
https://blogs.brighton.ac.uk/materialsdip/category/materials-evaluation/

Materials Evaluation Framework

Below I will explain how each question linked to my principles.

Principle  4: Materials should be current

As majority of my students were born in the noughties, being current is plays a huge
role in establishing the rapport. Being current, to me, is trying to find common
grounds and show respect for their interests.

Principle 1: Materials should be relevant

This criterion links to the process of materials design. Though many coursebooks aim
to cater for widest possible audience it is important for this audience to be
identifiable. Teenage students coming to the UK for summer holiday course would
most likely not appreciate a business coursebook

Principle 8: Materials should be varied

I often find students huff at the instruction ‘Now turn to page ..’. Varying materials
caters not only for various learning styles but enables the teacher to ‘channel’ input
through various media and change interaction patterns.

Principle 1: Materials should be relevant.

This is a slightly different angle on relevance. The second criterion looks at the
learning context and whether it fits learner needs, whereas this point carefully
considers learner profile.

3/6
https://blogs.brighton.ac.uk/materialsdip/category/materials-evaluation/

Principle 3: Materials should be realistic.

Principle 5: Materials should be authentic.

Principle 9: Materials should be visually appealing.

Though I have previously considered realism as a principle, I later realised how

visual attractiveness ties in with this. If the purpose of visuals in published material
could be seen as a tool to get students to voice their opinions, speculate and
brainstorm then it goes without saying that those should be far more than just
realistic. Authenticity, in my opinion, makes the book more appealing and stimulating.
As discussed earlier with regards to not all practice needing to be meaningful I would
say the same about the limited necessity for authenticity. To some extent, authentic
material most definitely allows learners to immerse themselves into L2  without
leaving the classroom and prepares them for this eventuality.

Principle 6: Materials should encourage communication.

This criterion is particularly close to me as a non-native speaker. As a student of

English I rarely got the opportunity to communicate in English. Though I have
developed good all-around knowledge of how English language is formed we hardly
ever got to actually speak it. Materials should strive to encourage this from their
learners. Tomlinson (2011: 15) recognised this principle as stemming from SLA
research as : Materials should provide the learners with opportunities to use the
language to achieve communicative purposes. It reminded me about one of the most
common reason students mention in their needs analysis – travel. English can easily
be seen a must have to pack for a holiday, a gap year, volunteering trip, work
experience and so much more. It is both teacher’s and materials’ role to enable our
students to feel confident in those contexts.

Principle 8: Materials should be varied.

As with the previous question this is to ensure our learners get to explore the
language on many levels. Language is conveyed across multiple medias and it
should therefore also be taught and learnt this way.

4/6
https://blogs.brighton.ac.uk/materialsdip/category/materials-evaluation/

Principle 10: Materials should be flexible to adaptation and supplementation

How rigid is the order or layout of the material? Does it cater for different learner
types and needs allowing the teacher to adapt activities to individual characteristics
of the class? Is the language taught feasible to supplement and therefore provide
learners with required practice?

Principle 2: Materials should be challenging.

I truly believe that though explicit teaching has its place in EFL it can only be
validated if used to encourage learners to notice what they have been exposed to
through input analysis.

I hope the above framework will help me to evaluate and improve on materials I use
and produce in my teaching context. Designing the questions enabled me to revisit
my principles in context of materials design as well as learners I tend to teach.

Determined to use is I had a go at evaluating a coursebook we often use for our

teenage summer holiday students. Given this is a ‘teen’ title let’s see how highly will it
score…

5/6
https://blogs.brighton.ac.uk/materialsdip/category/materials-evaluation/

 References

McDonough, J., et al. (2013) Materials and Methods in ELT: A Teacher’s Guide.
(3ndedn) Chichester:Wiley-Blackwell.

McGrath, I. (2013) Teaching Materials and the Roles of EFL/ESL Teachers: Practice
and Theory.London: Bloomsbury.

Mishan, F. & Timmins, I. (2015) Materials development for TESOL, Edinburgh:

Edinburgh University Press

Johnson, K., et al. (2008) A step forward: investigating expertise in materials

evaluation. ELT Journal 62 (2): pp.157-163.

Tomlinson, B. (2012) Materials development in Language Teaching (2nd Edition,

Cambridge University Press

Tomlinson,B.(2012) Materials development for language learning and teaching.

Language Teaching 45 (02)

6/6