Vietnam National University, Hanoi

University of Languages and International Studies

Faculty of post- graduate studies
Course Design and Materials Development
for English Language Learning

MATERIALS
GROUP 7:
EVALUATION
Nguyễn Minh Yến
Nguyễn Phương Anh
Ngô Vũ Hoàng Minh
Phạm Thị Thu Thảo
Lê Hải Phong
 TABLE OF CONTENTS

What and how to How to make use

examine materials
01 of the findings
from the analysis
Teachers’ need
02
03
Evaluation
process
04 Macro/ Micro-
05 evaluations of Task-
based teaching
 01
What and how to examine
materials
Presenter: Nguyễn Minh Yến
 1.1. What should be examined in materials?
THE PUBLICATION
● Place of the learner’s materials in any wider set of materials
● (e.g: whether answer keys are only available in the teacher’s materials,
how the student’s material relates to any audio or video recordings)
● Published form of the learner’s materials
● Subdivision of the learner’s materials into sections
● Subdivision of sections into sub-sections
● Continuity: how a sense of continuity or coherence is maintained
● Route: whether the order in which the material can be used is
predetermined
● Access
 1.1. What should be examined in materials?
THE DESIGN
● Aims
(e.g: the development of ‘general English’, ESP, or specific skills)
● Principles of selection and principles of sequencing
(e.g: a particular syllabus type and use of corpora)
● Subject matter and focus of subject matter
(e.g: cross-curricular, content, storylines, topics,...)
● Types of teaching/learning activities
+ what they require the learner to do
+ manner in which they draw on the learner’s process competence
(knowledge, affects (attitudes and values), abilities, skills)
● Participation
● Learner roles and teacher roles
● Role of the materials as a whole
 1.2. How to examine materials?

LEVEL 1 LEVEL 2 LEVEL 3

“OBJECTIVE DESCRIPTION” “SUBJECTIVE ANALYSIS” “SUBJECTIVE INFERENCE”

What is there in the What is required of users What is implied from the
materials materials
Lorem ipsum dolor sit amet, consectetur adipiscing
elit. Duis sit amet odio vel purus bibendum luctus.
 1.2. How to examine materials?
LEVEL 1 STATEMENT OF DESCRIPTION
● the publication date
WHAT IS ●
THERE the intended audience
IN THE ● the type of materials
MATERIALS (e.g: general/ specific purpose/ supplementary/ main
course,...)
● the amount of classroom time required
● how the materials are to be used
(e.g: for self-study/ in any order,...)
 1.2. How to examine materials?
LEVEL 1 PHYSICAL ASPECTS OF THE MATERIALS
● the published form
WHAT IS
THERE (e.g: durable vs. consumable, worksheets vs. bound book,
IN THE paper print vs. electronic)
MATERIALS ● number of pages
● use of colour
● the total number of components in a complete set
(e.g: student’s book/ workbook/ audio materials,…)
 1.2. How to examine materials?
LEVEL 1 MAIN STEPS IN THE INSTRUCTIONAL SECTIONS
● How the material is divided into sections
WHAT IS
THERE (e.g: units, audioscripts, answer keys, tests,...)
IN THE ● How “units, “modules”, “blocks”...are divided into
MATERIALS subsections
(standard pattern or recurring features)
● The means of access into the materials that are provided
(e.g: indexes, search facilities, detailed contents listing,
hyperlinks, …)
● How the various sections and means of access into the
materials are distributed between teacher and learners.
 1.2. How to examine materials?
LEVEL 2 We should divide the materials into their constituent ‘tasks’,
and then analyse three key aspects of each task:
WHAT IS
REQUIRED
OF USERS ✓ How: a process through which learners and teachers are to
go.
● Turn-take: the role in classroom discourse that the
learners are expected to take.
● Focus: whether the learners are asked to attend to the
meaning of the language, its form or both.
● Mental operation: the mental process required
(e.g: repetition, deducing language rules, or
broader processes such as hypothesising, negotiating,...)
 1.2. How to examine materials?
LEVEL 2 ✓ With whom: classroom participation concerning with
whom (if anyone) the learners are to work.
WHAT IS (e.g: alone, in pairs/groups, or with the whole class)
REQUIRED
OF USERS ✓ About what: content that the learners are to focus on.
● content of the input and of the learner’s expected output:
(e.g: written/ spoken/ individual words/ sentences/
extended discourse,...)
● Where it comes from (e.g: the materials/ the teacher/ the
learners themselves,...)
● its nature (e.g: grammar explanations/ personal
information/ fiction/ general knowledge,...)
 1.2. How to examine materials?
● make statements about the overall aims of the materials and
LEVEL 3 the basis for selecting and sequencing tasks and content.
● Conclude about teachers and learners roles
WHAT IS
IMPLIED + by examining how various sections of the material are allocated
to teachers and learners
+ by analyzing tasks under turn-take and under input and output
source
● produce a general statement about the nature of the demands
placed upon learners to accomplish their learning.
● Conclude about the role of materials as a whole in facilitating
language learning and teaching ( e.g: Do they endeavour to
guide all classroom work or do they simply intend to stimulate
teachers’/learners’ own decision-making?
02
How to make use of the
findings from the analysis
Presenter: Nguyễn Phương Anh
TITLE
 Materials
designers

Teachers’ Researchers
professional in language
development teaching
 Teachers’ needs
from materials
Presenter: Ngô Vũ Hoàng Minh

03
 Sources
• Professional traits (e.g.
• Personal traits (e.g. language proficiency,
age, sex, cultural knowledge of the target
background language & culture length
educational & types of teaching
experience areas & levels
background interests
of expertise quality &
belief) amount of teacher
training received)
 Types of needs
• Self-perceived needs
• Needs perceived by others
• Objectively measured needs
 Influences
• teachers’ perception of administrative needs
• measured learners’ needs
• teachers’ perception of learners’ needs
• teachers’ wants
Theoretical model of course
design
Actual model of course design
 Teachers’ empowerment
• Objective measurements of coursebooks
• Strict & systematic selection procedures
• Methods & feedback routes of users’
evaluation
• Wider perspectives in teacher development
• Acknowledgement of teachers’ non-teaching
expertise and workload
 Teachers’ forums (by publishers)
• Evaluation meetings
• ‘take your pick’ sampling meetings
• Users meet to become producers
Production stage
In usage
In usage
In usage
 Macro and Micro
evaluations of Task-based
teaching

04
Presenter: Phạm Thị Thu Thảo
Macro- evaluations of
Task-based teaching
 Questions

Question 1: To what extent was the program/ project effective in

meeting its goals?

Question 2: In what ways can it be improved?

Beretta & Davies (1985)
- Compare learning outcomes:

Experimental group (Project of Task-based learning)

Vs

Control group (traditional classes: structural-oral situational

method)
Beretta & Davies (1985)
- Use 3 tests:
+ Task-based test (favors experimental group)
+ Structural test (favors control group)
+ Neutral test (contextualized grammar, dictation, listening and
reading comprehension, etc.)
Beretta & Davies (1985)
- Findings
+ Experimental group: excels at task-based and neutral tests.
+ Control group: better at structural test

⇒ (+) Project (task-based teaching): better acquisition of structures for

application without much explicit grammar teaching

⇒ (-) Difficult to plan and conduct the evaluation

⇒ Just examining group-biased tests may give misleading data

Beretta & Davies (1985)
- Findings
+ Experiment group: excels at task-based and neutral tests.
+ Control group: better at structure test

⇒ (+) Project (task-based teaching): better acquisition of structures for

application without much explicit grammar teaching

⇒ (-) Difficult to plan and conduct the evaluation

⇒ Just examining group-biased tests may give misleading data

Macro-evaluation studies
Possible problems:
- Teachers’ shortage of spoken English and communicative
competence.
- Teachers’ lack of understanding in the rationale of the tasks and
materials.
- Concerns of teachers in the course content (lack of grammar)
- Real world relevance
NOTES

- Teachers “ should take care to ensure the curriculum dictates the

use of the textbook rather than allowing the textbook to dictate
the content of the course’ (2007, p125)
Micro - evaluations of Task-
based teaching

05
Key ideas of micro-evaluations

- knowing your students

- the outcomes, processes and experiences of student learning.
- an integral part of teaching.
- involving cycles of reflection, data gathering, decision and
action.
Types of information in micro-evaluations

- Information regarding the learners opinion about the task.

- Information about how the task was performed.

- Information about what learning took place as a result of performing
the task.
Different approaches to evaluating task

- Student-based evaluation
- A response based evaluation
- Learning based evaluation
Elis (1998)
1. describing a task in terms of its objectives.
2. planning the evaluation by deciding on:
- the objectives and purpose
- the scope of the evaluation
- who will conduct the evaluation
- the timing
- the types of information
3. collecting the data for the evaluation
4. analysing the data
5. conclusion and recommendation need to be made
Simons(1997): an unfocused information- gap task.
Simons’ aim: whether the task was successful in eliciting meaningful
communication.

Student A: describe a route marked on the map

Student B: draw in the route on his/ her map

→ Findings:
+ Two maps were not identical
+ They created a number referential difference.
 Five general categories were identified:
1. Telling
2. Questioning
3. Acknowledging
4. Responding
5. Miscellaneous

→ an effective device for learners to use the second language

communicating
Freeman (2007): a dicto_ gloss task ( focused task)
Freeman’s aims: to establish both accountability and improvement
Students: listen to the text three times
1. answer a multiple choice question.
2. note down the key content words.
3. focus and take notes on the use of different linguistic forms.
--> Students were largely successful in achieving the outcome of the
task.
=> Micro- evaluation is a good opportunity for evaluating teaching
techniques and materials.
Material Evaluation from
Publishers’ view

05
Presenter: Lê Hải Phong
How?
- Piloting
- Reviewing
- Focus group
- Questionnaires
- Expert panels
- Cooperation with academics and materials
developers on research projects
- Desk research & competitor analysis
5.1. Piloting
Why?
- Test the material in real-life contexts
- Engage in-house staff
- Promote the products
- Build trust
5.1. Piloting
● Receive materials
01 Piloters ● Comments (diary,
annotations)

02

03
5.1. Piloting
Pilots comments on:
- What went right or wrong
- Ordering of the activities
- Achievement of the objectives
- Students’ questions
- Time appropriateness
- ...
5.1. Piloting
● Receive materials
01 Piloters ● Comments (diary,
annotations)
● Compile feedback
02 Editorial team
● Analyze

03
5.1. Piloting
Editorial team:
- Collate comments into a single report
- Compare and analyze reports in different pilot
centers
- Conduct cross-referencing against other
evaluation forms
- Decide on final reports for authors
5.1. Piloting
● Receive materials
01 Piloters ● Comments (diary,
annotations)
● Compile feedback
02 Editorial team
● Analyze
● Receive the final
03 Authors feedback
● Revise
5.2. Reviewing
Who?
- Experienced teachers

⇒ how material will work in real classrooms

- Academics, experts

⇒ objective perspective
5.2. Reviewing
How? Reviewers:
- Receive materials
- Answer a review sheet
5.2. Reviewing

Review sheet for Unit 1 (Messages Student’s Book, Level 1) (Goodey and Goodey
2005)
5.3. Other forms:
- Focus group
- Questionnaires
- Expert panels
- Cooperation with academics and materials
developers on research projects
- Desk research & competitor analysis
 References
● Tomlinson, B. (Ed.). (2011). Materials development in language
teaching (2nd ed.). Cambridge University Press.
CREDITS: This presentation template was
created by Slidesgo, including icons by
THANKS
FOR
Flaticon, and infographics & images by
Freepik

LISTENING