In His Great Name

Farhangian Teacher Education University

Shahid Bahonar Faculty

Course: Material Evaluation (Content Analysis)

Week 10

Winter & Spring 2021 (1399-1400)

(parishayeste@yahoo.com)
Review: An External Evaluation -----> Internal Evaluation Process
(EE-----> IE Process)

Macro-Evaluation --------> Inappropriate/potentially appropriate-------------

(EE)

Exit

Adopt/
Micro-Evaluation ………….> Inappropriate/appropriate---------> select
(IE)

Exit
 Steps in Evaluating a Teaching Material /Task:
Bases on:
Who? What?
Step 1: Choosing a task/text to evaluate When? How?
Why?

Step 2: Describing the task •Checklist

•Questionnaires/surveys
•Interviews
Step 3: Planning the Evaluation •observation

Step 4: Collecting the information for the evaluation

Step 5: Analyzing the information

Step 6: Reaching conclusions and making recommendations

Step 7: Writing the report

 Step 4: Collecting the Information for Evaluation:
Step 5: Reporting the Results & Reaching the Conclusion

Data Analysis through Software:

Means , SD and Graphs
 Step 4: Collecting the Information for Evaluation:

a. Checklist

I. Physical Features/Attributes
• General appearance items
• Layout & design
• Visuals
 Data Collection Instrument:
a. Checklist for EE

Mean Score is required

and reported
‫میانگین‬

E.E.

Means
‫میانگین‬

Items, A1, Items Items Items

A2,A3,A4,A5,A B1,B2,B3,B4,B5,B6,B Items A1,A2,A2,A4,A5,A6,A C1,C2,C3,C4,C5,C6,C7,C8,
6,A7,A8 7,B8,B9,B10,B11,B12 C1,C2,C3,C4 7,A8,A9,A10 C9,C10,C11,C12,C13,C14
Step 4: Collecting the information for the evaluation

Collecting the information for the

evaluation of Prospect 1
 Results : A Sample
Please note that this is just a start.
‘Vision 1’ was first evaluated externally through the External Evaluation Checklist
developed by AbdelWaheb (2013). The instrument has two main factors (i.e., Physical
Features/Attributes and Efficient Outlay of Objectives and Supplementary Materials) each
with three subscales or constituents. The first factor was assessed through ‘General
Appearance’ (8 items), ‘Layout and Design’ (12 items) and ‘Visuals’ (4 items). An example is
‘The textbook has a complete and detailed table of contents’ (item 6, Layout and Design)’.
The second factor was also touched through subscales of ‘Book Objectives’ (10 items),
‘Teaching Aids’ (8 items) and ‘Teaching Materials’ (13 items). An example is ‘The
activities used enable the learners to use English outside the classroom situation’
(item 5, Teaching Materials) (see Appendix 2 for complete items of the E.E.
checklist). All the items were in Likert-type format, ranging from ‘lacking=0’ to
‘excellent=4).
 Results : A Sample (Cont’ed)
The questionnaires were first checked for ambiguity or face validity or cultural issues by
two ELT experts . After, the English version questionnaires were given to 108 student-teachers
of Farhangian Teacher Education University selected randomly from three branches (i.e.,
faculties) of Tehran, Iran. The participants were all females with age range of 20-22 who were
senior students at the time of this study. Of 108 questionnaires distributed, 89 were returned.
After obtaining the data, they were analyzed for descriptive statistics, using SPSS Software
(version 21). The analyses showed that ‘Physical Features’ obtained a mean score above 2
(X=2.32, SD=.51). However, the subscale ‘Efficient Outlay of Objectives and Supplementary
Materials’ obtained the mean scores below 2 (X=1.86, SD=.52), showing the participants’
negative perceptions of Vision 1’s Teaching aids, teaching materials and objectives. Overall, the
external evaluation factor gained the mean value of 2.10 and SD of .46 showing the similarity
of views among the participants. Although a higher mean score was found for ‘General
 Results : A Sample (Cont’ed)

Appearance’ items (X=2.66, SD=.52) none of the subscales received positive

views of the participating student-teachers. Overall, the results showed that
relatively the prospective teachers were not consent with external features and
attributes of the textbook. Table 1 below shows the descriptive statistics results
which are also graphically represented by Graph 1.
 Graphs
3
 Bar Graph 2.5

1.5

1
Series1
0.5
Series2
0

 Histogram
 Data Collection Instrument:
a. Checklist for IE
I. Content/subject matter
Lacking Poor Adequate Good Excellent
0 1 2 3 4

1. The subject and content of the book is

relevant to students’ needs as EFL learners.
2. The subject and content of the textbook is
generally realistic.
3. The subject and content of the textbook is
interesting, challenging and motivating.
4. There is sufficient variety in the subject and
content of the textbook.
5. The materials are not culturally biased and
they do not portray negative stereotypes.
6. The content is obviously realistic (i.e.,
being taken from L1 materials not initially
intended for ELT purposes).
 Checklist for EE (Cont’ed)

Lacking Poor Adequate Good Excellent

0 1 2 3 4

7. The content is presented in a

communicative or real-world way?
8. The texts/content have been unacceptably
simplified.
9. The subject matter is presented (either
topically or functionally) in a logical,
organized matter.
10. The content serves as a window into
learning about the target language culture.
11. The content contains real-life issues that
challenge the readers to think critically about
his/her worldview.
12. The text selections are representative of a
variety of genres
Good Luck
Good Luck