CHAPTER 20 ASSESSMENT GEOFF BRINDLEY

What is assessment?

Refers to a variety of ways of collecting information on a learners language ability or

achievement. Measurement instruments, such as test, qualitative methods of monitoring and
recording students learning such as observation, simulation or project work.

Proficiency assessment: Refers to the assessment of general language abilities acquired by the
learner independent of a course of study. Is often done through the administration of standarised
commercial language proficiency tests.

Assessment of achievement: Aims to establish what a student has learned in relation to a

particular course or curriculum.

Formative assessment: Assessment carried out by teachers during the learning process with the
aim of using the results to improve instruction.

Summative assessment: Assessment at the end of a course, term or school, often for purposes of
providing aggregated information on programme outcomes.

INTERPRETATION OF THE ASSESMENT

Norm referenced: Ranks learners in relation to each other: e.g a score percentage in an
examination reports a learners standing compared to to other candidates.

Criterion referenced: Learners performance is described in relation to an explicitly stated

standard.

They two key requirements for any assessment is VALIDITY AND RELIABILITY.

VALIDITY
It is related to questions about what the test is actually assessing, e.g is the test telling you what
you want to know?.

There are three types of validity:

1. Construct validity: the content of the test/assessment reflects current theoretical

understandings of the skills being assessed.

2. Content validity: whether it represents an adequate sample of ability

3. Criterion related validity: the results correlate with other independent measures of
ability.
 4. Consequential validity: the extent to which a test or assessment serves the purposes for
which it is intended.

RELIABILITY
It is an estimate of the consistency of its marks. A realiable test is one where a students will get
the same mark if he or she takes the test, with a different examiner.
is concerned with ascertaining to what degree scores on test or assessments are affected by
measurement error, by variation in scores caused by factors unrelated to the ability being
assessed (e.g conditions of administration, test instructions, fatigue, guessing, etc).

The consistency of test results can be estimated in terms of TEST RETEST RELIABILITY, where the
same test is given to a group at two different points in time or by administering two equivalent
forms of the same test.
Practicability: e.g it is not excessively expensive
It stay within appropriate time constrains
It is easy to administer
It has scoring procedure
Time efficient

Authenticity: is likely to be enacted in the real world.

The dregree of correspondence of the test characteristics of a given language test
characteristics of a given language test task to the features of the target language task.

PURPOSES OF ASSESSMENT

1. Selection : whether learners have sufficient language proficiency to be able to undertake

tertiary study

2. Certification: to provide people with a statement of their language ability for employment
purposes.

3. Accountability: to provide educational funding authorities with evidence that intended

learning outcomes have been achieved.

4. Diagnosis: to identify strengths and weaknesses.

5. Instructional decision making: to decide what material to present next or what to revise.

6. Motivation: to encourage learners to study harder.

 Discrete item tests: Related to phoneme discrimination, grammatical knowledge and
vocabulary. For example multiple choice and large number of items.

Integrative tests: more practical test, in which learners need to use linguistic and
contextual knowledge to reconstitute meaning of spoken or written texts. An example is
dictation or cloze.

Washback: the impact of assessment and testing on teaching and learning.

Topical knowledge: knowledge of the world that can be mobilized in test.

Affective schemata: emotional memories influencing the ways test-takers behave.

CHAPTER 21 EVALUATION , FRED GENESEE

What is evaluation?

Is a process of collecting, analyzing and interpreting information about teaching and

learning in order to make informed decisions that enhance student achievement and the
success of educational programmes.

Components of evaluation:

1. Articulate purposes for evaluation: Purpose of evalution is first e.g to decide whether
to continue new materials.

2. Identify and collect relevant information: Information that is relevant to the purpose is
identified and collected. Eg the teacher uses student scores on test and his observations
of performance to make decisions.

3. Analyze and interpret information: Once collected, the information is analyzed and
interpreted: feedback from students, their employers and teachers is interpreted
impressionistically.

4. Make decisions: finally, decisions are taken: the materials are kept or rejected. Each
student is assigned to an ESL or non-ESL.

Evaluation goes beyond students achievement and language assessment to consider all
aspects of teaching and learning, and to look at how educational decisions can be
informed by the results of alternative forms of assessment.

Classroom based evaluation is concerned with questions about:

 1. Suitability of general instructional goals and objectives associated with individual
lesson or unit plans.
2. Effectiveness or instructional methods, materials and activities used to attain
instructional objectives.
3. Adequacy of professional resources required to deliver instruction.

Classroom based factors

Students needs and goals, learning styles, attitudes toward schooling and second
language, learning and interests and motivations. There are other such as, community
attitudes, availability of resources and time outside school to complete assignments
Instructional and other factors to consider in classroom-based evaluation

Students needs and instructional

Abilities objectives

Community
Attitudes instructional
plans
Teachers abilities
And training

Time and resources instructional student achievement

practices
Professional support
In the disctrict

Including students in the evaluation process as active partners serves to:

*Make them aware of learning abjectives so they are better able to allocate time and energy to
fulfilling designated instructional objectives

*Instill a sense of ownership and responsibility for learning that can enhance achievement
CHAPTER 8 ASSESSING READING , ALDERSON

TYPE OF TEXTS: Narrative

Descriptive
Argumentative

Ways of reading: Intensive

Extensive

Subskills: Skimming
Scanning
Interpret

Bottom up Approach: Bottom up approach: Specific to general

Top down approach: General to specific

Type of questions in reading

Type 1: Literal comprehension: the information is explicit

Type 2: questions involving reorganization or reinterpretation
Type 3: questions of inference
Type 4: questions of evaluation
Type 5: questions of personal response (awareness)
Type 6: questions concerned with how writers say what they mean (interpreting the words)

Types of genre in reading

Academic reading:
General interests articles ( magazines, newspapers, etc)
Technical reports
Reference material (dictionaries)
Textbooks, theses
Essays, papers
Test directions
Editorials and opinion writing

Job- related reading

Messages
Letters/emails
Memos
Reports (job evaluations, project reports)
schedules, labels, signs, announcements
Forms, applications, questionnaires
Financial documents (bills, invoices )
Directories (telephone, office)
Manuals, directions

Personal reading
Newspapers and magazines
Letters, emails, greeting cards, invitations
Messages, notes, lists
Schedules (train, bus, plane, etc)
Recipes, menus, maps, calendars
Advertisements (commercials, want ads)
Novels, short stories, jokes, drama, poetry
Financial documents (checks, tax forms, loan applications)
Forms, questionnaires, medical reports, immigration documents comic strips, cartoons.

MICROSKILLS

MACROSKILLS

Some principal strategies for reading comprehension

1. Identify your purpose in reading a text
2. Apply spelling rules and conventions for bottom up decoding
3. Use lexical analysis (prefixes, roots, suffixes, etc) to determine meaning
4. Guess at meaning (of words, idioms, etc) when you arent certain
5. Skim the text for the gist and for main ideas
6. Scan the text for specific information (names, dates key words)
7. Use silent reading techniques for rapid processing.

TYPES OF READING

1. PERCEPTIVE: Involve attending to the components of larger stretches of discourse: letters,

words, punctuation and other graphemic symbols. Bottom up processing is implied. Text
are short, focuses on the form, bottom up is used mostly (e.g reading aloud- fluency and
phonetics).

2. SELECTIVE: Is largely an artifact of assessment formats. In order to ascertain ones reading

recognition of lexical, grammatical, or discourse features of language within a very short
stretch of language, certain typical tasks are used: picture cued tasks, matching, true-false,
multiple choice, etc. formal aspect and not dealing with the content.
 Texts can be short or medium, focuses more on the form than in meaning and bottom up
and top down are equally used.

3. INTERACTIVE: the reader in a psycholinguistic sense, interact with the text, that means the
reader brings to the text a set of schemata for understanding it. Examples are anecdotes,
short narratives and descriptions. The focus of an interactive task is to identify relevant
features (lexical, symbolic, grammatical and discourse). Texts are medium, focuses more in
the meaning than form, and top down is more common than bottom up.

4. EXTENSIVE: Applies to texts of more than a page, up to and including professional articles,
essays, technical reports, short stories and books. Texts are long, focuses on the meaning
and Top down is used in most extensive task.

PERCETIVE READING

1. READING ALOUD: The reader sees separate letters, words and or short sentences and
reads them aloud one by one. Any recognizable oral approximation of the response is
correct.
2. WRITTEN RESPONSE: the readers task is to reproduce the probe in writing. (to write what
he or she already read). If an error occurs, make sure you determine its source.
3. MULTIPLE- CHOICE: Students must be able to decipher amongst words that sound the
same but have different meanings. The use of minimal pairs to help students
distinguish minor differences between English vowel sounds and the words meanings.
there are different types
a. Minimal pair distinction : minimal pairs are pairs of words or phrases in a
particular language that differ in only one phonological element,
b. Grapheme recognition task

4. PICTURE CUED ITEMS: Test takers are shown a picture, along with a written text and are
given one of a number of possible tasks to perform.
a. Picture cued word identification (e.g point the word that you read here)
b. Picture cued sentence identification: with the same picture, the reader might read
sentences and then point to the correct part of the picture.
c. Picture cued true/false sentence identification (e.g the pencils are under the table t/f)
d. Picture cued matching word identification: readers read the word and writes the
appropriate letter beside the word (e.g clock _____ )
e. Multiple choice picture cued word identification: test takers might see a word or
phrase and the be directed to choose one of four pictures that is being described. To
transfer from a verbal to a nonverbal mode. (e.g rectangle and under a picture of a
rectangle and other forms)
SELECTIVE READING

1. MULTIPLE CHOICE: it is easy to administer and can be scored quickly. The items may have
little context but might serve as a vocabulary or grammar check.
a. Multiple choice vocabulary/grammar tasks (e.g he is not married. Hes _______
(single))
b. Contextualized multiple choice vocabulary/grammar tasks: (complete dialog )
c. Multiple choice cloze vocabulary /grammar task: (complete a text with the mssing
words that are down in form of multiple choice)

2. MATCHING TASKS: Most is used for vocabulary

a. Vocabulary matching task: (e.g write the letter of the definition on the right matches
word on the left)
b. Selected response fill-in vocabulary task: put a sentence wuith a missing word and
under a list of words that are part of the text. The reader has to choose which word is
best to each sentence.

3. EDITING TASKS:Correct mistakes, to edit a text.

a. Multiple choice grammar editing task: (e.g choose the letter of the underlined word
that is not correct)

4. PICTURE CUED TASKS:

a. Multiple choice picture cued responde: test takers read a sentence or passage and choose
one of four pictures that is being described. sentences are more complex at this level.
b. Diagram labeling task : test takers read a series of sentences or definitions, each
describing a labeled part of a picture or diagram. Their task is to identify each labeled
item.(una foto donde aparece una bicicleta con sus partes y abajo en letras,diferentes
oraciones describiendo dichas partes, entonces el estudiante debe colocar la letra en la
parte correspondiente.)

5. GAP FILLING TASKS (FILL IN THE BLANK)

a. Sentence completion tasks (un dialogo donde debe completer lo que dira el otro , de
acuerdo al context o que se ha leido).

INTERACTIVE READING TASKS

1. CLOZE TASKS: The ability to fill in gaps in an imcomplete image (visual, auditory or
cognitive) and supply (from background schemata) omitted details.
a. Cloze procedure, fixed- ratio deletion (every seventh word)- (completar un texto de un
essay, poniendo las palabras correctas en los lugares en blanco.
 b. Cloze procedure, rational deletion (prepositions and conjunctions)

FALTA DE READING!

FRAMEWORK FOR TEST DESIGN

TECHNIQUES FOR TESTING READING

The cloze test and gap filling tests

Close tests are typically constructed by deleting from selected texts every n-th word and simply
requiring the test taker to restore the word that has been deleted.
Gap filling tests are different, the test constructor does not use a pseudo random procedure to
delete words, here she or he decides, rationally, which words to delete, but tries not to leave
fewer than five or six words between gaps.

Multiple choice techniques

Are common to test students text comprehension. They allow testers to control the range of
possible answers to comprehension questions and also control students tought processes when
responding. The negative aspect is that the may present students with possibilities they may not
have thought of (tricking, false measure of their understanding).

Matching techniques (alternative objective techniques)

Here two sets of stimuli have to be matched against each other, for example, matching headings
with paragraphs.

Ordering tasks
Candidates are given a scrambled set of words, sentences, paragraphs or texts and they have to
put them into their correct order.

Dichotomous items
Items with only two choices. Students are presented with a statement which is related to a target
text and have to indicate whether this is true or false, or whether the text agrees or disagrees with
the statement. The problem is that students have a 50% chance of getting the answer right by
guessing alone.(e.g the text, aunder the text a set of statements, a lot of people want to
photograph Brenda a) right b)wrong c) doesnt say).

Editing tests
Consist of passages in which errors have been introduced, which the candidate has to identify. The
erroes can be in multiple choice format or open by asking candidates to identifiy the error in a
sentence and to write the correction opposite the line.

Daddy like to be with me ___ answer likes to be with me

Or, daddy likes be with me ___ answer to be with me

The C test
The second half of every second word is deleted and the students has to restore it.
Example: it i_ claimed th_ this tech___ : it is claimed that this technique.

The cloze elide test

Intrusive word technique. In this procedure it is introduced words to the text instead of delete
them, the students have to identify which words are the ones introduced. Example: test are
actually a very difficult to construct: test are actually very difficult to contruct. The negative aspect
is that we have to be sure to insert words that dont belong, the second is where insert the words,
it is not good to use insertion procedures because the candidate can identify the patron and
simply count the words.

Short answer tests

Test takers are simply asked a question which requires a brief response(for example: what is the
relation between marketa and karel?. Answer: husband and wife). The justification is that it is
possible to interpret students responses to see if they have really understood. The difference with
multiple choice is that here they explain their alternative and in the other they selected one that
may be at random or by eliminating others. The negative aspect is that is difficult to predict all
responses and interpretations of short answer questions.

The free- recall test

Students are asked to read a text, to put it to one side, and then to write down everything they
can remember from the text. (extended production response ). Helps to provide a picture of
learner processes, reveals information about how information is restoeed and organized. Need to
be in the first language otherwise is a test of writing as well as reading. To score it , is best to count
ideas and ignore meaning relationships. The negative thing is that may be more a test of memory
than understanding.

The summary test

Students read a text and then are required to summarise the main ideas, either of the whole text
or of a part, or those ideas in the text that deal with a given topic. Scoring the summaries may be a
problem but it is best for teachers to write own summaries and accept the main ideas that are
included in the model. If the summary can relate to real world task, the response will be easier for
students. Another problem is that students may understand the text but for them may be difficult
to put the ideas in writing, specially if they have a certain amount of time. It is better to ask the
students to write the summary in their own language. Another solution is to put a multiple choice,
showing different summaries and the choose the appropriate one.

The gapped summary

Students read a text and then read a summary of the same text, from which key words have been
removed. Their task is to restore the missing words, which can only be restored if students have
both read and understood the main ideas of the original text. A way to make it easier is to provide
a bank of possible words and phrases to complete the gapped summary.

Information transfer techniques

Associated with graphic tests, such as diagrams charts and tables. The students task is to identify
in the target text the required information and then to transfer it, often in some transposed form,
onto a map a table or whatever. Problems is that may be cognitively or culturally biased. Ifthey do
not know how to make a map or a table, it would be difficult to do. Other thing is that for students
may take to much time doing the tasks because they are thinking which word goes where and
trying to undertand the tasks itself, so the result may be poor.

MULTIMODALITY

What is multidmodal communicative competence?

Has undergone refinements regarding the specific skills and strategies involved. Learners need
specific strategies and skills in order to understand and produce oral and written tasks. Theses
strategies include ose of appropriate language forms (grammar, syntax), pragmatics and
negotiation of meaning, which must be adequate to the context of situation. They allow us to
perceive the link between grammar and functions of language, necessary requirements for any
communication.
Multimodal communicative competence involves the knowledge and use of language concerning
the visual, gestural, audio and spatial dimensions of communication, including computer-
mediated-communication.it seem that mcc will allow ESL/EFL learners to be better prepared for
different literacy practices In their professional and sociocultural experiences with native and non
native speakers of English.

Why is multimodality important in the teaching and learning process?

First, teachers need to understand that to communicate is to work in making meaning, and that
visual structures realize meanings as linguistic strctures do also, and thereby point to different
interpretations of experience and different forms of social interaction.
Learners have the need to go beyond practical expertise in computer based technologies of the
school to encompass an understanding of semiotic frameworks of analysis.
Teachers have to understand the 3 metafunctions: (mackenhorarik)
1. We use language to talk about our experience of the world
2. Use language to interact with other people
3. When we use language, we organize our messages in ways in which indicate how they fit
in with other messages around them and with the wider context in which we are talking or
wrting.
Metafunctions: KRESS
1. Representational: structures verbally and vissualy construct the nature of events, the
objects and particpantes involved.
2. Interactive: verbal and visual resources contruct the nature of relationships among
speakers/listeners, writers/readers.
3. Compositional: meaning are concerned with the distribution of the information value or
relative emphasis among elements of the text and image.

Language in isolation to language as communication.

Now in pedagogy language has changed from being considered as isolated, that means focus on
the grammatical components form the language itself, to know be an instrument of
comminucation and find the importance to the social interaction in the classroom, for developing
a meaningful and fluent use of English.

ASSESING LISTENING