EVALUATION

THE NATURE AND SCOPE OF EVALUATION

What is evaluation?

Evaluation is something that we do as part of our everyday lives. If we are involved in any activity, we
normally know which aspect of it is going well and which is not and why. We can then say that we are
evaluating the activity.

In the classroom too, if you ask yourself the question, ‘Did all students participate in the class?' or ‘Was
the exercise pitched at the right level?' or ‘Did some students find it difficult?' you probably know the
answer. If you do, then you have been evaluating your own teaching.

Evaluation, then, is quite a natural activity, something that we normally do as part of our everyday work.
This we do consciously or unconsciously and formally or informally. It is not something that we always
do consciously and systematically. For example, if you were not sure of the answer to the question 'Did
all students participate in the class to the same extent?', it could be that you weren't paying much
attention to it because there were many other things, probably more important, on your mind. It could
also be because you didn't know that it was important and that you should have observed it carefully.
On the other hand, it might have been that you knew that it was important, relevant or crucial, but you
didn't know how to evaluate it.

But first of all, why should we be aware of what is happening and how it is happening? And second,
what can we do with that awareness?

In thinking back to one’s class, one is actually reflecting on one’s own teaching, what one had intended,
the consequences of one’s intentions and the reasons for those consequences. One is, in other words,
trying to consciously look back on and interpret one’s own class. Further, this was possible since one had
some idea of what happened in class and why. In other words, one had evaluated one’s class, maybe not
very consciously, but one was generally aware of what was going on in the classroom.

Why evaluate?

If we asked the question Why should we evaluate at all?' the answer would probably be straight-forward
and simple : so that we may know how things are going on, so that if they are not going well, we could
probably do something about it. Yes, it seems that as teachers we would like to manage our classrooms
efficiently so that maximum learning takes place and that learning is enjoyable. Most of us seem to do
some amount of evaluating, sometimes unconsciously and informally. However, the question is, do we
need to do it more consciously and systematically, in a deliberate and planned way? Why?

A lot of what we do in class is probably routine since a lot of what happens there is routine. Therefore,
we are not very often aware of why we do what we do. In fact, teachers are known to be quite
notorious for denying what they actually do in class, not because they don't like to admit it but because
things happen in such quick succession that- they can't keep track of it all. For example, in a training
programme, when teachers were shown video recorded lessons of their own classes, they made these
comments : ‘l didn't know I talked so much!' ‘l thought I moved around the class, but I seem to be
standing at the front all the time!'

Why do teachers deny what they do in class? This could be mainly because the teaching-learning
process is not a neat and predetermined set of intentions and outcomes. In fact, there is little
congruence between what is intended and what is achieved due to a variety of variables/factors that
influence teaching/learning.

If this is so, it seems necessary then to closely monitor the different aspects of the classroom and get
detailed feedback about these aspects for creating a more conducive teaching/learning situation. What
are these different aspects or components?

A teaching-learning context includes all the factors that contribute to the learning process such as the
syllabus/objectives, materials, methodology, teacher input/performance and student
input/performance. These are all part of the curriculum and must be evaluated to get a complete,
holistic picture of the whole process.

A term which is very often interchangeably used with evaluation is testing or assessment. When we
carry out assessment or administer a test, we measure the performance of students and the results of
such a procedure may determine whether a learner moves into a higher class or may inform us how well
s/he performed in the test. Assessment is also made towards the end of a course to find out what the
learner has learnt. It therefore has a limited perspective with a focus on the product of learning, while
evaluation focuses on the means and is intended to serve the learning process.

The scope of evaluation

Changes in our views on language and language learning could bring about changes in approaches to
language teaching-learning and in the role of teachers and learners. These changed views have also
influenced the concept and practice of evaluation in the classroom. For example, evaluation includes
looking in a principled and systematic way at all the factors that affect learning. What is the
motivation for undertaking such a comprehensive review? First, it is our innate tendency to understand
what worked well and what did not work well. Knowing that a student has passed or failed a test does
not help us understand what worked well and what did not work well.

Let us look at an example: Suppose that you have changed from a "traditional' approach to a more
'communicative' one and you want to find out whether it is effective. There are a number of ways in
which we could evaluate the merits and demerits of the new approach.

A fairly standard practice is to administer a language test to the learners at the end of the programme.

What interpretations can you make of the test results? Tick the right alternatives:

a) The communicative approach was on the whole not successful.

b) It was successful in some cases since all the students did not get low scores on the test.
c) The approach used was not communicative.
d) The teacher talked most of the time.
e) The learners were not motivated enough.
f) The learners' previous language background was not adequate for the communicative approach
to be effective.
g) The teacher's own language was not satisfactory.
h) The classroom atmosphere was not conducive to the new approach.
i) The atmosphere outside the class was not conducive to the approach.
j) The subject teacher's negative attitude affected the classroom climate.
k). The Principal was critical of the "noisy' classes.

These and probably other factors could as well have been responsible for the "failure' of the
programme? However, scores on tests, especially end-of-course ones do not tell us enough about a
course. We will need to examine other factors that contributed to the failure. Further, for a course
evaluation we not only expect to know whether something worked or not but also explanations for
what happened. This is possible only when we get detailed feedback about different aspects.
Therefore evaluation involves examining the various aspects of the teaching-learning process to get a
better understanding of what is not effective, what is more effective and what appears to be of no use
at all. In order to do this, evaluation must be on both the means (process) and the product of the
teaching-learning process.
We could think of evaluating, say, the materials or the books prescribed in order to see whether they
are appropriate for your learners. Or, you may want to monitor the methodology used i.e. classroom
organization, nature of learner participation, amount of teacher vs. student talk, etc.

You might want, if you would like a holistic picture of the entire teaching-learning situation, to examine
all the important factors that contribute to the process. We should try and include as many aspects as
possible so that our evaluation is comprehensive.

Another motivation for evaluation is to bring about change. The feedback or information gathered
during the evaluation can be used to suggest changes to the existing practice. In a sense, therefore, all
evaluation is, to some extent, innovative, provided the feedback is put to proper use. For example,
one of the aims of the monitoring and evaluation study of the CBSE course is to recommend to the
Board the different changes suggested by the evaluation data in the areas of prescribed books, the
approach(es) in use, teacher training, testing pattern, etc. Very often, however, evaluation studies are
taken up which only describe or at the most explain the existing practices, but the data is seldom used
to bring about any innovations. The problem here could be that the agency responsible for effecting
change may not be interested in innovations because of administrative and financial implications.
Whatever that may be, we have to keep in mind that evaluation is a very powerful tool that can be
wielded to bring about changes in the (education) system provided we are interested in bringing about
the change. By the same token, it means that evaluation must be carried out in a systematic and
planned manner and that the job of those involved in evaluation is a very responsible one.
Summing Up

So far, we have discussed the nature and scope of evaluation. We have seen the need for evaluating all
aspects of the teaching-learning process, including that of student learning or outcome which is in our
case the ability of students to use the language. We have stressed the importance of formative
evaluation during a course, which involves gathering systematic information about all the components
that contribute to language learning, as well as summative evaluation, which provides a statement of
students' end-of-course performance. The two purposes of evaluation have been highlighted : one, to
explain and confirm existing procedures in order to understand the situation and two, to bring about
change in the existing system. This is possible when evaluation is based on feedback and the feedback is
put to proper use. Therefore, evaluation is seen as a powerful instrument of change.

Review question

From what we have discussed so far, mention the main limitations of using only test results as a means
of evaluating the strengths and weaknesses of the teaching-learning process.

Answer:

Teaching-learning involves many factors such as syllabus/objectives, materials, methodology, teacher

factors, student factors which include learning outcomes. If we look only at test results, we will know
nothing about the other factors that are equally important. Therefore, our interpretation of the
strengths and weaknesses of the teaching-learning process will be at best severely limited and at worst
distorted. That is, test results, if used as a summative measure, will inform us only about the product,
not the process of learning.
 EVALUATION OF TEACHING-LEARNING
Let us now look, in detail, at the factors that contribute to teaching-learning in the classroom and
examine how we can evaluate the way they influence learning. A question that is crucial to evaluation in
the classroom is that of ‘Who should evaluate?'

As we have already seen, there are a variety of factors that contribute to the teaching-learning process
in the classroom. These are: syllabus/objectives, materials/books used, methodology/ classroom
procedures, teacher inputs such as his/her beliefs, attitudes, aptitude, training, experience and student
inputs. And students inputs include the students' socio-economic status, home background, parents'
education, motivation, their first language/mother tongue, attitude, interest, and so on.

All these factors and many others which we may not be able to label or list so easily do influence
learning in the classroom and outside. And that learning, in turn, affects these different factors.
Therefore we can see how complex the phenomenon of learning in general is.

Factors that affect classroom learning

The different factors that affect classroom learning are listed below:

Classroom atmosphere/climate
Syllabus /objectives
Principal's support/ understanding of the curriculum

Prescribed materials and other materials

Teaching strategies/techniques
Classroom management/ organisation
Classroom teaching-learning (Interaction)
Other teachers' views/inputs

Teacher factors : motivation, attitude, experience, training, qualifications, beliefs, language competence,
etc.

Student factors : attitude, motivation knowledge of other languages, socioeconomic background etc.

Can we study or evaluate all the factors that seem to be in some way related to the
phenomenon in question i.e. teaching-learning?

Well, we would like to evaluate all the factors that contribute to teaching-learning in order to get a
holistic picture of the situation. Therefore, it is desirable that we study as many factors as possible. But,
given the time constraints and other financial/administrative constraints, it may not always be possible
to carry out such a detailed evaluation. This could be because of two reasons : (i) we may have academic
reasons for not considering all the factors and (ii) if the evaluator is an outsider, s/he may not be in a
position to evaluate all the factors in a teaching-learning situation.

There are situations when we do not want to study all the factors in the teaching-learning process. For
example, if we are evaluating the efficacy of classroom techniques, we may not want to study the
effectiveness of materials or learner variables because we would like to know which strategies are more
effective. We may still be interested in relating it to student-related factors—process and product (Qns.
c & e). Therefore, there is a good reason for focusing on one or two aspects at a given time for a
particular purpose.

Who should do the evaluation?

We are familiar, in Indian schools and universities, with external evaluators assessing student outcomes.
It is usually summative, i.e. done at the end of the course mainly for certification purposes. These
outsiders/external examiners (who may or may not be teachers and if they are, not of the students they
are evaluating) are brought in on the grounds that they are more objective (than insiders) and do a more
credible assessment. The teacher's role in these exams is normally limited only to invigilation.
Schools/colleges (or exam centres) conducting external exams are required to keep question papers
confidential, make sure desks are at a certain distance from each other, issue numbers, hall tickets and
instructions and ensure that the exam is conducted according to the prescribed rules. What is the
teacher's and student's role here? Teachers here are mere Invigilators' and are very often forbidden
from even invigilating their own students' exams. Students are called "candidates', Since the students
they are assessing are just "candidates', it is believed that they have no vested interests and "blind
spots' which can affect evaluation.

Do you think these outsiders have no biases/convictions/perceptions of their own? What according to
you are the problems in external evaluation?

Indeed, even outside evaluators must have some preconceived notions about what is good and bad,
based on their previous exposure/training/ experiences. This would affect the evaluation especially
when the criteria for assessment are not clearly spelt out, which is the case with most of our exam
systems. The problem gets compounded when it is only one end-of-course measurement that we rely
on.

We are here, however, not talking about only summative evaluation. We are in fact emphasising the
need to evaluate as many aspects of teaching-learning as possible in a formative framework i.e. during
the course plus summative measures to complement it. The term formative suggests that there is a
focus on the process of learning in evaluation. We need to evaluate further not just to give marks but to
help teachers and students to understand difficulties and errors faced by learners. We have so far over-
emphasised assessing language ability to the total neglect of other aspects of teaching-learning.

The teacher as evaluator

We are therefore talking about what happens in the classroom, what causes it and what results from it.
If our focus is on the classroom, who do you think should be the evaluator? Of course, it should be the
teacher herself/himself.What are the advantages and disadvantages of the teacher carrying out
evaluation in the classroom?

In a sense the teacher, who is the insider in the classroom, has far greater [experience and
understanding of the situation than anyone else. S/he has deep insight into the working of the different
aspects and in particular understands the predispositions and relationships that affect the activities in
the classroom. Therefore, we can say that technically s/he is the most qualified for doing the job.
Secondly, s/he is right there on-site all the time. If we were to think of external evaluators, the duration
of their stay would be limited and further it gives rise to other questions of authenticity and validity of
the whole process.

There are, however, some problems involved in the teacher being the evaluator. One is that the teacher
is busy teaching or, to use a more acceptable term, facilitating learning!

Does the teacher have the time to evaluate? If so, how will s/he divide it between teaching/facilitating
and evaluating?

We have already seen how all of us, whether consciously or unconsciously, are evaluating almost
everything we are involved in. And this applies to teaching as well. Therefore, it is not as though as
teachers we need to think of evaluation as another component to be accommodated separately into the
time-table. In fact, the demarcation between teaching and evaluation is thin. And if evaluation is an
integral part of teaching, we will need to see it as another essential role of the teacher.

There is however another serious question that we need to consider: Are teachers equipped to carry out
evaluation? And, are they able to carry out systematic evaluation that is valid and dependable? Are all
(good) teachers (good) evaluators? The answer is probably ‘no'. For, proper, constructive evaluation is
not just common sense but it needs some orientation/training and experience.

Since we would like an insider perspective on evaluation and we reckon that the teacher needs some
training/orientation, it may not be humanly possible for the teacher to study all the factors
simultaneously. Another problem facing the teacher-as-evaluator could be time. We would however like
to stress the need for teachers to carry out evaluation in the classroom in a formative mode. This brings
us to the question of the other party in the learning process, a more important one at that - the
learners.

The learner as evaluator

Evaluation as we said earlier has come to be viewed as an integral part of the learning process and not
only as something external and imposed from the outside. Communicative approaches to language
teaching and learning emphasize that since evaluation is inherent in communicative interaction this can
be exploited in monitoring the learning process and in guiding learning itself. An essential feature of
becoming a successful learner appears to be the development of the ability to evaluate oneself and
others (peers in this case) to help one’s own learning. Thus evaluation is not to be seen as something
done by an outsider for assessing the performance of the learner on the basis of externally defined
criteria. It should be seen as a dynamic process in which the learner is involved interactively with others
in the class, first, in deciding the criteria for evaluation and secondly in actually evaluating it. The
feedback that both teacher and students receive in such a multi-way system would be meaningful and
useful to the participants. Therefore, an important point that emerges from this discussion is that
evaluation is not carried out in a static way with pre-determined external criteria, but in negotiation
with the different participants. Of course, this would mean that learners will need some orientation or
training to be able to do the evaluation in a systematic and principled way.

In fact, involving the learner in the evaluation process itself is a way of enabling him/her to do it well.
By actively participating in the teaching-learning-evaluation process, learners will gradually internalize
the nuances of the evaluation system and will be able to interpret the results/feedback in a way that is
meaningful to them which would enable them to become better learners. As a result of this, they will be
better able to participate in the evaluation process instead of having it imposed on them by
experts/outsiders.

Very often teachers feel that students are not ready, they are not "experts' or they don't know enough,
they may be able to judge certain aspects but not others, they will not be objective, etc. Some of these
apprehensions may be true, but we must remember that we have never or seldom given them a chance
to have a say in matters that relate to them! Therefore, what most of us have seen is, when we do ask
them for their opinion/judgement of their own performance, they are actually too critical and modest
and too severe on themselves (contrary to the popular belief that they will cheat the teachers).

Therefore what we are talking about is self-assessment or self evaluation. This also includes peer
evaluation which is what is implied in interactive learning, a notion that communicative approaches
strongly advocate. Cooperative/collaborative learning allows for deployment and exploitation of
learners' "strengths for mutual benefit. Peer evaluation concretizes this notion and enables evaluation
to take place in a non-threatening climate. We have often encountered the situation where evaluation
has achieved the opposite, unsuccessful learners getting frustrated and successful ones becoming
complacent. It is largely because the evaluation system treats learners as objects rather than as
interactive, responsible participants in the learning process.

How should we evaluate?

We have so far discussed the need to evaluate as many factors as necessary and possible in the
classroom, in order to get a complete picture of the on-going activities. One point that emerges when
we talk about the on-going nature of teaching and evaluation is that both are inextricably linked with
each other and that one feeds into the other. In fact, it is difficult to say when one stops and the other
begins. Therefore, one significant feature of feedback, especially within a formative mode, is that of
classroom evaluation.
We are perhaps only too familiar with how even class tests/exams (of language ability) have always
tended to serve the function of summative assessments. That is, scores on a 100-mark paper-pencil test
(=a typical written exam which at the most focuses on reading and writing skills) are awarded for tasks
which only reflect a product orientation. Qualitative descriptions focusing on on-the-way skills or on the
different steps the learner takes to complete the task are seldom a part of a class test. Discussions of
how a student has performed, where s/he has gone wrong and why, what will fetch a better score i.e.
how exactly s/he can improve the skill, get a low priority compared to the main job of teachers, in their
view, for which they are held accountable. The main and most important task of teachers, as we very
well know, is "syllabus completion'! Giving feedback for developing language skills seems only secondary
and incidental. On the contrary, we are here stressing the utmost value of feedback for evaluation in the
classroom.

The second question that arises is that of the modes or means we should adopt for evaluating the
different factors. There are informal as well as formal means of evaluation.

What are the different modes available to the teacher? Classroom observations, diaries, logs, journals,
questionnaires, checklists, quizzes, tests, etc. are all well known. These could be broadly seen as
measurement-based methods, e.g. tests, quizzes, attitude scales, checklists; and descriptive data-based
methods e.g. classroom observations, diaries, journals (discussed at length in Practice Teaching), These
could be used to evaluate any of the factors affecting classroom processes. For example, the teacher
might want to evaluate his/her own teaching - the kinds of interventions /inputs s/he provides - for
learning to happen. Or s/he may look at the interactions/ processes that are generated as a result of the
inputs.

Finally, let us look at the nature of feedback. The more descriptive and detailed it is, the more
informative it will be and will serve the purpose of giving meaningful information to the student for
improvement. Also, if the descriptions emerge from the student's own performance, it will be more
authentic, and useful to the student.

Review question

Let us return to our earlier question. Can we evaluate all the factors that contribute to classroom
learning? What is your view on this after you have read through the foregoing discussion on 'Who
should evaluate?'

Answer:

It is of course possible to evaluate factors that contribute to language learning. It is not, however,
necessary or feasible at all times to do so. This is especially true if the teacher is the evaluator, since
there are constraints on his/her time, expertise, etc. Also, we would like to suggest that we should focus
on those aspects that are crucial in a given situation for a given purpose.