Evaluation

Evaluation is the collection, analysis and interpretation of information about any aspect of an
educational program as part of a recognized process of judging its effectiveness, its efficiency and
any other outcomes it may have. It is concerned with assessing the effectiveness of teaching,
teaching strategies, methods and techniques. It provides feedback to the teachers about their
teaching and the learners about their learning.

What is Test?
In the classroom situation, the word test refers to formal and systematic (usually paper & pencil)
procedure to judge the knowledge, understanding and intelligence of the student.
What is Measurement?
It is a procedure of assigning numbers to a specified attribute in such a way that the numbers
indicate the degree to which the student possesses that attribute.
What is feedback?
It is the act providing information for the ongoing observations and assessment to the learners.
What is Evaluation?
It is a broader term than Test and Measurement which includes all types and examinations; to
check the knowledge as well as all aspects of the learner.
Evaluation is a process of judging / determining the value or something by certain appraisal (tool) to
assess the extent to which educational objectives are being realized.

What are the uses/purposes/importance/reasons/functions/roles of educational evaluation?

1. Helps in clarification of educational objectives are really being achieved

2. Helps in identifying the problems of Teaching and Learning process. So that desirable
changes can be made in TLS through curriculum
3. Helps in cognitive, conative and affective behavior of students

4. Helps in self appraisal of students as well as teachers

5. Helps in motivation of students as well as teachers

6. Helps in progress of TLS through innovative instructional procedures

7. Helps to adopt better educational policy and decision making

 8. Helps to provide data for research generalization
9. Helps in improvement of public relation and the mobilization of public opinion

Characteristics / principles of educational evaluation is -

1. Formal process to assess the educational objectives
 Cognitive (knowledge assessment)
 Conative (skill assessment)
 Affective (attitude assessment)
2. Quantitative assessment of teaching learning behavior
3. Qualitative assessment of teaching learning behavior
4. Continuous process and a periodic exercise of internal assessment of TLS
5. Comprehensive process to assess TLS
6. Child centered process which gives importance to learning rather than teaching
7. Remedial process which identifies the problems of teaching learning process, so that
necessary changes can be made in teaching learning behavior
8. Cooperative process which involves students, teachers, parents and peer-group.
9. Multi-dimensional process concerned with assessment of total personality of students
10. A technique which uses variety of measuring tools

Approaches / types/ techniques of evaluation in TLS

Based on time factor
 Placement evaluation – occurs beginning of the instruction to assess the overall learning
behavior of nursing students
 Diagnostic evaluation – to determine the causes of persistent learning problems especially
among average and slow learner
 Summative evaluation – occurs throughout the academic year to assess the TLS
 Formative evaluation – occurs at the end of program usually by the external agency to
assess the learning behavior of nursing students

Based on person
 Internal evaluation – carried out by the teachers and IQAC
 External evaluation – carried out by the external agency other than the college (MUHS)
Process of Continuous Internal Evaluation (CIE) System / Formative Evaluation
1. Strategic Plan of Course Curriculum (theory & practical subject)
2. Allotment and responsibility of Subject Experts for CIE
3. Communication of CIE of respective theory / practical subject
4. Mechanism of CIE (attendance, class test, midterm & pre-final exam, IA marks)
5. Grievance Red-ressal
6. Grading of Internal Assessment Marks

Steps of evaluation / how to evaluate the educational objectives?

1. Set the educational objectives
2. Operationally define the materials and methods used to measure the TLS
3. Develop the appropriate tools to measure the TLS
4. Check the validity and reliability of tools
5. Collect data on TLS and develop the methods for interpretation
6. Analyze and interpret the data on TLS with reference to stated objectives
7. Formulate the judgment for further course of action
a. Recording and reporting
b. Make changes in curriculum and teaching learning strategies
c. Make changes in administrative functions

Test construction
Different evaluation tools are constructed and evaluated by the teachers to measure the TLS of any
given educational institute. Test construction is a systematic and well organized process where the
set of activities involved in planning, developing and evaluating the test items concerned to
assessment of cognitive (knowledge), conative (skill) or affective (attitude) domain of learner.
There are various strategies in test construction where the test items are created and decided upon
to measure the learning behavior. Generally, for all educational test construction, the following five
steps are used -
 Planning the test
 Preparing the preliminary draft of the test
 Trying out the preliminary draft of the test
 Evaluating the test (validity and reliability)
  Construction of the final draft of the test

What are the purposes/importance/reasons of test construction?

1. To assess the learning behavior of students (Cognitive, Conative and Affective domain)
2. Test paper helps in clarification of educational objectives are really being achieved
3. The test results helps in identifying the problems in teaching and learning process.
4. The written test / exam helps in self appraisal of students as well as teachers
5. The test results motivate the students as well as teachers
6. The written exam promotes innovative instructional strategies
7. Regular written tests helps the students in progressive development at academics
8. Allows development of writing skill – selection of words, composing sentence, organize
paragraph, grammar, spelling etc.
9. The essay tests allows the students to express their own ideas
10. Helps in identification of advanced learner, average learner and slow learner

Construction guidelines / principles of test construction

 The test items should be adaptable to the level of understanding of different types of
learners / respondents.
 The items in the test should not be too easy or too difficult.
 The test items must be precise, clear and as simple as possible. It must have clarity in
reading and clarity in meaning.
 The item in the test must precisely reflect ‘Gets to the Point’.
 While constructing, one should be cautious that the items must not overlap.
 Avoid using non-functional words, or words that make no contribution towards the accurate
meaning of the test item.
 Avoids using stereotyped words.
 The item must represent very specific expected response in terms of time, place, object,
event, fact or a person while constructing objective type test (the memory must recall or
choose one response only).
 Avoid vague words like sometimes, usually, always, may be, might be, more or less, few,
more, small, large etc. as these words have no place in objective type questions /
statements.
 The item should not reflect relevant clue in objective type test. The clues in terms of similar
words, place and events may attract guessing. Always better to avoid items that provide
irrelevant clues.
  The test has to be constructed in such a way that the items don’t encourage guesswork in
objective type test.
 The length of statement should not be too small or too short in objective type test.

Techniques and tools used in evaluation of TLS

There are various tools and techniques of evaluation. Generally, they are classified into two
categories -
1. Quantitative technique / Achievement tests
 Oral
 Written
 Practical based

2. Qualitative technique / Standardized or non-standardized tests

 Interview
 Observation
 Rating Scale
 Checklist
 Questionnaire

Oral Test / exam

The oral exam (also oral test or viva voce) is a common practice in many educational institutes
where an examiner poses questions to the student in spoken form. The student has to answer the
question in such a way as to demonstrate sufficient knowledge of the subject in order to pass the
exam. Many programs require students to finish the program by taking an oral exam or a
combination of oral and exams in order to show how well a student has comprehended the
material studied in the program.

Written exam / test

Written exam / test are the evaluation tools used to measure the cognitive domain; either
administered on paper or on a computer. Presently it is widely used tool among all the educational
institutions in India. However, the written examination includes the following type of test.
1. Essay type (free response)
2. Short answer type (free response)
3. Objective type (fixed response)

Essay type test

In essay type test, the students respond to questions in the form of description or explanation
where student are free to select, relate and present ideas in their own words. This type of tests is
commonly employed to check the power of memorization, expression, recognition, etc. hence; the
teachers use this test very often to measure the level of interpretation and evaluation of the
learner.
Types of essay type test
Restricted response questions: limits the content and response by posing very specific questions or
question on specific problem.
 List the article used in some procedure
 Briefly explain any one health problems
Extended response questions: allows the students where they are free to select, relate and present
ideas in their own words to respond any factual information in the form of description or
explanation.

Scoring method / approaches to essay type test

 Analytical method: answer broken in to several points and score is assigned.
 Holistic method: read the entire essay and assign the score.
Strengths of essay type test
1. Easy to construct quickly with a low cost
2. Suitable for all type of subjects
3. Allows the students to express their own ideas
4. Allows to measure cognitive domain of student
5. Can test complex learning outcome
6. Allows development of writing skill – selection of words, composing sentence, organize
paragraph, grammar, spelling etc.

Limitations of essay type test

1. The results of this test may not consistent as the personal views, opinions, ideas, etc. of
valuator and student affect the response of a particular answer
2. The results of this test may not consistent as the personal views, opinions, ideas, etc.
between the valuators may affect the response of a particular answer
3. Time consuming when compared to objective type test
4. Accurate and precise evaluation may not possible
5. Poor handwriting, spelling error, poor organization of content may affect the answer
6. Learner may bluff by guessing
7. Encourages many words without specific information
 8. Time pressure may affect the responses to particular answers

Short answer type test

In short answer type test, the students respond to questions in the form of brief description or
explanation where student are free to select, relate and present ideas in their own words (usually
one to three paragraphs). This type of tests is commonly employed to check the power of
memorization, expression, recognition, etc. hence; the teachers use this test very often to measure
the level of interpretation and evaluation of the learner.

Objective type test

The objective type test are called selection type as they are answered by just writing one or two
words, or numerals, fill in the blanks, choosing one cut of multiple responses given, etc.

Types of objective type test (selection items / questions)

1. Constant alternatives (true/false, yes/no, agree/disagree)
2. Matching
3. Rearrangement types
4. Multiple choice

Principles of objective type test (selection items / questions)

1. The items must be precise, clear and as simple as possible
2. The item must represent very specific expected response in terms of time, place, object,
event, fact or a person (the memory must recall or choose one response only).
3. Avoid vague words like sometimes, usually, always, may be, might be, more or less, few,
more, small, large etc. as these words have no place in objective type questions
4. The item should not reflect clue to any one of the alternative. The clues in terms of similar
words, place and events may attract students.
5. The item must not be overlapped
6. The length of statement should not be too small or too short

Strengths of objective type test

1. Easy to construct, quick and accurate with a low cost
2. Allows to measure wide range of cognitive domain
3. Can test simple learning outcome
4. Prevent bluffing due extremely structured tool
 5. Minimizes the time due rapid answer
6. Scores are unaffected by the opinion of valuator
7. Scoring is easy, quick and consistent

Limitations of objective type test

1. Subjected to guessing
2. Not suitable for all type of subjects
3. Do not measure thinking skill / can assess only what & when but not how
4. Need long time for construction

Constant alternatives / alternative choice questions

This type of questions administered on paper or on a computer, where the student is required to
choose a right option from two or more alternative in the item. The choices may be True/False,
Yes/No, High/Low, Agree/Disagree and so on.

Strengths of constant alternatives / alternative choice questions

1. Can be used frequently as formative evaluation at the end of each class
2. Quick and accurate with a low cost
3. Allows to measure wide range of cognitive domain
4. Can test simple learning outcome
5. Time saving tool as it takes less reading time of student
6. Scores are unaffected by the opinion of valuator
7. Scoring is easy, quick and consistent

Limitations of constant alternatives / alternative choice questions

1. Subjected to extreme guessing as there are only two alternatives
2. Not suitable for all type of subjects
3. Do not measure thinking skill / can assess only what & when but not how
4. Tough to construction

Matching type items

Matching items is two adjacent lists or columns of related words, phrases, picture, or symbols
where each item in first list is paired with one item in another list. It is used when we need to
measure the learner’s ability to identify the relationship or association between similar items. The
item in the column for which the selection is sought is known as premises and the items in the
column for which selection is made are known as responses. The students are directed to look in to
the premises one after the other and match it with one of the right option.
Construction guidelines for writing matching type items
1. Parallel content: within one matching test item, use a common approach, such as all terms
and definitions or all principles and the scenarios to which they apply.

2. Plausible answers: all responses in first column should be plausible answers to the premises
in second column otherwise; the test loses some of its reliability because some answers will
be “give-aways.”

3. Clueless: ensure the premises don’t include hints through grammar (like implying the
answer must be plural) or hints from word choice (like using the term itself in a definition).

4. Unequal responses: it is an ideal that we should present more responses than premises, so
the remaining responses don’t work as hints to the correct answer.

5. Limited premises: due to the capacity limitations of working memory, avoid a long list of
premises in the first column; usually 5-6 items, even less might be better.

6. One correct answer: every premise should have only one correct response. Obvious, but
triple-check to make sure each response can only work for one premise.

7. Long and complex statement: should be kept as premises so that each is read once. Shorter
and simpler sentences are kept as responses where they can be read quickly.

Rearrangement items
In this type of items the learner is required to put in order a set of randomly presented material in
terms of size / event / time etc.
Example: following are the steps of nursing research. Arrange them in order of hierarchy –
1. Validity and reliability
2. Conceptual framework
3. Analysis and interpretation
4. Data collection
5. Hypothesis

Multiple choice questions

Multiple Choice Questions (MCQs) are one of the most widely adopted tools for assessing the
teaching learning behavior. It contains two parts; the stem and the options. The stem presents the
problem / question while the options present the set of possible answers. Out of options, only one
is correct or definitely better than others. The stem can be a question or incomplete statement.

Types of multiple choice questions

1. Correct answer type – only one option is correct
2. Best answer type – the best out of all options need to be selected

Construction guidelines for writing multiple choice questions

1. Necessary instruction to be made available for the user and consumer of MCQ
2. The stem must be precise, clear and as simple as possible
3. The stem must represent very specific expected response in the given options
4. Avoid vague words which leads ambiguity between the options
5. Organize the options in terms of time, size, events, alphabet etc.
6. The stem should not reflect clue to any one of the option. The clues in terms of similar
words, place and events may attract correct or wrong option.
7. The stem should not be too small or too long. Similarly, the options should not be too small
or too long.
Strengths of multiple choice questions
1. Quick and accurate with a low cost
2. Allows to measure wide range of cognitive domain
3. Can test simple to complex learning outcome
4. Minimizes the time due to rapid answer
5. Scores are unaffected by the opinion of valuator
6. Scoring is easy, quick and consistent
7. Useful for large group

Limitations of multiple choice questions

1. Probability of guessing to certain extent
2. Not suitable for all type of subjects
3. Do not measure thinking skill / can assess only what & when but not how
4. Need long time and skill for construction
5. Difficult to find sufficient number of options

Checklist (closed ended question / dichotomous question)

A checklist is a tool used to measure specific criteria that gathers information in order to make a
judgment regarding the matter being assessed. In simpler words, it is a method of recording
whether a characteristic is present or absent or an action had or had not taken place; thus it
provides simple yes/no, true/false, agree/disagree judgment. Checklist contains two dimensional
matrixes in which a series of questions / statements is listed on one dimension (usually vertically)
and the response options are listed on the other. The content of questions / statements in the
checklist includes; specific concepts, skills, process or attitude etc.

Construction guidelines for writing checklist

1. Necessary instruction to be made for the user and the consumer of checklist
2. The questions / statements must be precise, clear and as simple as possible
3. The list of questions / statements must be in sequence
4. Every question / statement in the list must represent very specific expected response
5. Avoid vague words like sometimes, usually, always, may be, might be, more or less, few,
more, small, large etc. as these words have no objectivity
6. The item must not be overlapped
7. The length of question / statement should not be too small or too short

Strengths of checklist
1. Inexpensive and widely used tool
2. Allows to measure wide range of conative (skill) and affective (attitude) domain
3. Can test application of knowledge in to practice
4. Scoring is easy, quick and consistent
5. Useful for large group
6. Can be used for self appraisal

Limitations of checklist
1. Measures specific characteristic whether present or absent
2. Need long time for preparation list
3. Need skill for tool construction
4. Sometimes, scores might be affected by the observer / interviewer

Rating Scale (closed ended question)

Rating scale is a tool used to rate the qualitative phenomena / characteristic in the bipolar
descriptive or numerical continuum which has specific intervals in the scale. In other words, rating
scale is defined as a closed-ended question used to represent respondent feedback in a
comparative form on specific / particular aspect. In simpler words, it is a tool in where one person
simply checks off another person’s level of performance. The number of characteristics in the scale
may be three point, five point, seven point or more.

Types of rating scales

1. Numeric rating scale
2. Graphic rating scale
3. Descriptive graphic rating scale

Numeric rating scale: is a tool contains quantitative symbolization of an attribute where the
number measure or rate the qualitative phenomena or performance of something. For example:

To what extent the students have performed in the unit

5 4 3 2 1
(5 outstanding, 4 above average, 3 averages, 2 below average, 1 unsatisfactory)

How satisfied were you nursing care?

0 1 2 3 4 5 6 7 8 9
(extremely satisfied) (extremely dissatisfied)

Graphic rating scale: is a tool contains bipolar descriptive continuum which has specific qualitative
attribute from lowest to highest. For example:

To what extent the students have participated in the event?

Never Seldom Occasionally Frequently Always

Descriptive graphic rating scale: is a tool contains bipolar descriptive continuum which has
descriptive qualitative attribute. It does not use numbers; but divide the assessment into series of
verbal phrases to indicate the level of performance. For example:

Ability of IM injection

Demonstrates Demonstrates Demonstrates Demonstrates no

outstanding proficiency with minimal proficiency
proficiency direction proficiency
Strengths of rating scales
1. Inexpensive and easy to administer
2. Allows to measure wide range of affective (attitude) domain
3. widely used tool to carry out structured observations
4. Makes quantitative judgments after assessing qualitative attributes
5. Makes qualitative judgments after assessing quantitative attributes
6. Provide more flexibility to judge the level of performance
7. Easy scoring, quick and consistent

Limitations of rating scales

1. Scores might be affected by the observer / interviewer; Sometimes, it is difficult to fix up the
rating about many aspects of the attributes
2. Sometimes, it is difficult for the observer / interviewer to rate when two characteristics are
alike or more or less similar (logical error)
3. Halo effect in the judgment may take place; chances like, the rater may over estimate the
qualities of a known person and underestimate those of unknown
4. Need time for assessment

How to assess the qualities or characteristics of the tools? / Evaluation of tools

There are some benchmark criteria for assessing the qualities of evaluation tools / instruments.
They are - Validity, Reliability, Practicability and objectivity.
Validity: is the degree (accuracy) to which it measures what it supposes / intends (criteria) to
measure. The types of validity includes –
1. Content validity: is the degree to which an assessment measures the subject matter in terms
of continuity, sequence and integrity of tool.
2. Predictive validity: is the degree to which the current performance on an assessment
predicts the future performance on another assessment.
3. Concurrent validity: is the degree to which the performance on one assessment accurately
estimates the current performance another assessment or another task.
4. Construct validity: is the degree to which the performance on one assessment may be
explained by the presence of or absence of some traits.
Reliability: is concerned with the extent to which it measure and gives consistent results in terms of
accuracy, consistency, and precision. The reliability of an instrument is said to be reliable or
acceptable when the score is 0.70 or more after statistical analysis.
Practicability: refers to the development of evaluation instrument is capable of being administered,
interpreted and scored with reasonable ease within the time limits.
Objectivity: is the quality of a test or instrument, which permits equally competent scores to obtain
identical or very closely similar score on the same test administered to an individual / attribute.