14/02/2023

Learning objectives
Program
Evaluation

Professor Dr Shabana Ali 1. Discuss the

2. Identify the ways of
Head of Anatomy Department importance of
assessing the
Assistant Dean Medical Education evaluating a teaching
effectiveness of an
session/ course/
educational program
program.

1 2

Quality Assurance Evaluation

An ongoing, continuous process of

evaluating (assessing, monitoring,
guaranteeing, maintaining, and
improving) the quality of a institutions/ • Systematic collection and analysis of
program. information related to the design,
It focuses on both accountability and implementation, and outcomes of a
improvement, providing information and program, for the purpose of monitoring to
judgments through a consistent process improve the quality and effectiveness of a
and well-established criteria. program

3 4

1

Program evaluation
• In medical education, a program could be class
session, a course, or a clerkship rotation in medical
school or the whole of an educational program.
5 6
Reasons of program evaluation
• To determine the effectiveness of programs for participants.
• To document the achievement of program objectives.
• To decide about possible changes in program staff to improve
program effectiveness.
7 8
14/02/2023
• Evaluation, an integral part of an educational
program, is the process of judging the effectiveness
of educational experience through careful
appraisal.
• According to Ralph Tyler, Evaluation is the process
of determining to what extent the educational
objectives are being realized.
• Evaluation is a continuous systematic process that
differs from measurement.
Program Evaluation: a process used to determine whether the
design and delivery of a program were effective and whether the
proposed outcomes were met.
• To measure the progress of a program
• To identify any problem and to resolve any conflicts.
• To enhance utilization of available resources.
• To provide baseline information for future evaluation and planning.
• To modify and to make any remedial measures.
2
 14/02/2023

Types of evaluation
Types of Program Evaluation
Internal evaluation
• Evaluation by teacher through teacher made and
standardized tests, observation and other techniques
• Formative Evaluation: Evaluation done to improve or change a program
External evaluation while it is in progress.
• Summative Evaluation: Evaluation focuses on outcomes of a program.
• External evaluation is organized and conducted by an
external agency, other than the college.

9 10

Formative evaluation
• is done during an instructional program.
• The instructional program should aim at the attainment
certain objectives during the implementation of the program.
• It is done to monitor learning and modifying the program if
needed before its completion.
• Formative evaluation is for current students
Summative evaluation
• is done at the completion of an instructional program
• whose duration may vary from a semester to whole year and to
know whether the student is competent enough for certification.
• worth or merit of a program

11 12

3
 14/02/2023

How do we evaluate ?

• Who will do this evaluation ?

• What do you need to know?
• Why do you need to know it?
• Will the evaluation make a
difference?

13 14

Goals of Program Evaluation Elements of a Systematic Evaluation

• Judging the worth of an education/ training program (cost/benefit analysis).

• Program met the original objective set by administration and participants.
• Program did not have any unintended consequences.
• Learners are able to transfer the information obtained to another setting.
• Be aware of the audience for which the program data will be shared and the
population from which it will be collected.
• Secure support of the stakeholders.
• Identify the person to plan evaluation.
• Define the purpose of the evaluation and the use of the results.
• Specify what is judged and formulate evaluation questions.
• Determine who will supply the needed evidence and/or data.

15 16

4
 14/02/2023

Elements of a Systematic Evaluation Models of program evaluation

• Choose the evaluation approach. CIPP Model

• Determine timeline, budget, needed resources.
• Choose data collection and use plan followed by analysis.
• Monitor and complete the evaluation and make judgments. Kirkpatrick’s model

Logic model

17 18

19 20

5

21
CIPP model
3)Process evaluation
• monitors the strategy adopted
to implement the program and
gets constant feedback and also
identifies the defects in the
program.
23
14/02/2023
CIPP model
1 ) Context Evaluation 2 ) Input Evaluation
• the resources and • Input evaluation mainly
background within the evaluates stakeholders,
program, cultural context resources and policies to
• Alignment of program implement goals.
objectives with the need
for program
22
When to Collect Evaluation Data
Prior to the
4) Product Evaluation Program
• Finally, the product evaluation
looks into the final product, i.e.
whether the curriculum has
accomplished the desired
objectives. Product evaluation
provides data to match with the
mission and goal of the After the During the
program. Program Program
24
6
 14/02/2023

Collecting Data How to Collect Data

Types of •Quantitative Written Product

Interviews/foc
Data •Qualitative Observations
us group
Questionnaires
/Surveys
Reviews/Portfo
lios

Sources of •Single Source Review of

Records
Cost-benefit
Analysis
Self-
Assessment
Data •Multiple Sources

25 26

CIPP MODEL

27 28

7
 14/02/2023

Sequence these events

• Make a time- line
• Design Evaluation Plan What data do you need to collect? How will you
collect the data? What resources do you have available?
• Data Collection ..Methods : 1) Quantitative ,Pre/Post Test, Survey,
2)Qualitative, Focus Groups , Interviews
• What is an objective? Specific, measurable change the program is
designed to accomplish
• Report the findings
• Analyze Data Steps: 1) Prepare the data for analysis 2) Analyze the data 3)
Interpret the results
• What is a goal? General statement of what you want to accomplish

29 30

Thank you

31