Professional Documents
Culture Documents
For
April 2022
Session overview and timeline
10:00-12:00 2 hours M&E for Health Insurance system Facilitator presentations; Group
discussions
2
Methods Remarks
3
Training Methodology
•The sessions of this five-day training workshop are designed based on the principles of adult learning
and participatory approaches. Methodologies include lectures accompanied by PowerPoint slides, group
discussions, group work, activities and presentations, brainstorming and mapping exercises and games
for session linkages.
•
•Module 1 Introduction to Monitoring and Evaluation (M&E In General
• At the end of this training participants will demonstrate capability in:
4
Session one
•Materials
• Slips of paper, 1 for each participant, each with a different M&E term written on it
• Copy of training workshop schedule
• LCD Projector
• PowerPoint presentation
5
• Facilitator Note: This session is the “icebreaker.” During this time participants and
facilitator(s) will introduce themselves and their relative experience, current position
and organization.
• In addition to “getting to know you,” the main purpose of this introductory activity is
for participants to become familiar with monitoring and evaluation & complaints &
Grievance handling related terms, as well as for the facilitator(s) to get a brief
assessment of participants’ understanding of these terms.
6
IntroductoryActivity
•Introductory Activity
•Welcome participants and invite them to take a seat any where they’d like; as you are welcoming
them, provide each participant with a slip of paper with an M&E related term on it.
•
•Once all participants have entered and taken their seat say, “Welcome to this monitoring and
evaluation training workshop. As you can see, I have given each of you a piece a paper with a word
written on it; I also have one myself. We are going to go around the room and I would like each
person to introduce his/herself, his/her job title and the organization which you are
representing. After you have finished, I would like you to define the word on your piece of paper, to
the best of your ability, and describe how this word relates to monitoring and evaluation. I’ll begin.
My name is (name). I am the (job title) for (organization). My term is (read term allowed). This word
means (define) and relates to M&E because (explain).”
7
Introduction
• The Ethiopian Health Insurance Service (EHIS), the former Health Insurance Agency,
has been implementing a community-based health insurance program and currently
embarking on rolling out social health insurance program. As of 2020/21, the CBHI
program was able to cover more than 800 woredas and provide financial protection for
healthcare to more than 40 million beneficiaries.
• As part of its Ten-Year Strategic Plan, the EHIS is embarking to achieve universal
health insurance coverage in 2030. Strengthening the monitoring and evaluation
system is one of the core initiatives of the EHIS for the successful implementation
of planned activities in the ten-years period.
8
Continue….
• Once this activity is complete, ask participants to try and remember their word, how
they have defined it and linked it to monitoring and evaluation; request that they take
note of how their ideas about this word and its relation to monitoring and evaluation
may or may not change throughout the course of this session
9
Methodology
• This training is aiming to capacitate staff from in the health insurance system on
M&E and complaints management system. Therefore, the training will be delivered
in different phases.
• In this training will be participants from RHBs, zone, werda, contracted health
facility and EHIS branches,. Then the other phases will cascade comprise of
training participants from woredas, to keble
• These cascaded trainings will be managed by those who took TOT and will be
mentored by staff from EHIS head quarter.
10
Training overview
•Training Overview
•Briefly orient participants on the sessions and schedule of this training workshop. If
you have made copies of the schedule, distribute them to participants. In position of
individual copies, paste a copy of the training schedule in a visible part of the
training room.
•
•Training Goals and Objectives
•Introduce the goal and training objectives of this workshop.
• Facilitator Note: Goals and objectives should be exhibited in a PowerPoint presentation or a large flip
chart
11
Continue….
• The goal of this training workshop is to orient and sensitize EHIS Project partners
and implementing stakeholders on the fundamental aspects, systems and processes
of monitoring and evaluation for Monitoring and Evaluation
•The objectives of this monitoring and evaluation training workshop are as follows:
(1) To highlight the importance and use of M&E for development
(2) To introduce the national M&E plan,
(3) Provide conceptual clarity for monitoring and evaluation
(4) To introduce the framework for monitoring and evaluation of EHIS interventions
(5) To introduce indicators and share available strategic information for EHIS
(6) To sensitize on data quality and strengthen project level M&E functions
(7) To generate commitments and an action plan for M&E
12
Introduction ……
13
Continue…..
• Facilitator Note: Ask participants to keep the objectives in mind throughout the sessions. Do they feel the
objectives are being met?
•Session Conclusion
•Wrap up this session by saying, “So we are all here to receive training on monitoring
and evaluation, but why do we need monitoring and evaluation? What purpose does it
serve?”
14
Introduction…
15
Module one session two
Why M&E?
•Materials
• LCD Projector
• PowerPoint presentation
•Group Discussion
• Begin this session by asking participants, “What did you do when you got up this morning
16
Continue…..
• Facilitators Note: The purpose of this exercise is to relate monitoring and evaluation to a
mirror; monitoring and evaluating is the “mirror” of our training .
• If no one mentions looking in a mirror, ask leading questions, such as, “What did you after
you got dressed?” or “Did you shave/put on make-up this morning?
• If yes, did you do so while looking in a mirror?” Elaborate on this point by asking, “How
would it be to shave/put on make-up without looking in the mirror?
• What is the difference between shaving/putting on make-up with and without looking in the
mirror?” Relate this to monitoring and evaluation; with M&E we can reflect on our project
and its process, just as we use a mirror to see our own reflection when we shave/put on make-
up – it allows us to see what we’re doing and reduces the chance of error.
17
Continue …..
•Facilitator Presentation
•Exhibit PowerPoint presentation. The following topics should be included in this
presentation:
(1)M&E as a tool to address emerging challenges
(2)M&E as a National priority for Health Insurance System
(3)M&E as an element in the process of managing for impact
• This presentation is intended to illustrate how M&E contributes to EHIS programs on
both the Regional and national scale.
•Session Conclusion
• Conclude this session by discussing how M&E functions to mitigate impact.
18
Monitoring definition
•Monitoring Definition
• Monitoring is the routine process of data collection and measurement of progress toward program objectives.
• It is a continuous form of reflection involving examining what we are doing and routinely looking at the
quality of our services.
• Monitoring is “an internal process undertaken by management carried out to assess progress at regular
intervals throughout the life of a project or program.”
• Monitoring cycle
•Monitoring is a four-step cyclical process where each step works to inform the other.
• Firstly, a monitoring plan must be prepared; this means creating a system for monitoring the progress of your
project.
• The next step is to collect and analyze data.
•
•For example, if your organization was running an health service utilization, a way to monitor its success
would be to collect and analyze data regarding how many beneficiary are utilize health service in health
facility ?
19
Mentoring cycle continue….
•If your monitoring plan has been successful, you should be able to answer the following
questions:
(1)Are we on the way to our planned objective(s)?
(2)To what extent are planned activities being implemented?
(3)Are project activities being carried out correctly, on time and within budget?
(4)How well are services being provided?
(5)What services are we providing to whom, when, how often, for how long and in what
context?
(6)Are the objectives and targets reasonable?
• Do we need to take corrective action?
• Facilitator Note: It is recommended that you display a few Examples at this juncture to
reiterate certain points about monitoring
20
Evaluation Definition
•Evaluation Definition
• Evaluation is concerned with the relevance, effectiveness, efficiency, impact and sustainability of what has been done.
• It is the process of identifying and reflecting on the effects of a process, thus far, and judging the value of it.
• It utilizes social research methods involving measurements over time, study design and special studies, to systematically
investigate a program’s effectiveness.
• Sometimes evaluation requires a control or comparison group
21
Deafferenting b/n Monitoring and Evaluation
• Concerned with efficiency of resource deployment • Concerned with effectiveness
• Absolutist view (“are we doing what we said we would?”) • Philosophical view (“is what we’re doing actually worth doing?”)
22
Session Conclusion
•Session Conclusion
•Conclude this session by informing participants that monitoring and evaluation systems must be in place in
order for organizations to demonstrate accountability for the past, enhance management decision-
making in the present and to promote organizational learning for the future.
•
•Without effective monitoring and evaluation systems in place there would be limited learning from both
successes and failures resulting in lost opportunities to know how to handle a mistake the next time it occurs or
benefit from effective approaches or successes.
•
•The recent history of much development and aid work is that there has been limited institutional learning
despite huge efforts and expenditures. There are too many examples of old mistakes being repeated despite
serious attempts at quality planning!!
•
•By monitoring and evaluating our programs we reduce the risk of program duplication and error, and
increase accountability.
23
Monitoring and Evaluation function
24
M&E function continue…..
•In this sense M&E facilitates action. Learning and feedbacks generated by critical reflection are applied to programs
and projects to improve actions.
•
•This action, in turn, influences day to day management and decision making. The conclusions drawn by monitoring and
evaluation procedures determine processes of leading, planning and organizing staff as well as controlling activities,
people and other resources in order to achieve particular objectives.
26
EHIS Indicators and selection criteria
10:00-11:00 1 hour Key EHIS Indicators and definition Facilitator presentation
27
National core indicator
28
EHIS Indicator….
•Materials
•LCD Projector
•PowerPoint Presentation
• Facilitator Note: It is important to keep the atmosphere exciting and keep participant’s
energy high. The next two sessions will have your participants listening to you lecture
and present for 2 hours! Remember to talk to them, not at them. Engage them;
encourage questions and participation
•Exhibit the PowerPoint presentation on the Fundamentals of M&E; it should cover the following points:
¾ Defining indicator selection criteria
¾ Overview of the 27 Core National Indicators
29
EHIS Indicator….Continue
30
EHIS Indicator….Continue
• Indicator Definition
31
EHIS Logical frame work
32
CONTINUE….
33
Continue…
• Process indicators measure the activities which are being conducted to achieve the outputs and
outcomes
•i.e., number of training workshops conducted or number of site visits.
•
• Output indicators are used to measure the product of the immediate inputs of project activities; they
describe the goods and services produced by program activities.
•i.e., number of staff members trained or the number of IEC materials distributed
• Outcome indicators are used to measure the effect of an intervention on systems or behaviors
resulting from the achievement of an immediate goal after a specified interval of time; commonly
for short-term assessment, such as one or two years after a program or project has been
implemented.
•i.e., the number of clinics which meet new quality standards
•
• Impact indicators measure the actual change in conditions of key problems or unmet needs which
are linked to the ultimate program goal; the impacts which program/project activities have had on a
population over a longer period of time.
34
Measuring indicator
•Measuring Indicators
•Explain how to create a numeric indicator, i.e., a percentage. Explain that you must first have a numerator and a
denominator.
•
•For example
•Facilitator Note: Demonstrate this on a blackboard/flipchart. Create one or two examples to ensure participants
understand the process of generating a numeric indicator.
•
•Remind participants that when measuring indicators, it is important to consider where and how the information
will be obtained; these two factors can greatly influence data.
•Developing Indicators
•Indicators and the information utilized to realize them must be attached to each level of the program planning
hierarchy; impact, goal, objectives, intermediate results, activities (inputs, outputs, processes) and outcomes.
•
• When developing indicators, the five core evaluation questions must be answered: 1) Relevance 2)
Efficiency
3) Effectiveness
4) Impact
5) Sustainability
35
Measuring indicator continue……
•The information system should be varied in order to capture cross-cutting issues and to incorporate views of
stakeholders and beneficiaries; this contributes to participation and transparency.
• Facilitator Note: Emphasize the importance of taking note of both unintentional positive and negative
outcomes and impacts of the program to participants
•Group Exercise
• This exercise is intended for groups of two. Once everyone has paired off, indicator checklists should be
distributed
•Request that each pair work together to identify the appropriate hierarchical level of the listed indicators. Twenty
minutes will be provided for participants to complete this activity.
•Facilitator Note: This activity has been designed to assess participants’ knowledge regarding indicators
•
•In some circumstances indicators may be applicable to more than one level; inform participants of this so they are
aware that they can check more than one level.
•
•When the allocated time has expired, review the indicators and their levels with participants. Explain why each
indicator is located in its particular position.
•
•Allow time for questions. 36
DATA Quality Improvement
37
Data Quality
•Materials
•LCD Projector
•PowerPoint presentation
•White Board
•Markers
• Facilitator Presentation
•Exhibit PowerPoint presentation on data quality. The presentation should cover the following main
points:
¾ Define data and data quality
¾ Criterion for measuring data quality
¾ Criterion for establishing good data
• Steps for verifying data
• Emphasize that data quality is a continuous commitment! In order to maintain data quality there
needs to be training and re-training on reporting systems and indicator definitions for new staff
members. The data needs to be used to improve actions and narrative documentation must be kept
to support quantitative accomplishments
38
Data definition
•Data Definition
• Factual information used as a basis of reasoning, discussion or calculation
• Must be recorded; if it is not written down, it is not data
•
•Data Quality Definition
• How well the information collected represents the program activities
• Refers to the worth/accuracy of the information collected
• Data that reflects true performance
• Focuses on ensuring that data management process is of a high standard
•
•Measuring Quality Data
•The data is only as good as its quality. The quality of data is measured against six criteria:
1) Validity – data which clearly, directly and adequately represents the result that was intended to be
measured
39
Data quality continue…..
1) Reliability – if the process were repeated over and over again it would yield the same result
2) Integrity – data have been protected from deliberate bias or manipulation for political/personal reasons
3) Precision – data are accurate with sufficient detail
4) Timeliness – data are current and information is available on time
5) Confidentiality – clients are assured that their data will be maintained according to national and/or international standards for data
•
•Good quality data can be recognized by checking the FACTS
•F – Formatted Properly
•A – Accurate
•C – Complete
•T – Timely
•S – Segmented properly
40
Continue…
•Facilitator Presentation
•How is it that we can verify data? With RATS R – Raw data (i.e., forms)
• A – Aggregated data (i.e., tables)
• T – Trends (i.e., consistent changes over time)
• S – Segmented data (i.e., split by sex, age, etc.)
•Following the conclusion of this module, participants should have a working understanding of:
9 Quality data
9 The components of good data
9 The actions required for quality data improvement
9 Verifying data
Facilitator Note: After conducting a brief Q&A with participants regarding the day’s sessions, select one group to
prepare a brief presentation on the sessions covered during ppt presentation to be presented on the morning.
Inform the group who will be undertaking this task that the presentation should be a maximum of 10 minutes.
41
Material- Facilitator
•Materials – Facilitator
• LCD Projector
• Previously prepared PowerPoint presentations
•
• Markers
• Pens/pencils
• Handout: Slips of paper, one for every participant, each with one monitoring and
evaluation related word written on it
• Handout:
• Handout: National Indicator Checklist, EHIS Reportioning and Recording tools
•
42
Martial- participants
• Pens/pencils
• Hand out
• M&E, Compliant and Gravedance handling manual
• Laptop
43
• Data Ana1/211111½ Day allowed
• lysis
•8:30-10:00
•
Facilitator Presentation (Reporting flow presentation
1 ½ hours
• ½ hr
•Discussion on reporting flow
•Materials – Facilitator
• LCD Projector
• Previously prepared PowerPoint presentations
•
• Markers
• Pens/pencils
• Handout: Slips of paper, one for every participant, each with one monitoring and evaluation related
word written on it
• Handout:
• Handout: , EHIS Reportioning flow hand out
•Data Quality and Feedback Systems
•Facilitator presentation, Group Activity
•8:30-10:00
•1 ½ hours 8:30-10:00
•1 ½ hours
•Data Quality and Feedback Systems
•Facilitator presentation, Group Activity
•
Facilitator Presentation 1 day
hours
46
Reportting and Recording tools
47
Monthly Kebele CBHI Section Activity Summary Report
In the reporting period
CBHI Membership
Male Female Total
Paying
Number of CBHI members newly enrolled Non - paying
Total
Paying
Family members newly registered (including HH
Non - paying
heads)
Total
Paying
Number of CBHI members renew their
Non - paying
membership
Total
Paying
Total family members with membership renewal
Non - paying
(including HH heads)
Total
Paying
ID provided out of total registered members Non - paying
Total
CBHI Contribution Collection In the reporting period
Expected money to be collected from registered members
Contribution from paying member (actually collected)
Amount deposited to Bank
# of receipts unreturned from previous years
Receipt voucher-RV # of receipts received this year
# of receipts returned in this month
Quarterly Woreda Scheme Health Insurance Activity Summary Report
Region:_____________________________________
Reporting
Name of Zone/Sub-City:__________________________________ Period:___________________________
Name of Woreda/Scheme:_________________________ Submission date:___________________
Total # of staff at the beginning of the fiscal year (report only at the beginning of the year)
Number of new staff recruited
Total # of staff resigned
Total # of staff at the end of the fiscal year (report only at the end of the year)
Quarterly EHIA Branch Health Insurance Activity Summary Report
Reporting
Region:_____________________________________ Period:_______________________________
Newly
Existing/previously included in
Total
reported the reporting
Reporting EHIA Branch Profile period Remark
# of CBHI Woreda Schemes assessed for data quality improvement by EHIA Branch
Agenda
Participants
64
TH A N K YO U !