Professional Documents
Culture Documents
METHODS, PROCESS
MIGOM
PARNAVA
KARTIKEY
1
Presentation Outline
What is Monitoring & Evaluation
Guiding principles
Who to involve
• EVALUATION
• A systematic process to determine the extent to which service needs and
results have been or are being achieved and analyse the reasons for any
discrepancy
3
Attribute Monitoring Evaluation
Main focus Collecting data on progress Assessing data at critical stages of
the process
Sense of completion Sense of progress Sense of achievement
Time focus Present Past-Future
Main question What is happening now to Have we achieved our goal
reach our goal
Attention level Details Big picture
Inspires Motivation Creativity
Periodicity Continuous throughout the Intermittent; at the beginning or end
whole process of significant milestones
Supports Implementation of a plan Designing the next planning cycle
5
EVALUATION
OUTCOME
• What was • How was
planned? • What was it
achieved? achieved?
STRATEGY FUNCTIONING
6
Effectiveness Efficiency
Impact
Monitoring
+
Evaluation 7
MONITORING AND EVALUATION (M&E)
• Key functions:
8
NEED OR PURPOSE • Whether heading in right
direction
Problem • Identify problems in planning
and implementation
identification
and cause
analysis
Developing
implementation
plan
9
GUIDING PRINCIPLES
• Focused and feasible
• Timely
• Useable
• Sensitive
• Ethical
10
WHO NEEDS AND USES M&E INFORMATION?
To improve
programme • Managers
implementation
• Donors
To inform and improve
• Governments
future programmes
• Technocrats
• Donors
• Governments
Inform stakeholders
• Communities
• Beneficiaries
11
WHO CONDUCTS M & E?
• Program implementer
• Stakeholders
• Beneficiary
12
WHAT TO MONITOR?
15
M & E QUESTIONS
• What is being done?
• By whom?
• Target population?
• When?
• How much?
• How often?
• Additional outputs?
• Resources used?
16
MONITORING AT DIFFERENT LEVELS
2. Middle level: concerned with getting the desired output with the
inputs utilized. Need to exercise supervision, provide support and
take timely corrective action
• Programs
• Services
• Processes
• Conditions
18
ASPECTS OF PROGRAMME
• Inputs
• Processes
• Outputs
• Outcomes
• Impacts
19
Input Outputs Outcomes
Components Activities Target Short-term Long-term
(What was (What was groups (Learning and (Ultimate
invested) done) (Who was action) impact)
reached)
Staff Workshops Participants Awareness Improved
Volunteers Meetings Patient Knowledge health outcome
Time Counselling Clients Attitude Social
Money Facilitation Citizens Skills Economic
Materials Assessments Motivations Environmental
Equipment Training Action Disease
Recruitment Behaviour prevalence
Decisions
C A T SO LO
20
REVISED NATIONAL TUBERCULOSIS CONTROL PROGRAM
Availability of trained
faculty
Availability of funds
in IEC/training heads
INPUT
Availability of printed
INDICATORS material for
handouts, venue
21
REVISED NATIONAL TUBERCULOSIS CONTROL PROGRAM
Trainings conducted
22
REVISED NATIONAL TUBERCULOSIS CONTROL PROGRAM
Number of smear
positive PTB
diagnosed
Number of
OUTPUT registered TB
INDICATORS patients with
non-HIV status
23
REVISED NATIONAL TUBERCULOSIS CONTROL PROGRAM
Increase in case
notification
rates
OUTCOME Improvement in
INDICATORS treatment
success rates
24
REVISED NATIONAL TUBERCULOSIS CONTROL PROGRAM
Reduction in TB
prevalence
Reduction in TB
incidence rates
IMPACT
Reduced number of
INDICATORS deaths
25
TYPES OF EVALUATION
• Retrospective Evaluation:
• when programs have been functioning for some time.
• Prospective Evaluations:
• when a new program within a service is being introduced
26
Retrospective Evaluation, determines what actually
happened (and why)
• Summative evaluation
Evaluation of the degree to which a program has achieved its desired
outcomes, and the degree to which any other outcomes (positive or
negative) have resulted from the program.
28
WHO CONDUCTS EVALUATION?
29
IE Methodology
Selection of Districts
Upto 30 million- 2 districts per 30 million- 100 million- 3 >100 million- 3-4 districts per
quarter districts per quarter quarter
Selection of patients
32
GUIDELINES FOR EVALUATION (FIVE PHASES)
D: Reporting Findings
33
PHASE A: PLANNING THE EVALUATION
• Determine the purpose of the evaluation.
Program Staff:
• Does this program provide our Funding bodies:
clients with high quality service? • Does this program provide its clients
• Should staff make any changes in with high quality service?
how they perform their work, as • Is the program cost-effective?
individuals and as a team, to • Should they make changes in how
improve program processes and they fund this program or in the level
outcomes? of funding to the program?
37
Evaluation Area Evaluation Question Examples of Specific
(Formative Measurable Indicators
assessment )
Staff Supply Is staff supply sufficient? Staff-to-client ratios
39
PHASE C: COLLECTING AND ANALYSING
INFORMATION
• Develop data collection instruments
• Analyse data
40
Pretesting or piloting
41
GATHERING OF QUALITATIVE AND QUANTITATIVE
INFORMATION: INSTRUMENTS
42
Quantitative tools:
• Surveys/questionnaires
• Registries
• Activity logs
• Administrative records
• Patient/client charts
• Registration forms
• Case studies
• Attendance sheets
43
Other monitoring and evaluation methods:
44
Impact flow diagram (cause-effect diagram)
45
PHASE D: REPORTING FINDINGS
46
• Checklist: e.g.
• Resources:
• Is at least one trained MO officer available in the health facility?
• Is a full time trained Lab Technician (LT) available for sputum
microscopy?
• Have provisions been made for sputum collection when LT is absent?
49
RNTCP supervisory register
Recommendation
Name of the Health facility visited
Name and designation of supervisor filling this form
Date and time
Observations on actions taken based on previous visit
Key observation
Politico administrative commitment and resource management
Diagnosis
Drugs and laboratory consumable
DOT and Follow up
TB-HIV activities
Records and reports
ACSM activities
DOTS Plus
Finding of home visit
50
PHASE E: IMPLEMENTING EVALUATION
RECOMMENDATIONS
• Develop a new/revised implementation plan in partnership with
stakeholders.
51
National Health Mission
53
MISSION STEERING GROUP
• Highest policy making and steering institution under NHM.
• Provides broad policy direction to the mission.
• Advises the Empowered programme committee of the mission in
policies and operation
• Exercises the main programme and governance for Health
sector
• Chairperson- Union minister of Health & Family welfare.
• Fully empowered to approve financial norms in respect of all
schemes and components which are part of NHM.
54
COMMON REVIEW MISSION
• It was setup as a part of the Mission steering group’s mandate
of review and concurrent evaluation.
• Appraisal was conducted in November 2007.
• The task of the NRHM CRM was to assess the progress of the
NRHM on 24 parameters.
• 52 members
• Ninth CRM was held from 30th October – 6th Novemebr 2015
55
Suggested outline of community based monitoring
activity
• The Monitoring committee at each respective level reviews and
collates the records coming from all the committees dealing
with units immediately below it
• Also appoints a small sub-team drawn from its NGO and PRI
representatives who visit a small sample of units under their
purview and review the conditions there
56
• The Monitoring committee sends a periodic report (Quarterly for Village,
PHC, Block and District levels; Six monthly for State level) to the next
higher level committee
57
Level Agency Activity (quaterly in all except state which is 6monthly)
Village Village health and •Reviews village health register, village health calender
sanitation committee •Reviews performance of ANM,MPW,ASHA
•Sends brief 3 monthly report to PHC committee
PHC PHC monitoring and •Reviews and collates reports with all VHSC
Planning committee •NGO/PRI subteam conduct FGD in 3 sample village under PHC
•Visit PHC review records discuss with RKS members
•Sends brief 3monthly report to block committee
Block(CH Block monitoring and •Reviews and collates reports with all PHC
C) planning committee •NGO/PRI subteam conduct FGD in one PHC ,interview MO
•Visit CHC review records discuss with RKS members
•Sends brief 3 monthly report to District committee
District District monitoring •Reviews and collates reports with all PHC
and planning •NGO/PRI subteam conduct FGD in one PHC ,interview MO
committee •Visit CHC review records discuss with RKS members
•Sends brief 3 monthly report to District committee
PHC •Overview of village •PHC health •PHC monitoring Quarterl •Standard agenda
level monitoring plan and planning y items
•Staffing, supplies and •Charter of committee, PRI •Report from VHSC
services availability at citizen’s members etc. •Record of select
PHC health rights •PHC RKS village FGD
•Quality of care at PHC at PHC members •Interview of MO PHC
•Exit interview of PHC
patients
59
Levels Main issues for monitoring Reference Who When Tools
documents
Block •Overview of health service in •CHC health •CHC Quaterly •Standard agenda
block plan monitoring items for CHC
•Staffing, supplies and •Charter of and planning committee meeting
service availability at CHC citizen’s committee •Reports from PHC
•Quality of care at CHC from health rights incl PRI committee
people’s perspective at CHC members etc. •Records of visits to
•CHC RKS select PHCs
members •Interview of MO in
•Facilitation charge of CHC
by nodal •Report of the
NGO/CBO district health
mission
District •Overview of all public health •District •District 6 monthly •Standard agenda
services in the district(except health plan health items for District
services provided by •Charter of monitoring committee meeting
municipal bodies) state citizen’s and planning •Reports from Block
specific health schemes health rights committee health committee
•Quality of care at district at District •Public •Records of visits to
hospital and sub-divisional hearing select sub-divisional
hospital facilitator hospitals/CHCs
team 60
Levels Main issues Reference documents Who When Tools
for monitoring
State All issues of •State health plan, state •State health •Six monthly •Report from
rural public PIP monitoring and committee district health
health •NHRC planning meetings committee
services/NHM recommendations and committee •Annual •Record of visit
in the satte state govt component of •State peoples independent to select
including state NHRC national action rural health reports,public districts
specific health plan watch meetings •Report of state
schemes •All NHM schemes report/citizens health mission
ASHA,JSY, United report by civil •Report of
funds expenditure society groups district public
•IPHS and functioning •Public meeting hearing
of various level facilities of state mission •Independent
•National health with civil society reports
programmes and family representative
planning insurance
scheme
•PPP and related
regulations
•State health budget
and expenditure
61
CONCLUSION:
WHY MONITORING AND EVALUATION
62
63
Thank you
64