ROUTINE HEALTH INFORMATION SYSTEMS

A Curriculum on Basic Concepts and Practice

MODULE 2:
Indicators and Data Collection and Reporting

SESSION 1:
Indicators
The complete RHIS curriculum is available here:
https://www.measureevaluation.org/our-work/ routine-health-
information-systems/rhis-curriculum
SESSION 1 : Learning Objectives

Participants will be able to:

•Define and identify relevant health indicators
•Mention 5 characteristics of good indicators
•Identify factors to consider while selecting indicators
•Give examples of good indicators
What Are Health Indicators?

With the goal of good health in mind,

think of an indicator as
“…a measure that helps quantify the achievement o
a goal.”
―Mark Friedman
An Indicator Is …….

• a variable (its value changes)…

• that measures (objective calculation of value)…

• key elements of a health intervention (program, service, or

project)
o (inputs/processes, outputs, outcomes, impact)
 Types of Health Indicators

INPUTS AND OUTPUTS OUTCOMES IMPACT

PROCESSES

• Availability of • Number of • Full • Life expectancy at

drugs per care children immunization birth
level immunized rate • Infant mortality
• Basic equipment • Number of interventions • Prevalence of
per activity type new cases in • Incidence and malnourished
• Number of staff the facilities prevalence of children under 5
per category and • Hospital bed certain diseases,
per capita occupation such as HIV,
• Number of drug rate malaria, TB
shortages • Number of TB
• Number of cases detected
surgical or
obstetrical acts
done

Source: Adapted from WHO. Global reference list of 100 core health indicators. Retrieved from http://www.who.int/healthinfo/indicators/2015/en/.
 EXAMPLE: Indicators for Reproductive,
Maternal, and Child Health Monitoring
INPUTS AND
PROCESSES
• Health financing
• expenditure per target
population (children,
women, etc.)
• General government
• expenditure on health as a
percentage of general
government expenditure
• Health workforce
• midwives, per 10 000
population
• Governance
• presence of key policies
to promote maternal and
child health
• Information
• births registered
• deaths registered (with
cause)
Source: Adapted from Monitoring, evaluation and review of national health strategies: a country-led platform for information and accountability. Geneva, World Health
Organization, 2011.
OUTPUTS
• Service access and
readiness
• facilities that offer
and meet tracer
criteria for basic and
comprehensive
obstetric care, per 10
000 pregnant women
• caesarean section rate
in rural populations
• facilities that offer
and meet tracer
criteria for child
health services, per
1000 children
OUTCOMES IMPACT
• Coverage of interventions • Health status
• antenatal care
• births attended by skilled health personnel • under-5 mortality
• immunization coverage • maternal mortality
• family planning needs satisfied ratio
• child mortality by
• children with diarrhea receiving oral rehydration
therapy major cause of death,
by sex and age
• children with fever receiving antimalarials
• insecticide-treated bednet use
• antiretroviral prophylaxis among HIV-positive • Financial risk
pregnant women protection
• vitamin A supplementation among children • out-of-pocket
payments as a
• postnatal care
percentage of total
health expenditure
• Risk factors and behaviours
• contraceptive prevalence
• access to safe water
• access to improved sanitation
• low birth weight among newborns
• early initiation of breastfeeding
• children who are stunted or underweight
 Indicator Domains

• Health status
• Risk factors
• Service coverage
• Health systems

Source: Adapted from WHO. Global reference list of 100 core health indicators Retrieved from
http://www.who.int/healthinfo/facility_information_systems/en /
What Is the Rationale for Health Indicators?

Indicators are vital in health interventions because,

when collected and used regularly, they can:
•Provide a reference point for health intervention planning,
management, and reporting

•Allow managers of health interventions to assess trends and

identify problems

•Act as early warning signals for corrective action

What Makes an Indicator “SMART” ?
(Characteristics of Good Indicators)

• Specific: Indicator is concrete, detailed, focused, and well

defined
• Measurable: Indicator tells how many or how much and can
be measured with identified measurement sources
• Agreed upon: Stakeholders vested in a specific M&E question
should agree that the indicator is relevant
• Relevant: Indicator generates data that can answer the question
of interest
• Timebound: Indicator specifies time frame of what it is
measuring
Characteristics: Relevant

• Answers the question of interest: too many indicators

have been defined without being based upon a specific
information need
• Linked to a public-health impact or to achieving the
objectives needed for impact
Example: Childhood vaccines program
Indicator: % of infants receiving measles vaccine (coverage)
Example: Program to increase access to ORS for childhood
diarrhea through community-based distributors (CBDs)
Indicator: # of ORS packets distributed in past month by CBDs
Characteristics: Specific

Or “valid” = measures a specific disease, service

provided, practice, or task
• Indicator measures what it is intended to measure:
provides direct information about the result it
intends to measure
• If indicator not “feasible,” sometimes need to use
“proxy indicator”
Specificity (or Validity): Class Activity

• Is the number of antenatal care (ANC) visits a valid

indicator of a focused ANC package?
• Is the maternal mortality ratio a valid indicator of the
impact of a family planning program on women’s
health?
• Is the number of children who received the DTP3
vaccine a valid indicator of immunization coverage?
Characteristics: Measurable

• Quantifiable: using available tools and methods

• Precise: operationally defined in clear terms and
documented in an Indicator Reference Sheet
• Reliable: consistently measurable in the same way by
different observers
• Feasible: the resources (human, physical, financial)
needed are available
Indicator Reference Sheet: Minimum Information

• Description
• Plan for data collection
• Plan for data analysis
• Plan for data quality check
• Performance table
Characteristics: Time Bound

Provides a measurement over periods of

interest, with data available for all
appropriate intervals

Timeliness considerations:
• Reporting schedules
• Recall periods
• Length of time over which change can be detected
Characteristics: Agreed Upon

• Stakeholders vested in a specific M&E question

should agree that the indicator is relevant
• Agreement between various health-system levels
• Agreement between various national health programs
and health services managers
Common Indicator Metrics

Counts
# of providers trained
# of condoms distributed
Calculations: percentages, rates, ratios
% of facilities with trained provider
Index, composite measures
Index on infection control and prevention
DALY (disability-adjusted life years)
Thresholds
Presence, absence
Predetermined level or standard
Factors to Consider When Selecting Indicators

 Logic/link to framework
 Programmatic needs/information for
decision making
 Resources
 External requirements (government, donor,
headquarters)
 Data availability
 Standardized indicators
 Alignment with national and international standards
Common Pitfalls in Indicator Selection

• Indicators not linked to program activities

• Poorly defined indicators
• Indicators that cannot realistically be collected
• Process indicators to measure outcomes and impacts
• Indicators that are insensitive to change
• Too many indicators
Pitfalls in Selecting Indicators

• Indicators not linked to program activities

• Intermediate result (IR): Expanded access to malaria treatment

services

• Activities: Train providers in current clinical protocols

• Inappropriate indicator: % of facilities with adequate

conditions to provide care

• Better indicators: # of clinicians trained; % of facilities with

a trained provider

• The program is not aiming to affect facility conditions, only

provider skills.
Pitfalls in Selecting Indicators

• Data needed for indicator not available

• Inappropriate indicator: % of days per quarter that service
delivery points have stockout of drugs

• Data issue: Information on stockouts may not be

collected daily

• Better indicator: % of service delivery points that had a stockout

of drugs at some time during the past quarter

• If relying on routine data, an indicator definition must depend

on how data are collected (see Session 2) in this module.
Pitfalls in Selecting Indicators

Indicator does not accurately represent desired outcome

• IR: Access to effective treatment among children <5 years old with malaria

• Inappropriate indicators: % of children <5 years old who received

artemisinin-based combination therapies (ACTs); % of people who received
ACTs for malaria infection who are children <5

• Better indicator: % of children <5 years old who were diagnosed with
malaria in the past 2 weeks who received ACTs

• What does it mean if inappropriate indicators increase? Decrease? Do they reflect

the desired program effect?
Indicator Systems: How Much Is Enough?

Rule of Thumb
•At least one or two indicators per key activity or result (ideally, from
different data sources)
•At least one indicator for every core activity (such as distribution of
insecticide-treated nets, indoor residual spraying, training, behavior
change communication)
•No more than 8–10 indicators per area of significant program focus
•A mix of data collection strategies/sources
Choosing the Right Number of Indicators
Good Indicators

• Provide information useful for program decision making

• Are consistent with international standards and other reporting
requirements, as appropriate
• Are defined in clear and unambiguous terms
• Are nondirectional; “independent”
• Have values that are:
o Easy to interpret and explain
o Precise, valid, and reliable measures
o Comparable across relevant population groups, geography, and other
program factors, as needed
Change in Indicators over Time
Activity: Handout 2.1.1

1. Share Handout 2.1.1 on change in indicators

over time.

2. Participants read the text individually

3. Discuss in the large group.

“Not everything that can be counted counts,
and not everything that counts can be
counted.”

― Albert Einstein
 Summary: Guiding Principles for Selecting
Indicators

• Ensure that the indicators are linked to the information

needs for health interventions and are able to measure
change.
• Ensure that standard indicators are used to the extent
possible.
• Consider the cost and feasibility of data collection and
analysis.
• Keep the number of indicators to the minimum that are
necessary, and include only those needed for program
and management decisions or for reporting.
Q&A
 ROUTINE HEALTH INFORMATION SYSTEMS
A Curriculum on Basic Concepts and Practice

This presentation was produced with the support of the United States Agency for International Development
(USAID) under the terms of MEASURE Evaluation cooperative agreement AID-OAA-L-14-00004.
MEASURE Evaluation is implemented by the Carolina Population Center, University of North Carolina at
Chapel Hill in partnership with ICF International; John Snow, Inc.; Management Sciences for Health;
Palladium; and Tulane University. The views expressed in this presentation do not necessarily reflect the
views of USAID or the United States government.