3 M E Toolkit 2004

MONITORING AND EVALUATION TOOLKIT
HIV/AIDS, TUBERCULOSIS AND MALARIA
DRAFT 14/01/2004
World Health Organization
Table of contents
WHY THIS KIT?.......................................................................................................................................3 WHO IS IT FOR?......................................................................................................................................3 WHAT ARE ITS CONTENTS? ..............................................................................................................3 BASIC ELEMENTS OF M&E ................................................................................................................4 BOX 1: CHECKLIST OF FEATURES OF A GOOD M&E SYSTEM. .................................................................5 OVERALL FRAMEWORK FOR M&E ................................................................................................6 TABLE 1: THE M&E FRAMEWORK, WITH EXAMPLE AREAS, KEY QUESTIONS, AND INDICATORS ............7 LEVELS OF MONITORING AND EVALUATION..........................................................................10 HIV/AIDS .................................................................................................................................................13 SUMMARY TABLE FOR HIV/AIDS.........................................................................................................15 General resources .............................................................................................................................18 Technical assistance .........................................................................................................................19 Software products ............................................................................................................................19 Guidelines.........................................................................................................................................20 TUBERCULOSIS ....................................................................................................................................21 SUMMARY TABLE FOR TUBERCULOSIS ..................................................................................................21 General resources .............................................................................................................................23 Technical assistance .........................................................................................................................24 Software products ............................................................................................................................24 Guidelines.........................................................................................................................................24 MALARIA ................................................................................................................................................25 SUMMARY TABLE FOR MALARIA ...........................................................................................................27 General resources .............................................................................................................................29 Technical assistance and software products ....................................................................................29 Guidelines.........................................................................................................................................29 FREQUENTLY ASKED QUESTIONS ................................................................................................32 TECHNICAL QUESTIONS .........................................................................................................................32 OPERATIONAL QUESTIONS .....................................................................................................................36 ANNEX A..................................................................................................................................................40 ANNEX B..................................................................................................................................................62 ANNEX C..................................................................................................................................................74
This document was based on a collaboration between WHO, UNAIDS, The Global Fund to Fight AIDS, Tuberculosis and Malaria, USAID, CDC, UNICEF and the World Bank
M&E toolkit, Draft 14.01.04
Why this kit?

With the global momentum to scale up the response to the three main infectious diseases, HIV/AIDS, tuberculosis (TB) and malaria, public health practitioners need to provide various levels of accountability for their activities or policies to a variety of constituencies. It is becoming increasingly important for countries to be able to report accurate, timely and comparable data to donors and national authorities in order to secure continued funding for expanding health programmes and, most importantly, to utilize this information locally to strengthen evolving programmes. In sum, this toolkit is one step towards assuring that countries are able to measure, report, and use good quality health information in a manner that meets both donor and country needs. It is particularly important for national programme implementers and managers to have access to the quality information they need to make adjustments and programmatic and technical decisions. Much progress has been made in the monitoring and evaluation (M&E) of these three diseases through the international community from global disease partnerships such as UNAIDS, StopTB, and Roll Back Malaria. Existing M&E guidelines and materials have been developed through the collaborative work of many partnership constituents such as WHO, bilateral agencies and NGOs. Developed with the support of international funders and M&E experts, the purpose of this toolkit is to gather in one place the "essentials" of agreed upon best practice in M&E, by applying a common M&E framework for the three diseases and providing users with references to key materials and resources. Although labeled as a "monitoring and evaluation" toolkit, this document will focus mainly on the monitoring component. This toolkit aims to assist countries in the following: Formulation of a national M&E strategy by providing an overview of key issues to consider; Design of sustainable M&E systems that can be used to report on results and impact during the implementation stages of scaled up programmes; Implementation and quality control of M&E systems and reporting of progress; and Evaluation, review and improvement of M&E systems over time as the scale up of interventions to reduce morbidity and mortality associated with HIV/AIDS, TB and malaria occurs.
Who is it for?
This information package aims to provide those working at the country level on M&E systems linked to expanded HIV/AIDS, TB and/or malaria programmes with rapid access to key resources and standard guidelines. Users include national disease programme managers and project leaders and donor agencies, technical and implementing agencies and NGOs to better harmonise information demands.
What are its contents?

The toolkit includes a standard framework for the development of a range of M&E guidelines and tools, a summary of agreed upon illustrative core indicators for the three diseases, and references to more detailed indicator manuals on specific programme areas. M&E toolkit, Draft 14.01.04 3
In addition, this toolkit addresses frequently asked questions in relation to implementing M&E for HIV/AIDS, TB and malaria programmes. Note to users: The illustrative indicators presented have been developed for the national level, although many of them can be used at various levels. Country users should design or modify their health information collection system bearing in mind the different information that needs to be collected for use at different levels in order to construct the big picture that these indicators allow. Additionally, new technologies and developments will result in the need to periodically revise and update the illustrative indicators presented here. This is the first time indicators from these diseases have been brought together in one manual. The approach includes the need to develop and improve indicators at different levels over time based on feedback. It is therefore a work in progress. We will identify areas which require refinement in future editions as necessary. This document is available electronically at the websites of partners involved in its production.
Basic elements of M&E

While significant progress has been made in country M&E, much disease-specific M&E has been done in a vertical, isolated fashion that is often not linked or triangulated with other sources. For example, a surveillance system for HIV may be in place but not functioning well, and behavioral studies may have been done, though not necessarily using the same sampling methodologies or indicators. Extensive evaluation of a donorsponsored project may have been carried out in an important area of programming, without the results ever being shared with others in the field. In short, the utility of much of the disease-related measurement in a country may be lost because there is no coherent M&E system that can be used to capture necessary information on multiple diseases for users at different levels. In addition, many countries rely on population surveys such as DHS and MICS that are funded through external donors to gather information on the impact of their own and donor-supported programmes. This produces data that may be valuable in the broader M&E context, but may not be well integrated with traditional sources of health information, such as national health information and surveillance systems. A common, comprehensive and coherent M&E system has several advantages. It contributes to more efficient use of data and resources by ensuring, for example, that indicators and sampling methodologies are comparable over time and by reducing duplication of effort. Where resources are scarce, this is an important asset. Data generated by a comprehensive M&E system ought to serve the needs of many constituents, including programme or project managers, researchers or donors, eliminating the need for each to repeat baseline surveys or evaluation studies when they might easily use existing data. From the point of view of the national programme, a coherent M&E system helps ensure that donor-funded M&E efforts best contribute to national needs, rather than simply serving the reporting needs of specific international donors or organizations. A further advantage is that it encourages coordination and communication between different groups involved in the national response to diseases. Agreement among the major donor, technical and implementing agencies on the basic core M&E framework will reduce the burden of requests for data from different agencies. Shared planning, execution, analysis
or dissemination of data collection can reduce overlap in programming and increase cooperation between different groups, many of whom may work more efficiently together than in isolation. Countries have different M&E needs, dictated in part by the state of their HIV, TB, and/or malaria disease burdens. Yet successful M&E systems will share common elements. A list of some of these elements is given in Box 1. Box 1: Checklist of features of a good M&E system.
M&E UNIT

An established M&E unit within the Ministry of Health with designated technical and data management staff A budget for M&E that is between 5 and 10 percent of the combined national HIV/AIDS, TB, and malaria budgets from all sources A significant national contribution to the national M&E budget (not total reliance on external funding sources) A formalised (M&E) link, particularly with appropriate line ministries, NGOs and donors, and national research institutions aimed at enhancing operations research efforts A multisectoral working group to provide input and achieve consensus on indicator selection and various aspects of M&E design and implementation Epidemiological expertise in the M&E unit or affiliated with the unit Behavioural/social science expertise in the M&E unit or affiliated with the unit Data processing and statistical expertise in the M&E unit or affiliated with the unit Data dissemination expertise in the M&E unit or affiliated with the unit Well-defined national programme or project plans with clear goals, targets and operational plans Regular reviews/evaluations of the progress of the implementation of the national programme or project plans Guidelines and guidance to districts and regions or provinces for M&E Guidelines for linking M&E to other sectors Coordination of national and donor M&E needs A set of priority indicators and additional indicators at different levels of M&E Indicators that are comparable over time A number of key indicators that are comparable with other countries An overall national level data collection and analysis plan, including data quality assurance A plan to collect data and periodically analyse indicators at different jurisdictional levels of M&E (including geographical) Second-generation surveillance, where behavioural data are linked to HIV/STI surveillance data
CLEAR GOALS
INDICATORS
DATA COLLECTION & ANALYSIS
DATA DISSEMINATION
An overall national level data dissemination plan A well-disseminated, informative annual report of the M&E unit Annual meetings to disseminate and discuss M&E and research findings with policy makers, planners and implementers A clearinghouse for generation and dissemination of findings A centralised database or library of all HIV/AIDS, TB, and malaria-related data collection, including ongoing research Coordination of national and donor M&E dissemination needs
Overall framework for M&E

Indicators are used at different levels to measure what goes into a programme or project and what comes out of it. The most commonly used framework for the selection of indicators for M&E is the input-process-output-outcome-impact framework illustrated below. For a programme or project to achieve its goals, inputs such as money and staff time must result in outputs such as stocks and delivery systems for drugs and other essential commodities, new or improved services, trained staff, information materials, etc. These outputs are often the result of specific processes, such as training sessions for staff, that should be included as key activities aimed at achieving the outputs. If these outputs are well designed and reach the populations for which they were intended, the programme or project is likely to have positive short-term effects or outcomes, for example increased condom use with casual partners, increased use of insecticide-treated nets (ITNs), adherence to TB drugs, or later age at first sex among young people. These positive short-term outcomes should lead to changes in the longer-term impact of programmes, measured in fewer new cases of HIV, TB, or malaria. In the case of HIV, a desired impact among those infected includes increased survival time and behavioral change. For additional information on M&E frameworks, readers may be interested in visiting the following UNDP and MEASURE Evaluation sites: http://cfapp1.undp.org/undpweb/eo/evalnet/docstore3/yellowbook/ http://www.cpc.unc.edu/measure/publications/evalman/ Note: In many instances, behavioral change is considered a outcome of an HIV programmes efforts. For the purposes of this guide, however, behavioral change is considered an impact indicator. Measuring impact requires extensive investment in evaluation, and it is often difficult to ascertain the extent to which individual programmes, or individual programme components, contribute to overall reduction in cases and increased survival. In order to establish a cause-effect relationship for a given intervention, studies with experimental or quasi-experimental designs are necessary to demonstrate the impact. Therefore, focus is given here to output and outcome indicators, which are often more easily collected than impact indicators and used in the short to medium term for programme strengthening and reporting. As a programme or intervention matures, users may consider evaluating impact, using information that has been collected through the programmes life and/or by undertaking special evaluation studies. As impact evaluation is not the focus of this document, the undertaking of such studies is not discussed here. M&E toolkit, Draft 14.01.04 6
Table 1 presents a generalised M&E framework for AIDS, TB and malaria. Examples of the areas measured at each level, key questions to answer, and indicators are provided. The aim of Table 1 is to familiarize users with this framework in order to facilitate the use of this toolkit. This is particularly relevant for users familiar with other interpretations of the different levels. For example, the Global Fund to Fight AIDS, TB and Malaria (GFATM) generally defines process as a mixture of inputs and outputs, and coverage as a mixture of outputs and outcomes. Depending on the level of programme development, there may be some overlap in indicators to measure inputs, processes and outputs. For example, where trained personnel are available to the programme, they represent an input for the programme. However, where human resources are lacking, trained personnel may be an output for the programme. Table 1: The M&E framework, with example areas, key questions, and indicators
Level Area Policy Disbursement INPUT (strategies, policies, guidelines, financing) Infrastructure Coordination Human resources Key questions National strategic plans for each disease, including M&E and operations research plans exist Policy and guidelines exist Funds have been disbursed Supply chain is in place Coordination is established Infrastructure and equipment Human resources for health services delivery and supervision are recruited, adequately motivated, trained and deployed Human resources for supportive environment are trained and deployed Indicator example Policy and guidelines in place at national level Funding availability and release Distribution node selected Sentinel site selected Providers selected Coordination mechanism in place for technical and operational issues Coordination mechanism in place for political issues Number of people trained per number of people initially targeted by training Number of people trained according to national standards for an intervention Number of people trained for an intervention per 1,000 people in need of the intervention Number with adequate supervision and motivation
PROCESS (human resources, training, commodities)
Level
Area Drugs and commodities
Key questions Drugs are consistently available to consumers at the right time and place Providers are equipped and available to people seeking care Standard treatment guidelines and utilization manuals have been developed and produced Intervention is accessible in a large number or majority of districts or other administrative unit Resources are available for supervision
Service delivery, technologies
OUTPUTS (services, numbers reached, coverage)
Knowledge, skills and practice
Target population knows about the benefit of the intervention
Indicator example % of drug distribution nodes reporting on stock status (repletion, shortage, consumption, quality, losses) on a monthly basis % of drug distribution nodes/facilities reporting no drug shortage % of selected providers equipped for the intervention (laboratories, nursing, others) Number of districts or other administrative unit with at least one drug distribution center % of districts or other administrative unit with at least one drug distribution center Number of districts or other administrative unit with the required number of providers of the intervention % of districts or other administrative unit with the required number of providers of the intervention Number of districts or other administrative unit with designated sentinel/provider operating according to guidelines for the intervention % of districts or other administrative unit with designated sentinel/provider operating according to guidelines for the intervention Number of target population with desired health behaviour/attitude % of target population with desired health knowledge/attitude
Level
Area People on treatment, people benefiting from intervention
Key questions A majority of target population is covered by the intervention
OUTCOMES (changed behaviours, coverage)
Changed behaviour
IMPACT (biology and quality of life)
Morbidity, mortality
Increased number or proportion of target population adopting behaviours which reduce their vulnerability to infection, morbidity, and/or mortality Increased number or proportion of target population adopting beliefs and practices that create a supportive environment Majority of target population is in better health as a result of the intervention
Indicator example Number of target population covered by intervention % of target population covered by intervention Number of target population receiving first line therapeutic regime % of target population receiving first line therapeutic regime Ratio of first to second line treatment regime Number of target population with desired health seeking behaviour (risk reduction, health care seeking) % of target population with desired health seeking behaviour (risk reduction, health care seeking)
Number of target population showing clinical (and measurable) signs of recovery after 6, 12 months % of people showing clinical (and measurable) signs of recovery after 6, 12 months Disease prevalence at regional or national levels
A note on target populations and denominators: In many cases, it may be difficult to determine the denominator, or population, to use when assessing, for example, coverage. We have therefore focused on numerators, or the subset of the population that is affected or benefits from interventions. In this toolkit, though, denominators should also be included where possible (if percentages are given, numerators should also always be reported to allow assessment of coverage). The publication Estimating the Size of Populations at Risk for HIV (UNAIDS/IMPACT/FHI, 2002) may help readers in addressing the challenges faced in determining denominators when working with hidden populations. In this toolkit, the term target population refers to the group of people who benefit from an intervention. The target population can be the total population or a smaller group such as youth. In designing interventions, efforts should be made to clearly define the target population. Definition of these is usually based on knowing whom diseases affect most, directly and indirectly. For example, the definition of a target population for HIV/AIDS interventions is often based on the epidemic state. In generalized epidemics where HIV M&E toolkit, Draft 14.01.04 9
prevalence is consistently over 1% in pregnant women, the target population could very well be the general population. However, in concentrated and low-level epidemics where HIV prevalence is concentrated within groups with specific risk behaviors, the target group may be defined as a sub-group of the general population that shares these same behaviors.
Levels of Monitoring and Evaluation

This section presents illustrative core output, outcome, and impact indicators for HIV/AIDS, TB, and malaria. Users should be aware that these indicators have been developed, discussed and agreed upon by a wide range of international and national experts and donors. They have been developed for the specific purpose of minimizing information demands on countries while also assuring that indicators address specific international needs. The indicator development process was guided by five major principles: Building on existing indicators; Harmonizing with other international frameworks such as the Millennium Development Goals (MDG); Minimizing the number of indicators to be collected; Covering a wide range of programme areas and sectors related to the different diseases; and Addressing country programme needs. Input and process indicators are generally common across the three disease areas and are therefore not specified for each. While there are some differences across the three diseases, these indicators generally take on the following forms: Generic input indicator: Existence of national policies, guidelines, or strategies. This is a yes / no question. Reporting of overall budget allocation is included as an input. Generic process indicator: Number of persons trained, number of drugs shipped/ordered, etc.
For each disease, general programme areas have been defined. In the case of HIV/AIDS, for example, these include prevention, treatment, care and support, and supportive policy/implementation environments. A summary table showing the different programme areas as well as indicators is presented for HIV/AIDS, TB, and malaria. When looking at the summary tables, readers should be aware that sub-programmes often contribute to multiple outcomes and impact. Although the tables give the impression of a linear progression, assumptions regarding the overall outcome and impact of each subprogramme should be made with caution. Except for some output and outcome measurements (referred to as "counts", see below), specific information is provided for each of the indicators presented in the summary tables. This information can be found in the more detailed explanation of each indicator. Information provided for each indicator includes:
10
Rationale for use Definition, including numerator and denominator Measurement - i.e. details on instrument and process Data collection platform, sources i.e. survey, vital registration, in/out-patient registers, facility surveys, inventories, surveillance and sentinel reports Recommended periodicity Resources i.e. reference groups, technical assistance sources, guidelines Outputs and outcomes here are also monitored and reported as "counts" of increased capacity provided against a need that has been estimated as a pre-condition for change and they can be quantified through direct observation or an annotated inventory. For example, it may be easier to collect the number of health providers trained in a specific area through a record review. For these "counts", the toolkit does not provide a detailed description, and the definition of associated terms -where relevant-appears under the detailed description of outcome indicators. Table 2 provides an overview of the service delivery areas and main objectives for HIV/AIDS, TB and Malaria. Table 2. Overview of service delivery areas and objectives for HIV/AIDS, TB and Malaria
Delivery areas
HIV/AIDS Information, Education, Communication (IEC) Youth Education
TB Identification of infectious cases Prevention of transmission by treating infectious cases Prevention of TB among PLWHA
MALARIA Insecticide-treated nets (ITNs)
Malaria in pregnancy
Condom distribution
Prediction and Containment of epidemics Indoor Residual Spraying
Prevention
Programmes for specific groups Counseling and voluntary testing Prevention of mother to child transmission (PMTCT) STI diagnosis and treatment Post-exposure prophylaxis (PEP)
Information, education & communication (IEC)
11
Blood safety and universal precautions Antiretroviral treatment and monitoring Treatment Timely detection and quality treatment of cases Control of drug resistance Prompt effective antimalarial treatment
Prophylaxis and treatment for opportunistic infections HIV/TB
Monitoring of drug resistance
Support for orphans Care & Support Support for the chronically ill Strengthening of Civil Society Stigma
Systematic monitoring of performance in case management Supporting patients through direct observation of treatment
Home based management of malaria
Sufficient and quality ensured drugs and lab supplies Building and maintaining human resource capacity Health systems strengthening
Monitoring and Operations Research Health systems strengthening Monitoring, evaluation, and operational research
Supportive Environment
Health systems strengthening Coordination and partnership development (national, community, public-private) Monitoring, evaluation, and operational research Procurement and supply management capacity building Reduced adult HIV prevalence (ages 15-49) Reduced percentage of young people aged 1524 who are HIVinfected
Coordination and partnership development (national, community, public-private) Monitoring, evaluation, and operational research Operational research agenda targeting barriers to DOTS Procurement and supply management capacity building Reduced number of smear-positive cases per 100,000 population Reduced number of deaths from TB (all forms) per 100,000 population per year
Coordination and partnership development (national, community, public-private)
Procurement and supply management capacity building Reduced all-cause under 5 mortality (endemic areas) Reduced Malaria specific mortality
12
Reduced percentage of high risk groups (sex workers, clients of sex workers, men who have sex with men, injecting drug users) who are HIV infected Reduced percentage of HIV-infected infants born to HIV-infected mothers Increased survival among PLWHA Percentage of young people aged 15-24 reporting the use of a condom during sexual intercourse with a nonregular sexual partner Percentage of young people who have had sex before the age of 15
Reduced Malaria specific morbidity
Percentage of young people who had sex with more than one partner last year Percentage of high risk groups who have adopted behaviours that reduce transmission of HIV Percentage of adults on ARV treatment who gain weight by at least 10% at 6 months after the initiation of treatment
HIV/AIDS
This section of the toolkit provides an overview of the core indicators and general M&E resources for HIV/AIDS (in addition to those provided for each indicator). Each of the HIV/AIDS core indicators is applicable to all settings, with the exception of the
13
indicators covering injecting drug users (IDU) and HIV prevalence. The IDU indicator is applicable to countries where injecting drug use is an established, significant mode of HIV transmission. Countries with low HIV prevalence or concentrated epidemics should report on an alternative indicator of HIV prevalence among high-risk behavior groups, as opposed to prevalence among young people obtained from antenatal clinic sentinel surveillance. Alternative indicators may be found in the UN General Assembly Special Session (UNGASS) on AIDS document entitled Monitoring the Declaration of Commitment on HIV/AIDS Guidelines on the construction of core indicators (UNAIDS, 2002).
14
Summary table for HIV/AIDS

Service Delivery Area Information, Education, Communicati on (IEC)
Input Budget reporting by financial category, Monitoring and Evaluation Framework exists, Strategies and Guidelines developed and used, Implementing Partners identified
Process Increased numbers trained (health personnel, government, non-government, private sector), Commodities purchased (condoms, Drugs, Lab supplies [microscopes, reagents, slides])
Output HIV/AIDS radio/television programmes/newspapers produced* HIV/AIDS prevention brochures/booklets distributed* Peer educators active*
Outcome
Youth Education
Schools with teachers trained in life-skills based HIV/AIDS education (PI1) Young people exposed to HIV/AIDS education in school settings* (under development) Young people exposed to HIV/AIDS education out of school* (under development) Retail outlets and service delivery points with condoms in stock (PI2) Condoms sold through public sector* Condoms sold through private outlets* Sex workers & clients exposed to outreach programmes* (number and percentage**) MSM exposed to outreach programmes* (number and percentage**)
Prevention Condom distribution Programmes for specific groups
15
Service Delivery Area
Input
Process
Output Mobile populations exposed to outreach programmes* (number and percentage**)
Outcome
IDUs reached by prevention services (number* and percentage) (PI3) Large companies with HIV/AIDS workplace policies and programmes (number* and percentage) (PI4) Counseling and Voluntary Testing PMTCT Districts with VCT services* (PI5) People requesting counseling and voluntary testing (PI6)* HIV-infected pregnant women receiving a complete course of antiretroviral prophylaxis to reduce the risk of MTCT (number* and percentage) (PI8) STI comprehensive case management (PI9) Number of people who receive post-exposure prophylaxis* Districts with access to donor recruitment and blood transfusion (PI10) Transfused blood units screened for HIV (PI11)
Health facilities offering minimum package of PMTCT* (PI7)
STI diagnosis and treatment Post-exposure prophylaxis (PEP) Blood safety and universal precautions
16
Service Delivery Area Antiretroviral treatment(AR T) and monitoring
Input
Process
Output Health facilities capable of providing advanced interventions for prevention and medical treatment for HIV infected persons (TI2) Health facilities with capacity to deliver basic level counseling and medical services for HIV/AIDS (number* and percentage) (TI 3) Intensified TB case finding (among PLWHA) (TI 4) Counseling and voluntary testing for TB patients (TI 5)
Outcome People with advanced HIV infection receiving antiretroviral combination therapy (number* and percentage) (TI1)
Prophylaxis and treatment for opportunistic infections (OIs) HIV/TB
Treatment
Provision of cotrimoxazole preventive therapy and/or ART (TI 6 and TI 7)
Support for orphans
Care and Support
Orphans and vulnerable children whose households received free basic external support (number* and percentage) (CS1) Chronically ill adults whose households received free basic external support (number* and percentage) (undergoing adaptation) Number of NGOs dealing with HIV/AIDS services * Number of PLWHA support groups fighting against discrimination*
Support for the chronically ill
Supportive Environment
Strengthening of Civil Society Stigma
Monitoring, evaluation, and operational research
17
Service Delivery Area Health systems strengthening Coordination and partnership development (national, community, publicprivate) Procurement and supply management capacity building
Input
Process
Output Number of patients who are accurately referred*
Outcome
Number of networks/partnerships involved*
Percentage of service delivery points with sufficient drug supplies (under development)
Unit costs of drugs and commodities
* Outputs and outcomes here are also measured as "counts" of increased capacity provided against a need that has been estimated as a pre-condition for change and they can be quantified through direct observation or an annotated inventory. For these "counts" the toolkit does not provide a detailed description in the annexes., ** Both percentages and numbers are required. However, if a denominator can not be obtained, focus should be on raw numbers (the numerator).
Note for HIV/TB service delivery area: TB/HIV programmes are complex in that two separate disease programmes are brought together, with each having individual approaches and reporting mechanisms in place. For full details, refer to "Interim Policy on Collaborative TB/HIV Activities. WHO/HTM/HIV/2004.1 and WHO/HTM/TB/2004.330, and "A Guide to Monitoring and Evaluation for Collaborative TB/HIV Activities" (WHO, in preparation) Detailed descriptions for each of the indicators listed above are provided in Annex A. General resources Since the creation of UNAIDS, a number of M&E resource groups mainly at global level were established to improve coordination among key M&E players. Currently, there are a total of five groups: The UNAIDS Monitoring and Evaluation Reference Group (MERG) composed of cosponsors/Secretariat M&E focal points, bilateral agencies, research institutes, and individual experts that assists in harmonizing M&E approaches and improving methods.
18
The UNAIDS Estimates, Modelling and Projections Reference Group and UNAIDS/WHO working group on surveillance and estimates for HIV transmission and mortality. The Inter-Agency M&E coordination working group composed of key UNAIDS cosponsors, Secretariat and Global Fund to Fight AIDS, TB and Malaria M&E focal points that assists in improving coordination among global M&E actors. The UNAIDS Evaluation Unit composed of UNAIDS Secretariat staff that assists in the development of generic M&E systems for strategic information sharing. The Global Monitoring and Evaluation Support Team (GAMET) composed of World Bank personnel and staff seconded from technical agencies that focuses on M&E country support in World Bank-supported countries.
These resource groups have contributed to the development of the illustrative indicators presented here. UNAIDS and partners have been encouraging governments to set up a national level M&E reference/support group to provide advice on national M&E strategies, and to assist in mobilizing resources for M&E and optimizing the use of data. Where those groups exist, coordination among partners has tremendously improved. Technical assistance Although technical support to governments is available through M&E technical support groups in some countries, additional assistance can be sought from the Evaluation Unit at the UNAIDS Secretariat at UNGASSindicators@unaids.org for specific questions on UNGASS Declaration of Commitment (DoC) indicators, or at M-E@unaids.org for general M&E questions. Other sources of support for all the diseases include: CDC, Measure Evaluation, Partners for Health Reform Plus (USA), Institute for Health Systems Development (UK). Further support for HIV/AIDS includes: Measure Evaluation and Measure DHS, FHI, The Synergy Project. Software products UNAIDS has put at the disposal of countries a useful tool the Country Response Information System (CRIS) that has the potential to house all national data obtained on core and additional indicators and generate reports on those indicators. The CRIS includes two additional functions: resource tracking and research inventory. To learn more about the process of indicator development and the suggested actions to implement the DoC M&E framework, readers are encouraged to consult the Guidelines on construction of core indicators that exist in four languages (English, French, Spanish and Russian) and that can be downloaded from UNAIDS web site. For more information on the CRIS, also please visit the UNAIDS web site.
19
Guidelines The major sources for guidelines cited below are UNAIDS, WHO, USAID, CDC, MEASURE and FHI. The latest versions of these guidelines may be found on the Internet at: http://www.unaids.org http://www.who.int http://www.cpc.unc.edu/measure http://www.fhi.org http://www.cdc.gov http://www.usaid.gov UNAIDS/MEASURE (2000). National AIDS Programmes: A Guide to Monitoring and Evaluation. UNAIDS, Geneva. (http://www.cpc.unc.edu/measure/guide/guide.html) [SUGGESTION TO MOVE THIS TO FIRST ON THE LIST] UNAIDS (2002). Monitoring the Declaration of Commitment on HIV/AIDS Guidelines on the construction of core indicators (http://www.unaids.org/UNGASS/docs/JC894-CoreIndicators_en.pdf) UNAIDS/World Bank (2002). National AIDS Councils (NACs) Monitoring and Evaluation Operations Manual. UNAIDS/World Bank, Geneva. (http://www.unaids.org/publications/documents/epidemiology/surveillance/JC808MonEval_en.pdf) Centers for Disease Control and Prevention (2002). Strategic Monitoring and Evaluation: A Draft Planning Guide and Related Tools for CDC GAP Country Programs. Centers for Disease Control and Prevention, Atlanta. Family Health International (2002). Evaluating Programs for HIV/AIDS Prevention and Care in Developing Countries: A Handbook for Program Managers and Decision Makers. Family Health International, Arlington. (http://www.fhi.org/en/aids/impact/impactpdfs/evaluationhandbook.pdf) Family Health International (2000). Behavioural Surveillance Surveys (BSS): Guidelines for Repeated Behavioural Surveys in Populations at Risk for HIV. Family Health International, Arlington. (http://www.fhi.org/en/aids/wwdo/wwd12a.html#anchor545312) WHO/UNAIDS (2000). Second Generation Surveillance for HIV: The Next Decade. UNAIDS, Geneva. (http://www.who.int/emc-documents/aids_hiv/docs/whocdscsredc2005.PDF)
20
Tuberculosis
This section provides an overview of the core indicators for TB control and offers resources for more in-depth consideration of monitoring and evaluation in TB. The indicators are general in nature and appropriate for monitoring TB control, particularly through national TB control programmes. The indicators do not specifically address the additional monitoring needs of innovations in service delivery such as community-based care or engagement of the private sector. Similarly, only a limited number of indicators are provided for monitoring TB/HIV interventions and the management of multi drugresistant TB. Readers are guided to additional references for more comprehensive monitoring of such activities. A compendium of indicators for monitoring TB control activities is under preparation by the Working Group on Indicators whose partners are listed in the general resources. Many of the indicator definitions provided in this toolkit were drawn from a draft of the compendium. Summary table for tuberculosis
SubProgramme Identification of infectious cases
Input
Priority within overall health sector plan, TB policy consistent with international guidelines, mid-term operational plan consistent with international guidelines, financial resources consistent with operational plan,
Process Increased numbers trained (health personnel, government, nongovernment, private sector), Commodities purchased (Drugs, Lab supplies [microscopes, reagents, slides]), comprehensive laboratory
Output
Outcome New smear positive TB cases detected under DOTS (number* and percentage) (PI 1) New smear-positive cases registered under DOTS who smearconvert at 2 months of treatment (number* and percentage) (PI 2)
Prevention
Prevention of transmission by treating infectious cases
Prevention of TB among PLWHA
HIV seroprevalence among TB patients (PI 4)
Treatment
Timely detection and quality treatment of cases
network established
Individuals dually infected with TB and HIV who receive isoniazid preventive therapy (number* and percentage) (PI 3)
Number of death from TB per 100,000 per year(number* and
Population covered by DOTS (number* and proportion) (TI 1)
percentage)
21
Smear-positive TB cases registered under DOTS who are successfully treated (TI 2) (number* and percentage) Control of drug resistance New smear-positive cases registered under DOTS who default or transfer out of treatment (number* and percentage) (TI 3) (number* and percentage) Treatment facilities submitting accurate, timely and complete reports (number* and proportion) (TI 4) Patients cared for with directly observed therapy (DOT) during intensive phase (number* and proportion) (CS 1) Number of health facilities involved in DOTS with sufficient drug and laboratory supplies** (SE 1 and SE 2)
Systematic monitoring of performance in case management
Care & Support Supportive Environment
Supporting patients through directly observed therapy
Sufficient and quality ensured drugs and lab supplies
Building and maintaining human resource capacity
Number of health facilities and laboratories involved in DOTS with sufficient capacity for DOTS*
22
Monitoring, evaluation, and operational research (focusing on barriers to DOTS implementation)
Number of training activities conducted as per operational plan* Number and proportion of health workers receiving regular supervisory visits* Number of patients who are accurately referred* Health facilities and laboratories with capacity for DOTS implementation (number* and proportion) Number of networks/partnerships involved*
Health systems strengthening
Coordination and partnership development (national, community, public-private)
Procurement and supply management capacity building
Percentage of service delivery points with sufficient drug supplies (under development)
* Outputs and outcomes here are also measured as "counts" of increased capacity provided against a need that has been estimated as a pre-condition for change and they can be quantified through direct observation or an annotated inventory. For these "counts" the toolkit does not provide a detailed description in the annexes. ** Although this information focuses on reporting the number of health facilities, a detailed description of the associated indicator reporting on the proportion is provided.
The detailed description of each of the indicators listed above is provided in Annex B. General resources Tuberculosis Monitoring and Evaluation unit of Stop TB Department of World Health Organization: building capacity at country level for monitoring, evaluation and evidence-based planning, conducting global surveillance of epidemiological and financial trends in TB control
23
Stop TB Partnership Working Groups: Three operational working groups provide a focus for coordinated action and support monitoring and evaluation of country-level activities related to o DOTS expansion, including sub-groups on laboratories and public-private mix o TB/HIV o MDR-TB
Global Working Group on Indicators a partnership between the World Health Organization, World Bank, U.S. Centers for Disease Control and Prevention, International Union Against Tuberculosis and Lung Disease (IUATLD), Royal Netherlands Tuberculosis Association (KNCV), U.S. Agency for International Development (USAID) and Measure. Contact: cvincent@usaid.gov
Technical assistance International Union Against TB and Lung Diseases (IUATLD): www.iuatld.org

Royal Netherlands Tuberculosis Association (KNCV): www.tuberculose.nl U.S. Centers for Disease Control: www.cdc.gov (mqualls@cdc.gov) World Health Organization: www.who.int (dyec@who.int) World Bank: www.worldbank.org (dweil@worldbank.org)
Software products WHO EpiCentre software to manage quarterly reporting data Contact: WHO SEARO (Nani Nair, nairn@whosea.org)
Electronic TB Register (ETR): a computerized TB register capturing individual patient data available from the U.S. Centers for Disease Controls Botusa project in Africa Contact: Peter Vranken (pbv7@botusa.org). New Windows (Access) application combines features of EpiCentre and ETR, and is accompanied by "specifications" for software tool development Contact: WHO Geneva (Dan Bleed, bleedd@who.int).
Guidelines World Health Organization (2002). An expanded DOTS framework for effective tuberculosis control. http://www.who.int/gtb/publications/dots/pdf/TB.2002.297.pdf
World Health Organization (1998). Tuberculosis handbook. http://www.who.int/gtb/publications/tbhandbook/index.htm
24
World Health Organization (2002). Global Tuberculosis Control: WHO Report 2003. http://www.who.int/gtb/publications/globrep/index.html World Health Organization (2003). Management of Tuberculosis Training for health facility staff. http://www.who.int/gtb/publications/training/management_of_tb/pdf/who_cds_tb _2003_314i.pdf World Health Organization (1998). Laboratory services in tuberculosis control. http://www.who.int/gtb/publications/whodoc/who_tb-98258/en/98.258_org_management- .pdf World Health Organization (2001). The Use of Indicators for communicable disease control at district level. http://www.who.int/gtb/publications/indicators/tb_2001_289.pdf World Health Organization (2001). Good practice in legislation and regulations for TB control: An indicator of political will. http://www.who.int/gtb/publications/General/TB_2001_290legisl.pdf World Health Organization (2000). Guidelines for establishing DOTS-PLUS pilot projects for the management of multidrug-resistant tuberculosis (MDR-TB). http://www.who.int/gtb/publications/dotsplus/dotspluspilot-2000279/english/index.htm World Health Organization (2003). Guidelines for implementing collaborative TB and HIV programme activities. http://www.who.int/gtb/publications/tb_hiv/2003_319/tbhiv_guidelines.pdf World Health Organization (1998). Guidelines for conducting a review of a national tuberculosis programme. http://www.who.int/gtb/publications/whodoc/who_tb_98.240.pdf
Malaria
This section of the toolkit provides a generalized framework for monitoring and evaluation of specific interventions or service delivery areas within malaria control programmes. An overview of the indicators for M & E across interventions is presented and general resources that are available or in preparation. Each of the indicators is applicable to all malaria endemic settings, with the exception of the indicators covering impact and epidemics. The indicator for the prediction of epidemics should only be used for countries with epidemic-prone areas. With regard to monitoring impact, the primary indicator to be monitored by all African countries and high endemic settings is all-cause under-5 mortality, as measured by nationallyrepresentative, household surveys. Malaria-specific mortality cannot be measured routinely, as it is difficult to measure in malaria-endemic Africa. Symptoms and signs (such as anemia) are not specific and sensitive, making autopsy and verbal autopsy
25
inaccurate; and many deaths, especially in young children, may be malaria related rather than attributable to malaria exclusively without concurrent infections. Moreover, a majority of deaths do not occur in hospitals and are not routinely recorded in HMIS, and these are unlikely to be picked up in vital registration systems, which are often incomplete. .
26
Summary table for malaria

SubProgramme Insecticidetreated nets (ITNs) Inpu t Budget reporting by financial category, Monitoring and Evaluation Framework exists, Strategies and Guidelines developed and used, Implementing Partners identified Process Increased numbers trained (health personnel, government, non-government and private sector), commodities purchased (drugs, ITNs, insecticides, other, purchasing policy) Output Number of nets, LLNs, pretreated nets or retreatment kits distributed* Number of nets retreated* Number of sentinel sites established for monitoring insecticide resistance* Number of nets, LLNs, pretreated nets or retreatment kits distributed* Number of nets retreated* Outcome Households owning ITN (PI1)
Children under 5 using ITN (PI 2)
Malaria in pregnancy
Pregnant women using ITN (PI 3)
Prevention
Pregnant women receiving treatment (IPT) or chemoprophylaxis (PI 4)
Number of pregnant women receiving correct IPT* Malaria epidemics detected and properly controlled (PI 5) Number of homes and areas sprayed with insecticide* Number of targeted areas with IEC services*
Prediction and containment of epidemics Indoor Residual Spraying Information, education, and communication (IEC)
27
SubProgramme Prompt, effective antimalarial treatment
Inpu t
Process
Output Number of patients with uncomplicated and severe malaria receiving correct diagnosis and treatment* Number of health facilities with no reported stockouts of antimalarial drugs* (TI 2)
Outcome Children under 5 years of age with access to prompt effective treatment (TI1)
Treatment
Patients with severe malaria receiving correct treatment (TI 3)
Monitoring drug resistance
Number of sentinel sites established for monitoring antimalarial drug resistance* Number of caretakers recognizing signs and symptoms of malaria* % of budget spent of monitoring and operations research Number of patients who are accurately referred* Number of networks/partnerships involved*
Home-based management of malaria
Monitoring, evaluation, and operational research Supportive Environment Health systems strengthening Coordination and partnership development (national, community, public-private) Procurement and supply management capacity building
Percentage of service delivery points with sufficient drug supplies
* Outputs and outcomes here are also measured as "counts" of increased capacity provided against a need that has been estimated as a pre-condition for change and they can be quantified through direct observation or an annotated inventory. For these "counts" the toolkit does not provide a detailed description in the annexes..
The detailed description of each of the indicators listed above is provided in Annex C.
28
General resources Since the creation of Roll Back Malaria (RBM), a Monitoring and Evaluation Reference Group (MERG) has been established to improve coordination among key M&E players. The main function of the MERG is to act as an advisory body for the RBM Secretariat, hence to give technical guidance related to monitoring progress in malaria control. The actual M&E work is being implemented by National Malaria Control Programmes with support from the inter-country teams and RBM partners. General information on the activities and products of the MERG can be found at the following link: http://mosquito.who.int/partnership/wg/wg_monitoring/summary.htm. Technical assistance and software products Technical support to governments is available through a variety of sources, most notably through the RBM Monitoring and Evaluation Reference Group (MERG) and WHO headquarter and ergional offices, as well as RBM inter-country offices. Further, M&E technical support groups have been established in some countries through the broader RBM partnership. Guidelines More information on monitoring and evaluation of malaria control activities can be found in the following documents: General ! Roll Back Malaria. Framework for Monitoring Progress and Evaluating Outcomes and Impact. 2000. Geneva. Available online: http://rbm.who.int/cmc_upload/0/000/012/168/m_e_en.pdf. ! Roll Back Malaria/MEASURE. Guidelines for core indicators for assessing malaria intervention coverage from household surveys. 2004. In preparation. ! Roll Back Malaria. Monitoring and Evaluation Reference Group, Mortality Task Force. Meeting Minutes. 16 July 2003. Available online: http://mosquito.who.int/partnership/wg/wg_monitoring/summary.htm ! Roll Back Malaria. Malaria Indicator Survey (MIS). In preparation. ! WHO/UNICEF. Africa Malaria Report 2003. Available online: http://mosquito.who.int/partnership/wg/wg_monitoring/summary.htm Policies and guidelines WHO. Management of Severe Malaria: A practical handbook. 2nd Edition. Geneva 2000. Available online: http://rbm.who.int/docs/hbsm.pdf. ! WHO. The Use of Antimalarial Drugs: Report of an Informal Consultation, Geneva, 13-17 November 2000. Available online: http://rbm.who.int/cmc_upload/0/000/014/923/use_of_antimalarials.pdf. ! WHO. Antimalarial Drug Combination Therapy: Report of a WHO Technical Consultation, Geneva, 4-5 April 2001. Available online: http://rbm.who.int/cmc_upload/0/000/015/082/use_of_antimalarials2.pdf. ! WHO Regional Office for the Western Pacific. Malaria Rapid Diagnosis: Making it work. Meeting Report 20-23 January 2003. Manila. Available online: http://rbm.who.int/cmc_upload/0/000/016/750/rdt2.pdf.
!
29
Drug supply management ! Management Sciences for Health. Drug Management for Malaria. June 2000, revised July 2002. Rational Pharmaceutical Management Program. ! John Snow International. Logistics Indicators and Monitoring and Evaluation Tools. DELIVER Project. Available online: http://deliver.jsi.com/2002/Pubs/Pubs_Guidelines/index.cfm. Drug resistance ! WHO. Monitoring Antimalarial Drug Resistance. 2002. Report of a WHO consultation, Geneva, Switzerland 35 December 2001. Available online: http://rbm.who.int/cmc_upload/0/000/015/800/200239.pdf. Home-based management Roll Back Malaria/UNDP/World Bank/WHO TDR. Scaling up home-based management of malaria: from research to implementation. 2003. Geneva. In preparation. ! Carol Baume. A Guide to Research on Care-seeking for Childhood Malaria. Published by the Support for Analysis and Research in Africa (SARA) Project and the Basic Support for Institutionalizing Child Survival Project (BASICS II) for the United States Agency for International Development. Arlington, Virginia, April 2002. Available online: http://www.aed.org/publications/GuideResearch.pdf.
!
Vector control including insecticide-treated nets (ITNs) Roll Back Malaria. Scaling-Up Insecticide-Treated Netting Programmes in Africa: A Strategic Framework for Coordinated National Action. 2002. Geneva. Available online: http://rbm.who.int/cmc_upload/0/000/015/845/itn_programmes.pdf. ! Roll Back Malaria. Insecticide-Treated Mosquito Net Interventions: A Manual for National Control Programme Managers. 2003. Available online: http://rbm.who.int/cmc_upload/0/000/016/211/ITNinterventions_en.pdf. ! WHO. Space spray application of insecticides for vector and public health pest control: A practitioner's guide. Geneva, 2003 (document WHO/CDS/WHOPES/GCDPP/2003.5. Available online: http://www.who.int/ctd/whopes/docs/Brochure_Space.pdf.
!
Malaria in pregnancy Roll Back Malaria. Strategic framework for malaria control during pregnancy. 1 November 2002.
Malaria epidemics Hook C. Field Guide for Malaria Epidemic Assessment and Reporting. DRAFT for Field Testing. World Health Organization. 2003. Available online: http://rbm.who.int/cmc_upload/0/000/016/569/FTest.pdf. ! Roll Back Malaria. Prevention and Control of Malaria Epidemics: 3rd Meeting of the TSN, Geneva, 10-11 December 2001. 2002. Available online: http://rbm.who.int/cmc_upload/0/000/015/827/3epidemics_report.pdf. ! Roll Back Malaria. Malaria Early Warning Systems: A Framework for Field Research in Africa . Available online:
!
30
http://rbm.who.int/cmc_upload/0/000/014/807/mews2.pdf. Training and human resources development ! Roll Back Malaria. RBM Human Resource Needs Assessment & Planning Tool. 2003. In preparation.
31
Frequently asked questions

Technical questions 1. What is the difference between monitoring and evaluation? Monitoring is the routine tracking of the key elements of programme/project performance, usually inputs and outputs, through record-keeping, regular reporting and surveillance systems as well as health facility observation and client surveys. Monitoring helps programme or project managers determine which areas require greater effort and flag questions that might contribute to an improved response. In a well-designed monitoring and evaluation system, monitoring contributes greatly towards evaluation. Indicators selected for monitoring will be different depending on the reporting level within the health system. It is very important to select a limited number of indicators that will actually be used by programme implementers and managers. There is a tendency to collect information on many indicators and report this information to levels where it will not and cannot be used for decision-making. In contrast, evaluation is the episodic assessment of the change in targeted results that can be attributed to the programme or project/project intervention. In other words, evaluation attempts to link a particular output or outcome directly to a particular intervention after a period of time of implantation of a particular programme has passed. Evaluation helps programme or project managers determine the value or worth of a specific programme or project.
2. What is the difference between national and sub-national M&E? In view of scarce M&E resources at sub-national level, emphasis is placed on monitoring programme inputs and outputs and assessing whether or not implementation progresses according to a sub-national plan. A small facility assessment as part of a routine supervision could serve to provide information on the quality of care or the availability and utilization of services. At national level, both monitoring and evaluation are needed. Sub-national data is extremely relevant for national level M&E provided that national guidelines are followed to make aggregation possible. For example, if a country has actual data on condom distribution by district (or equivalent) instead of one national overall figure, monitoring of trends in condom use may become more meaningful and more accurate. 3. What is the difference between programme and project M&E? Programme refers to an overarching national or sub-national response to the disease. Within a national programme, there are typically a number of different areas of programming. For example, the HIV/AIDS programme has a number of subprogrammes such as blood safety, STI control, or HIV prevention for young people.
32
Project refers to a mix of interventions that aim at a specific population defined geographically or otherwise. In view of its wider scope (thematic, geographic, target population), programme monitoring tends to be more complex than project monitoring and requires strong coordination among all implementing agencies. Programme evaluation is even more difficult, especially for certain types of evaluations (outcome and impact evaluations). For such evaluations to be conducted, the design of the programme/project must include its own baseline and follow-up assessments measuring not only specific outcomes but also the level of exposure to the programme/project and its activities. (See question 4 for more details on evaluations) 4. When is the appropriate timing for an evaluation? The timing for a specific type of evaluation depends on the implementation status of a programme or project. There are four types of programme or project evaluations: Formative evaluation Process evaluation Outcome evaluation Impact evaluation
Formative evaluation is conducted in the design phase of a prevention and care programme to identify and resolve intervention and evaluation issues before the programme is widely implemented. Formative evaluation identifies transmission dynamics, assists in identifying effective interventions and helps define realistic goals. Process evaluation involves the assessment of the programme or projects content, scope or coverage together with the quality of implementation. If the process evaluation finds that the programme/project has not been implemented, or is not reaching its intended audience, it is not worth conducting an outcome evaluation. However, if process evaluation shows progress in implementing the programme/project as planned, then it is worth carrying out such an evaluation. In outcome evaluation, the evaluation is designed specifically with the intention of being able to attribute the changes to the intervention itself. At the very least, the evaluation design has to be able to plausibly link observed outcomes to a well-defined programme or project, and to demonstrate that changes are not the result of non-programme/project factors. If the evaluation shows a change in outcomes, then it is time for impact evaluation. True impact evaluation, able to attribute long-term changes to a specific programme or project, is very rare. Rather, monitoring impact indicators taken in conjunction with process and outcome evaluations are considered to be sufficient to indicate the overall impact. 5. Does evaluation require more than monitoring? As seen in questions 1 to 4, the objectives and the methodology used in monitoring and evaluation are different. In general, evaluations are more difficult in view of the
33
methodological rigor needed; without such rigor, wrong conclusions on the value of a programme or project can be drawn. They are also more costly, especially outcome and impact evaluations which require population-based surveys. 6. What is operations research? Operations Research (OR) is a rigorous type of evaluation that complements M&E systems. The main objective of OR is to provide programme managers and policy makers with the required information to develop, improve, or scale-up programmes. It can be thought as a practical, systematic process for identifying and solving programmerelated problems. The process has five key steps: 1. Problem identification and diagnosis 2. Selection of a programme strategy 3. Strategy testing and evaluation 4. Information dissemination 5. Information utilization and scaling-up Once operations research shows that a given intervention can be effective, tracking more generalized implementation is needed through a strong national M&E system. For example, if OR shows that sex education in selected high schools can reduce risk behavior, repeated behavioral surveys among a national sample of high-school students would be needed to reflect changes in risk behavior following the integration of sex education into the nation-wide curriculum. 7. Are all indicators equal? The M&E conceptual framework discussed earlier shows that the different types of indicators are not equal but linked to each other to reach the intended goals and objectives of a specific programme. Inputs such as money and staff time result in outputs such as stocks and delivery systems for drugs or other essential commodities, new or improved services, trained staff, informational materials, etc. If these outputs are well designed and reach the populations for which they were intended, the programme is likely to have positive outcomes depending on the context in which it operates. These positive outcomes should lead to changes in the longer-term impact of programmes on target populations or systems. 8. How often are different indicators measured? The frequency of reporting will depend on the place of the indicators within the M&E conceptual framework taking into account a reasonable time frame for an expected change and programme capacity for M&E. The following reporting schedules are suggested:
34
Type of indicator Input Process Output Outcome Impact
Frequency of measurement continuously quarterly quarterly 2 to 3 years 3 to 5 years
9. Why do we need standard indicators? The use of standard indicators provides the National Programme with valuable measures of the same indicator in different populations, permitting triangulation of findings and allowing regional or local inconsistencies and differences to be noted and addressed. This helps to direct resources to regions or sub-populations with greater needs and to identify areas for intensification or reduction of effort at the national level, ultimately improving the overall effectiveness of the national response. The use of standard indicators also ensures comparability of information across countries and over time. In designing their own evaluation activities, projects should also bear in mind the national standard for indicators in that field. Projects may have their own information needs that conform to a rigorous evaluation design. However, whenever possible they should choose indicators with standard reference periods, denominators, etc. that would allow the data they collect to be fed easily into the national M&E system. 10. What do we mean by a sound, comprehensive or coherent M&E system? A sound M&E system has the following key features: An established M&E unit in the government (Ministry of Health or national disease-specific councils) with formalized links with different line ministries (depending on the disease), research institutions, donors, and NGOs Well defined national programme goals and targets A national M&E plan including a set of priority indicators at different levels of M&E based on the national strategic plans; comparable over time; a subset comparable with other countries; and data collection, quality control, analysis, dissemination and use plan.
A coherent M&E system is closely linked to effective M&E coordination among key stakeholders, leading to more efficient use of resources and data. It helps ensure that donor-funded M&E efforts best contribute to national needs, rather than to simply serve the reporting needs of agencies. It also encourages communication between different groups involved in the national response. Shared planning, execution, analysis or dissemination of information can reduce overlap in programming and increase cooperation between different groups, many of whom may work more efficiently together than in isolation. 11. How do M&E of HIV/AIDS, Tuberculosis and Malaria fit into national health information systems? M&E toolkit, Draft 14.01.04 35
Building or strengthening national health information systems (NHIS) is a pre-requisite for proper monitoring of the three diseases and the response to them. Increased funding in the three disease areas creates an opportunity to strengthen not only programme or project specific health information, but also the health information and surveillance systems as a whole. HIV/AIDS, TB and malaria have different strengths related to the collection, dissemination, and use; opportunities exist for the three diseases to leverage each others strengths. An effective NHIS provides a solid basis for evaluations of large-scale programmes, ultimately leading to improved planning and decision-making. Urgent decisions such as how to allocate new resources to achieve the best overall result will become easier to make. Operational questions 1. How to select indicators from the core list provided in this toolkit? In deciding on a set of indicators, countries are not limited to the core list presented in this toolkit and should not necessarily collect all of them. The choice of indicators should be driven instead by the goals of the national programme or project. There is no point in collecting data on areas that are not relevant to the local context, bearing in mind that it costs time and money to collect and analyze data for each indicator. However, where they fit their needs, national programmes are encouraged to use the core indicators proposed in this toolkit to ensure standardization of information across countries and over time. The following guiding principles help in choosing the most appropriate set of indicators and associated data collection instruments: 1. 2. 3. 4. 5. 6. Use a conceptual framework for M&E for proper interpretation of the results (see above for suggested framework); Ensure that the indicators are linked to the programme or project goals and are able to measure change; Ensure that standard indicators are used to the extent possible for comparability between countries or population groups; Consider the cost and feasibility of data collection and analysis; and For HIV/AIDS, take into account the stage of the epidemic Keep the number of indicators to the minimum needed, with specific reference to the level of the system that require and will use which indicators to make programming and management decisions. Additional indicators can always be identified later.
2. Does planning data collection for selected indicators require different strategies? The cost, difficulty, and capacity required for collecting information increase as indicators shift from input through outputs and from outcome to impact.
36
Input and output data are often easy and cheap to collect. It should be possible to collect data for input and output indicators centrally from regular health monitoring systems, provided that such systems are functional . Programme planners should take strategic advantage of the increased attention to HIV/AIDS, TB, and malaria programmes to request funding for strengthening national health information and surveillance systems that can be used to report on all these as well as other disease-specific programmes. Data for many outcome and impact indicators are collected through more costly and difficult population-based or health facility surveys, requiring some expertise in research methods. Outcome measurement is usually more difficult in view of the sensitivity and specificity of each indicator. 3. How to capitalize on existing data collection efforts? In devising their data collection plans, countries should take into account to the extent possible: The timing of costly population-based surveys such as DHS in which modules can be included to obtain data on a number of indicators relevant to the three diseases; The existence of data already collected by agencies not directly involved in one of the three specific diseases, but that can help in monitoring,
4. How much from the total national programme budget should be allocated to M&E? Ensuring that resources are well used requires a coherent M&E system. It is, therefore, recommended that about 5-10 percent of the national programme budget are used for M&E. This percentage should be calculated taking into account external donor and national resources together. Also, between 3 percent and 5 percent of regional and district (where appropriate) financial resources should be devoted to M&E activities at those levels. Funders are increasingly realizing that project funds should be allocated to the development of an M&E system in order to assure that information related to the project can be collected, reported, and used. As a result, additional resources have become available as part of larger grants. This allows for the development of coherent systems rather than ad hoc efforts. 5. How to optimize the use of M&E funds? The following recommendations help ensure that M&E funds are properly invested: Develop systems rather than implement ad hoc data collection efforts. The initial cost is to be seen in light of the incremental benefit of more regular or more extensive data collection, ultimately resulting in a cheaper exercise.
37
Consider both short and long-term needs to ensure smooth continuity of national programmes. Mobilize key M&E players in the country through an M&E support group to avoid duplication of efforts. Use commonly agreed upon M&E frameworks for comparability purposes.
6. How to optimize the use of data? The ultimate goal of data collection is to ensure that data are fed back into the decisionmaking process. Data are powerful tools for advocacy, generating resources, and attributing changes to specific interventions and programming (or reorientation of programmes) where possible. Based on lessons learnt over the past years, the following steps help optimize the use of data: Produce quality data, requiring serious investment throughout the data collection process; Identify the different end-users, and present and package the data according to their needs, focusing on a minimal number of indicators at each level; Set up mechanisms for an efficient data-use system, including feedback through supervision at all levels, and assurances that data at a given level is relevant and actionable at that level. o Ensure government ownership throughout the data collection exercise, which means that national M&E capacities must be strengthened to guarantee uniform and quality data within a sustainable framework; o Ensure that an M&E support group with strong presence from the government, donor agencies, and academic institutions is established to guide the government throughout the development and implementation of national M&E strategies. This will improve the credibility of the data generated by the government; and o Allocate sufficient resources for the data-use plan. 7. How to avoid that donor demands drive all health information investments? To ensure that donor demands do not drive all health information investments with the risk of having different demands the following steps are recommended: Establish a platform under country leadership with strong donor involvement; Advocate for building a health information system that provides quality and timely information;
38
Use to the extent possible - commonly agreed upon M&E frameworks and standard indicators. Such frameworks are found in global M&E guidelines developed through a participatory process that involved M&E stakeholders from major donor agencies; In cases where two or more donors have multiple demands, refer to global guidelines to reconcile differences.
8. What are the key lessons learnt from successful M&E systems? 1. All implementing partners should collect complete input and output data. Many of them should collect process data. Far fewer should assess outcomes. Even fewer will assess impact. 2. Good M&E requires both internal self-assessment and external verification. Thus, while implementing partners should collect and verify their own internal data, an external agency should verify the completeness and accuracy of the data collected by those implementing partners. Supervisory visits should be based on the analysis of internal self-assessment and externally verified primary data 3. M&E systems must be as simple as possible. Most programmes and projects collect far more data than they use. The more complex an M&E system is, the more likely it will fail. 4. M&E systems must include a standardized core. If each implementing partner uses different systems or tools, the data cannot be analyzed or summarized effectively. The need for a standardized core does not preclude individual implementing partners from collecting additional situation-specific M&E data. 5. A specialized entity is required to collect, verify, enter and analyze primary M&E data from each partner. Without such an entity, data collection, verification and analysis are unlikely to happen. Ministries and other public agencies are seldom equipped to manage such a process. Increased resources devoted to HIV/AIDS, TB and malaria should be used build local capacity within such a national entity. 6. M&E must be built into the design of a programme and must beoperational when grant implementation begins, not added later. It is much harder and less effective to retrofit M&E after grant implementation is underway. 7. Sub national data are important for the national level data collection as they can be aggregated up to this level. However, subnational data are more relevant to programme managers in making day to day decisions. No matter how sound an M&E system may be, it will fail without widespread stakeholder buy-in. Thus, a large-scale, participatory process in the development and implementation of M&E strategies is essential to build ownership and buy-in from the start.
39
ANNEX A
Description of HIV/AIDS Indicators
40
PREVENTION INDICATOR (PI) 1:
YOUTH EDUCATION Schools with teachers trained in life-skills based HIV/AIDS education
Percentage of schools with teachers who have been trained in life-skills based HIV/AIDS education and who taught it during the last academic year.
RATIONALE V Schools that offer participatory and interactive life-skills training on individual, social and environmental factors that affect the risks of HIV transmission have proven to be more effective in bringing about behaviour change - delayed age at first sex, condom use, reduced number of sexual partners etc. - than more formal approaches that concentrate on information provision. The indicator provides useful information on trends in the coverage of life-skills based HIV/AIDS education within schools. DEFINITION OF INDICATOR Numerator: Number of schools with staff members trained in and regularly teaching life-skills based HIV/AIDS education Number of schools surveyed
Denominator: Note:
Analysis and reporting in percentage broken down by primary/secondary levels; public/private schools and combined is recommended
MEASUREMENT Principals/heads of a nationally-representative sample of schools (to include both private and public schools) are briefed on the meaning of life-skills based HIV/AIDS education and then are asked the following questions: 1. Does your school have at least one qualified teacher who has received training in participatory life-skills based HIV/AIDS education in the last 5 years? 2. If the answer to question 1. is yes: Did this person teach life-skills based HIV/AIDS education on a regular basis to each grade in your school throughout the last academic year? Platform: School survey or education programme review Frequency: Biennial REFERENCES
UNAIDS (2002) Monitoring the Declaration of Commitment on HIV/AIDS: Guidelines on Construction of Core Indicators. Geneva: UNAIDS. http://www.unaids.org/UNGASS/docs/JC718-CoreIndic_en.pdf
41
CONDOM DISTRIBUTION AND SALES Condoms available, retail

The proportion of randomly selected retail outlets and service delivery points that have condoms in stock at the time of a survey, of all retail outlets and service delivery points selected for survey.
RATIONALE This indicator reflects the success of attempts to broaden the distribution of condoms so that they are more widely available to people at locations and times when people are likely to need them. It measures actual distribution of condoms at designated points at any one point in time.
DEFINITION OF INDICATOR Number of retail outlets and service delivery points that have condoms in stock at the time of a survey Denominator: Total number of retail outlets and service delivery points that have been selected for the survey Numerator: Note: Sites in both urban and rural areas should be selected
MEASUREMENT A number of sites of different types (i.e. pharmacies, clinics, bars and clubs) are randomly selected for a retail survey from a standard checklist of venues where condoms should be accessible, including bars and night clubs, different classes of retail shops, STI clinics and other service provision points. While the indicator gives a single summary figure, the data can also be disaggregated by outlet type. Platform: Retail surveys (PSI protocol to evaluate social marketing programmes, WHO/GPA prevention indicator 3) Frequency: quarterly REFERENCES
UNAIDS/MEASURE (2000) National AIDS Programs: A guide to monitoring and evaluation. Geneva: UNAIDS. http://www.cpc.unc.edu/measure/guide/guide.html
42
PROGRAMMES FOR SPECIFIC GROUPS Injection Drug Users (IDUs) reached with prevention services
Percentage of injecting drug users who are reached with HIV/AIDS prevention services.
RATIONALE Providing services such as outreach, needle and syringe programmes and drug dependence treatment, including substitution therapy, to injecting drug users is essential, especially in countries with a significant or growing drug-related HIV epidemic. The purpose of this indicator is to estimate to what extent HIV/AIDS prevention services are provided to injecting drug users. DEFINITION OF INDICATOR Numerator: Number of regular injecting drug users, who were, in the past month, reached with (outreach) prevention services plus the number of injecting drug users in drug dependence treatment, either longer-term drug-free or substitution therapy Denominator: Estimated total number of injecting drug users Note:: Disaggregation by sex is recommended
MEASUREMENT N/a Platform: Programme monitoring (Service statistics from outreach projects and programmes, and treatment facilities for the numerator) Frequency: Biennial
REFERENCES
!
43
PROGRAMMES FOR SPECIFIC GROUPS Companies with HIV/AIDS workplace policies and programmes
Percentage of large enterprises/companies which have HIV/AIDS workplace policies and programmes.
RATIONALE The workplace is often a highly convenient and conducive setting for HIV control activities and workplace-based interventions have been proven to be effective. The indicator is useful even in countries where HIV prevalence is low because early action in educating workers on HIV prevention is essential if the serious economic and social consequences of HIV/AIDS are to be avoided.
DEFINITION OF INDICATOR Numerator: Number of employers with HIV/AIDS policies and regulations that meet all* criteria Denominator: Number of employers surveyed Note: Analysis and reporting by private/public sectors and combined is recommended MEASUREMENT Private sector employers are selected on the basis of the size of the labour force. Public sector employers should be the ministries of transport, labour, tourism, education and health. Employers are asked to state whether they are currently implementing personnel policies and procedures that cover a minimum of specified aspects (*see reference for details). Copies of written personnel policies and regulations should be obtained and assessed wherever possible. Platform: Survey of the 30 largest employers 25 private sector; 5 public sector Frequency: Biennial
REFERENCES
44
COUNSELLING AND VOLUNTARY TESTING Coverage of counselling and testing services

Percent of districts that have at least one centre staffed by trained counsellors providing specialized HIV counselling and testing services free or at affordable rates.
RATIONALE The coverage of quality VCT services will go a long way towards determining whether those services achieve their threefold aims of providing an entry point for care and support, promoting safe behaviour and breaking the vicious circle of silence and stigma. This indicator focuses particularly on coverage of specialised VCT services. DEFINITION OF INDICATOR Numerator: Number of districts that have at least one centre staffed by trained counsellors providing specialised HIV counselling and testing services free or at affordable rates
Denominator: Total number of districts Note: Analysis and reporting by district is recommended
MEASUREMENT Using key informants and health systems records of counsellor training, a list is constructed of all facilities offering counselling by trained counsellors and HIV testing services. Since price is a major part of accessibility, this should be considered in formulating this indicator. A suggested formula is: the price of voluntary counselling and HIV testing does not exceed one half of the daily minimum wage, or one half of the gross national product per person per day, calculated at purchasing power parity. Low or affordable prices may vary by district, and thus the measures should be adjusted. A further criterion is that the staff who provides counselling meet specified minimum national standards of training for counsellors. Facilities meeting the criteria for service provision, staff training and price are mapped by district or similar administrative unit. Platform: Health Systems records Frequency: quarterly REFERENCES

UNAIDS/MEASURE (2000) National AIDS Programs: A guide to monitoring and evaluation. Geneva: UNAIDS. http://www.cpc.unc.edu/measure/guide/guide.html UNAIDS (2004) National AIDS programmes. A guide to monitoring and evaluating HIV/AIDS care and support. Geneva : UNAIDS (in preparation).
45
COUNSELLING AND VOLUNTARY TESTING Population requesting testing and receiving results
Percent of people aged 15-49 surveyed who have ever voluntarily requested an HIV test, received the test and received their results.
RATIONALE This indicator aims to give an idea of the reach of HIV testing services in the general population and of the percentage of people who now know their HIV status. DEFINITION OF INDICATOR Numerator: Number of respondents having ever requested a test and received the results Denominator: Total number of respondents in the survey Note: Analysis and reporting by component and gender is recommended. It is suggested that data also be collected on those requesting an HIV test, receiving the test and receiving their results in the last 12 months MEASUREMENT In a general population or sub-population survey, respondents are asked whether they have ever requested an HIV test, whether they were tested and if so whether they have received the results. Additionally, it will be useful also to know the percentage of the population surveyed who have been tested and received the results in the last 12 months, a more time-sensitive measure. Platform: UNAIDS general population survey; DHS AIDS module; FHI adult BSS; youth BSS Frequency: every 2-3 years REFERENCES

UNAIDS/MEASURE (2000) National AIDS Programs: A guide to monitoring and evaluation. Geneva: UNAIDS. http://www.cpc.unc.edu/measure/guide/guide.html UNAIDS (2004) National AIDS programmes. A guide to monitoring and evaluating HIV/AIDS care and support. Geneva: UNAIDS (in preparation).
46
MOTHER TO CHILD TRANSMISSION Facilities with minimum package of services

The percentage of public, missionary, and workplace venues (family planning and primary health care clinics, ANC/MCH, and maternity hospitals) offering the minimum package of services to prevent HIV infection in infants and young children in the past 12 months.
RATIONALE This indicator provides critical information on the national availability of prevention and care efforts for women and infants. It is useful to programme planners in determining where services may be needed, or where facilities are providing the full spectrum of services to prevent HIV infection in women and infants. DEFINITION OF INDICATOR Numerator: Number of public, missionary, and workplace venues (family planning and primary health care clinics, ANC/MCH, and maternity hospitals) offering the minimum package of services to prevent HIV infection in infants and young children in the past 12 months. Denominator: All public, missionary, and workplace venues (family planning and primary health care clinics, ANC/MCH, and maternity hospitals) Note: Analysis and reporting by type of service is recommended
MEASUREMENT The information required for this indicator can be collected through a variety of different methods, and depends on resource availability as well as the amount of detail sought. It focuses on the minimum package of services which is defined by the type of clinical setting (see reference below). One option is to send a questionnaire to all public, missionary and workplace health facilities offering family planning and primary health care clinics, ANC/MCH, and maternity services. Another way to collect the relevant information is by adapting other instruments that already exist. Platform: Health facility surveys Frequency: 2-3 years REFERENCES
!
WHO (2004) National guide to monitoring and evaluating programmes for the prevention of HIV in infants and young children. Geneva: WHO (in preparation).
47
MOTHER TO CHILD TRANSMISSION ARV prophylaxis

Percentage of HIV positive pregnant women receiving a complete course of ARV prophylaxis to reduce MTCT in accordance with nationally approved treatment protocol (or WHO/UNAIDS standards) in last 12 months.
RATIONALE This indicator assesses the progress in preventing mother to child HIV transmission through the provision of ARV prophylaxis. DEFINITION OF INDICATOR Numerator: Number of HIV positive pregnant women receiving a complete course of ARV prophylaxis to reduce the likelihood of MTCT in accordance with nationally approved treatment protocol (or WHO/UNAIDS standards) in last 12 months. Denominator: Estimated number of HIV-infected pregnant women giving birth in last 12 months. Note: Brake down by type of service is recommended
MEASUREMENT The number of HIV-infected pregnant women provided with antiretroviral prophylaxis to reduce the risk of MTCT in the last 12 months is obtained from programme monitoring records. Only those women who completed the full course should be included The number of HIV-infected pregnant women to whom antiretroviral prophylaxis to reduce the risk of MTCT could potentially have been given is estimated by multiplying the total number of women who gave birth in the last 12 months (Central Statistics Office estimates of births) by the most recent national estimate of HIV prevalence in pregnant women (HIV sentinel surveillance antenatal clinic estimates). Programme monitoring records Central Statistics Office estimates of births Frequency: 2-3 years Platform: REFERENCES
48
SEXUALLY TRANSMITTED INFECTIONS DIAGNOSIS AND TREATMENT STI comprehensive case management
Percentage of patients with STIs at health care facilities who are appropriately diagnosed, treated and counselled.
RATIONALE The availability and utilization of services to treat and contain the spread of STIs can reduce the rate of HIV transmission within a population. One of the corner stones of STI control is comprehensive case management of patients with symptomatic STIs. This composite indicator reflects the competence of health service providers to appropriately provide these services, and the quality of services provided. DEFINITION OF INDICATOR Numerator: Number of STI patients for whom the correct procedures were followed on: (i) history taking; (ii) examination; (iii) diagnosis and treatment; and (iv) effective counselling on partner notification, condom use and HIV testing
Denominator: Number of STI patients for whom provider-client interactions were observed Note: Disaggregation by gender and for patients under and over 20 years of age Scores for each component of the indicator (i.e., history taking, examination, diagnosis and treatment, and counselling) must be reported as well as the overall indicator score MEASUREMENT Data are collected in observations of provider-client interaction at a sample of health care facilities offering STI services. Providers are assessed on history taking, examination, proper diagnosis and treatment of patients, and effective counselling including counselling on partner notification, condom use and HIV testing. Appropriate diagnosis and treatment and counselling procedures in any given country, are those specified in national STI service guidelines. Platform: Health facility survey based on WHO/UNAIDS revised guidelines on evaluating STI services and/or MEASURE service provision assessment (SPA) Frequency: biennial REFERENCES
UNAIDS (2002) Monitoring the Declaration of Commitment on HIV/AIDS: Guidelines on Construction of Core Indicators. Geneva: UNAIDS.
http://www.unaids.org/UNGASS/docs/JC718-CoreIndic_en.pdf
49
SAFE BLOOD Coverage of blood transfusion services

Percent of districts or regions with access to blood transfusion services which do not pay blood donors, and do not recruit donors from among relatives of the patient.
RATIONALE Many countries working to improve access to safe blood have established blood transfusion services including blood banks at the regional or district level, and are working systematically to enhance the recruitment of voluntary donors, and to reduce or eliminate reliance on blood donations from relatives and paid donors. This indicator assesses to what extent this has been implemented at the level dictated by national policy.
DEFINITION OF INDICATOR Numerator: Number of districts or regions with access to blood transfusion services which do not pay blood donors, and do not recruit donors from among relatives of the patient Denominator: Total number of districts or regions MEASUREMENT A district or region is considered to score positively on this indicator if at least 95% of blood transfused is supplied by a regional or provincial blood transfusion service that screens donors for risk behaviours and excludes donations from relatives and paid donors. Platform: MEASURE Evaluation Draft Blood Safety Protocol Frequency: quarterly REFERENCES
50
SAFE BLOOD Screening of blood units for transfusion

The percentage of blood units transfused in the last 12 months that have been adequately screened for HIV according to national or WHO guidelines.
RATIONALE Blood safety programmes aim to ensure that the overwhelming majority (ideally 100 percent) of blood units are screened for HIV, and those that are included in the national blood supply are indeed uninfected. This indicator gives an idea of the overall percentage of blood units that have been screened to high enough standards that they can confidently be declared free of HIV. DEFINITION OF INDICATOR Numerator: Number of blood units screened for HIV in the previous 12 months, and among those, the number screened up to WHO or national standards Denominator: Total number of blood units transfused in the previous 12 months Note: Brake down by components of the indicator is recommended MEASUREMENT The number of units transfused and the number screened for HIV should be available from health information systems. Quality of screening may be determined from a special study that re-tests a sample of blood previously screened, or from an assessment of the conditions under which screening occurred. In situations where this approach is not feasible, data on the percentage of facilities with good screening and transfusion records and no stockouts of test kits may be used to estimate adequately screened blood for this indicator. Platform: MEASURE Evaluation Draft Blood Safety Protocol Frequency: every 2-3 years REFERENCES
51
TREATMENT INDICATOR (TI) 1:
ANTIRETROVIRAL TREATMENT People on treatment

Percentage of people with advanced HIV infection receiving antiretroviral combination therapy
RATIONALE As the HIV pandemic matures, increasing numbers of people are reaching advanced stages of HIV infection. Antiretroviral combination therapy has been shown to reduce mortality amongst those infected and efforts are being made to make it more affordable even within less developed countries. Antiretroviral combination therapy should be provided in conjunction with broader care and support services including counselling for family caregiver. DEFINITION OF INDICATOR Numerator: Number of people with advanced HIV infection who receive antiretroviral combination treatment according to the nationally approved treatment protocol (or WHO/UNAIDS standards) Denominator: Number of people with advanced HIV infection Note: This indicator should be disaggregated by public/private services MEASUREMENT The numerator of this indicator is consists of the number of people receiving treatment at start of year plus the number of people who commenced treatment in the last 12 months minus the number of people for whom treatment was terminated in the last 12 months (including those who died). The number of people with advanced HIV infection is assumed to be 15% of the total number of people currently infected (for the purposes of this indicator). The latter is estimated using the most recent national sentinel surveillance data. The start and end dates of the period for which the number of people given antiretroviral therapy is given should be stated. Overlaps between reporting periods should be avoided wherever possible. Platform: Programme monitoring Frequency: Biennial REFERENCES
52
ANTIRETROVIRAL TREATMENT Health facilities capable of providing advanced level medical and psychosocial support services for HIV/AIDS
Percentage of facilities with the capacity and conditions to provide advanced level HIV care and support services, including provision of ART
RATIONALE This indicator measures the capacity of services specific to people living with HIV/AIDS. It is assumed that the systems and items measured in this indicator require substantial input and personnel training beyond what is routine for most health systems. DEFINITION OF INDICATOR Numerator: 1. Number of facilities with some components describing a list of advanced level services (see below for the list of services) 2. Number of facilities with all components for all services Denominator: 1. Total number of health facilities surveyed 2. Total number of facilities where identified services are offered or relevant Note: The specific components for each service should be presented individually MEASUREMENT The capacity to provide advanced level HIV/AIDS care includes: systems and items to support management of opportunistic infections and provision of palliative care for advanced care of clients with HIV/AIDS; systems and items to support advanced services for HIV/AIDS care; systems and items to support ART services; conditions to provide advanced inpatient care for clients with HIV/AIDS; conditions to support home care services; and post-exposure prophylaxis. Platform: Health facility surveys Frequency: every 2-4 years REFERENCES
UNAIDS (2004) National AIDS programs. A guide to monitoring and evaluating HIV/AIDS care and support. Geneva : UNAIDS (in preparation)
53
OPPORTUNISTIC INFECTIONS Health facilities capable of providing phrophylaxis and treatment for opportunistic infections (OIs)
Percentage of health facilities with the capacity and conditions to provide basic level HIV testing and HIV/AIDS clinical management
RATIONALE Many facilities that provide general curative care also provide services related to HIV/AIDS and are caring for HIV-infected clients. It is, therefore, essential to evaluate the status of existing capacity. DEFINITION OF INDICATOR Numerator: 1. Number of facilities with some components describing a list of basic services (see below for the list of services) 2. Number of facilities with all components for all services Denominator: 1. Total number of health facilities surveyed 2. Total number of facilities where identified services are offered or relevant Note: The specific components for each service should be presented individually MEASUREMENT The capacity to provide basic HIV counselling and medical services includes: a system for testing and providing results for HIV/AIDS; systems and qualified staff for pre- and post-test counselling; specific medical services relevant to HIV/AIDS including resources and supplies for providing these services; elements for prevention of nosocomial infections; trained staff and resources for providing basic interventions for prevention and medical treatment for HIV-infected persons. Platform: Health facility surveys Frequency: every 2-4 years REFERENCES
UNAIDS (2004) National AIDS programmes. A guide to monitoring and evaluating HIV/AIDS care and support. Geneva: UNAIDS (in preparation)
54
HIV/TB Intensified TB case finding among people living with HIV/AIDS

Proportion of clients attending HIV testing and counselling who test positive and who are screened for TB symptoms.
RATIONALE Identification of TB suspects is the first step in active case finding, and their prompt referral will promote early diagnosis and treatment of TB cases. In addition, TB symptom screening will help determine eligibility for TB preventive therapy. This indicator can be used for assessing intensified TB case finding in all situations where HIV counselling and testing is conducted or where PLWHA receive regular care and support, including HIV care clinics, inpatient medical services, VCT and PMTCT sites. DEFINITION OF INDICATOR Numerator: Number of HIV positive clients who are asked about TB and TB symptoms Denominator: Total number of HIV positive clients seen in each all situations where HIV care and support is provided or HIV counselling and testing is conducted In programmes where only HIV positive clients are screened: Numerator: Number of HIV-positive clients who are asked about TB and its symptoms Denominator: Total number of HIV-positive clients MEASUREMENT Data should be collected routinely at all HIV testing and counselling facilities (including those associated with PMTCT and within the private sector) and any situation where regular HIV care and support are provided. A suggested method of conducting the screening would be to ask patients whether they are currently on TB treatment. If not, they are then asked for some key symptoms of TB disease. By these questions all clients can be included in intensified TB case finding. Depending on local requirements the number of TB patients and TB suspects could also be collected from the same screen. Such screening may also form the basis of identifying HIV positive clients who have no evidence of active TB and would benefit from TB preventive therapy with isoniazid. Platform: Modified testing and counselling register Frequency: continuous data collection; reporting in quarterly returns
55
REFERENCES

WHO "A Guide to Monitoring and Evaluation for Collaborative TB/HIV Activities" , in preparation Interim Policy on Collaborative TB/HIV Activities. WHO/HTM/HIV/2004.1 and WHO/HTM/TB/2004.330
56
HIV/TB Counselling and voluntary testing for TB patients

Proportion of all TB patients who are tested for HIV and receive the results.
RATIONALE This indicator assesses to which extent staff involved in TB recognises the importance of HIV/AIDS testing and counselling and is able to act accordingly. The proportion of TB patients tested also gives an indication about the availability, accessibility and affordability of CT services and provides information necessary for targeting of resources, planning of activities and for monitoring the effectiveness of counseling and testing over time.
DEFINITION OF INDICATOR Numerator: Total number of TB patients who are tested for HIV (after having been offered VCT) and receive the results, over a given time period Denominator: Total number of tuberculosis patients registered over the same given time period MEASUREMENT The information will be collected from facility level, where each facility may have a specific recording system in place indicating whether the patient was referred for counseling and testing, taking into account measures of confidentially. The information of HIV test including HIV status should be recorded in the district TB register kept by the district TB co-ordinator. The reporting of the information will be incorporated into the existing TB reporting system. Platform: TB district register, quarterly reports, HIV laboratory Frequency: The data would be collected continuously and reported in quarterly cohorts
REFERENCES
!
57
HIV/TB Provision of cotrimoxazole preventive therapy

Percentage of HIV positive patients who are given cotrimoxazole preventive therapy.
RATIONALE Common HIV-related opportunistic infections contribute to the high mortality rates seen in HIV positive TB patients. A few studies from Sub-Saharan Africa have shown the benefit of cotrimoxazole preventive treatment (CPT) in reducing morbidity and mortality among HIV positive people on TB treatment. For this reason, CPT is recommended for HIV-positive adults and children living in Africa, and may be considered in other settings. The indicator measures the degree to which CPT is considered a component of the package of care offered to HIV positive patients with TB. DEFINITION OF INDICATOR Numerator: HIV-positive patients with TB who are given CPT Denominator: All HIV positive patients with TB who have been post-test counselled MEASUREMENT When HIV positive TB patients are provided CPT through the TB programme, a modified TB register should be used. A column can be added which can be ticked when CPT is given. These results should be reported at the completion of TB treatment in order to include all persons started on CPT over the course of their treatment. In cases in which CPT is provided through HIV treatment programmes, a referral system must be established to provide these results to the NTP at TB treatment completion on each person. In this case, a modified TB register (as described above) may still be used to collect this information once it has been reported back from HIV treatment programmes. Platform: Modified TB register, referral system to TB programme if collected by NACP Frequency: continuous data collection; reporting in quarterly returns at the end of TB treatment along with the outcome of TB treatment REFERENCES
Provisional WHO/UNAIDS Secretariat recommendations on the use of cotrimoxazole prophylaxis in adults and children living with HIV/AIDS in Africa (www.unaids.org/publications/documents/care/general/recommendations-eng.pdf)
58
59
HIV/TB Provision of ART

Proportion of HIV-positive TB patients who are given ART.
RATIONALE Experience from high and middle-income countries has shown that ART significantly improves the quality of life and enhances the survival of people with advanced HIV infection or AIDS. Since TB patients are one of the largest identifiable groups who are likely to be eligible for and benefit from ART, efforts should be undertaken to identify and treat eligible candidates. The indicator measures the degree to which ART is considered a component of the package of care offered to HIV positive patients with TB. DEFINITION OF INDICATOR Numerator: Registered TB patients who are HIV positive and given ART
Denominator: Registered TB patients who are HIV positive and have been post-test counseled MEASUREMENT In programmes where ART is offered through the TB programme, a modified TB register can be used by adding an extra column which can be ticked when an HIV-positive TB patient is given ART during the course of TB therapy. These results should be reported at the completion of TB treatment in order to include all persons started on ART over the course of their TB treatment. In cases in which ART is provided by HIV treatment programmes, a referral system must be established to provide these results to the NTP at TB treatment completion on each person. In this case, a modified TB register may still be used to collect this information once it has been reported back from HIV treatment programmes. Platform: Modified TB register with referral system (where appropriate) Frequency: continuous data collection; reporting in quarterly returns at the end of TB treatment along with the outcome of TB treatment REFERENCES

60
CARE AND SUPPORT (CS) 1:
ORPHANS AND VULNERABLE CHILDREN External support for households with orphans and vulnerable children
Percentage of orphans and vulnerable children whose households received, free of user charges, basic external support in caring for the child.
RATIONALE This indicator measures support coming from a source other than friends, family or neighbours (unless they are working for a community-based group or organization) given free of user charges to households with orphans and vulnerable children. DEFINITION OF INDICATOR Numerator: Number of orphans and vulnerable children residing in households that received: ! health care support within the past 12 months; ! emotional support within the past 3 months; ! school-related assistance within the past 12 months; ! other social support, including material support, within the past 3 months; and ! all four types of support Denominator: Total number of orphans and vulnerable children Note: Data should be analysed and reported by age (05, 69, 1014 and 1517 years) and gender when sample size allows. Depending on the epidemiological situation and available resources, programme managers may decide to aggregate age data into larger ranges (09, 1014 and 1517 years). MEASUREMENT As part of a household survey, household rosters can be used to identify all eligible orphans and vulnerable children (under 18 years of age). For each household with orphans and vulnerable children, a series of questions is asked about the types and frequency of support received and the primary source of the help. This survey tool may also be used in low-prevalence settings or targeted populations with similar but adapted methods. Platform: Household surveys Frequency: every 2-4 years REFERENCES
UNAIDS (2004) National AIDS programmes. A guide to monitoring and evaluating HIV/AIDS care and support. Geneva: UNAIDS (in preparation) 61
ANNEX B
Description of TB Indicators
62
IDENTIFICATION OF INFECTIOUS CASES New smear-positive TB case detection

Proportion of new smear-positive TB cases detected among the total estimated number of new semar-positive TB cases per year.
RATIONALE This indicator measures the DOTS programs ability to detect and identify smear-positive cases. If a country has low case detection, it should find alternative approaches to detecting new cases beyond the traditional methods. For example, the country should explore implementing DOTS in the private and NGO sectors as well as other areas where cases would be likely to present themselves. It is possible for the calculated detection rate to exceed 100 percent due to intense case finding in an area that has a backlog of chronic cases, over-reporting, over-diagnosis and the under-estimation of incidence. A case detection rate of 70% or greater is the global target. DEFINITION OF INDICATOR Numerator: Annual number of new smear-positive TB cases detected
Denominator: Total annual number of estimated new smear-positive TB cases (incidence) MEASUREMENT
The numerator is available from the TB Register or quarterly case detection reports. The denominator is estmation based on calculations by WHO from case notifications for each country and adjusted for countries with high HIV incidence. These estimations are reported every year by WHO in the annual Global Tuberculosis Control report.
Platform: TB Register; WHO estimates of incidence for countries Frequency: annually REFERENCES WHO. Compendium for monitoring TB control activities (in preparation)
63
PREVENTION BY TREATING INFECTIOUS CASES Smearconversion rate at end of 2 months of treatment

Percent of new smear-positive TB cases registered in a specified period that converted to smear-negative at the end of 2 months of treatment.
RATIONALE Sputum smear conversion after two months of treatment is a good predictor of eventual cure if treatment is completed. This indicator also has treatment implications since in some countries patients who have not converted their sputum smears after two months of treatment should extend the intensive phase of therapy. This indicator is useful for following trends within a country or region and for comparison between centers.
DEFINITION OF INDICATOR Numerator: Number of new smear positive pulmonary TB cases registered in a specified period that are smear negative at the end of 2 months of treatment Denominator: Total number of new smear positive pulmonary TB cases that were evaluated for smear conversion in the same period MEASUREMENT The numerator is the number of new smear positive pulmonary TB patients registered in a specified period (e.g., quarter or year) that had at least one negative smear result at the end of two months of treatment (intensive phase). This number can be obtained from the TB Register. Similarly, the denominator is the total number of new smear positive pulmonary TB cases registered for treatment during the same period and can also be obtained from the TB Register. Platform: TB Register Frequency: quarterly and annual REFERENCES ! WHO. Compendium for monitoring TB control activities (in preparation)
64
PREVENTION OF TB AMONG TB/HIV INFECTED PEOPLE Provision of isoniazid preventive therapy

Proportion of HIV positive clients given TB preventive therapy.
RATIONALE The risk of developing TB is significantly increased in PLHA. TB preventive therapy will reduce the incidence of active TB in PLHA. This indicator provides information on one of the main objectives for collaborative TB/HIV activities. DEFINITION OF INDICATOR Numerator: Number of clients given at least the first dose of TB preventive therapy Denominator: Total number of HIV-positive clients eligible for TB preventive therapy MEASUREMENT The data needed for this indicator will be collected at all HIV testing and counselling facilities (including those associated with PMTCT) or at HIV care services depending on where TB preventive therapy is to be administered. In these different situations clients will be screened for TB. Those clients found NOT to have TB will be offered TB preventive therapy according to locally determined guidelines. In programmes which need more complete data for accurate prediction of drug usage and supplies a reporting mechanism similar to that which is used for TB will need to be used. Staff would need to record attendance (usually monthly) for further drug supplies, and report new cases, continuing cases and completed cases on a quarterly basis. The indicator of choice would be the proportion of HIV-positive clients completing TB preventive therapy. Platform: TB preventive therapy register Frequency: continuous data collection; reporting in quarterly returns REFERENCES
WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities (in preparation)
65
TB/HIV Surveillance of HIV seroprevalence among all TB patients

Percent of all tuberculosis patients who are HIV positive.
RATIONALE Measuring HIV prevalence among tuberculosis patients will inform about the degree of overlap in the epidemics in any given setting as well as the contribution of HIV to the TB epidemic in any given setting. Estimating the prevalence of HIV among TB patients is an important step in planning TB control activities, planning and targeting integrated TB-HIV activities and monitoring the effectiveness of these activities over time. DEFINITION OF INDICATOR Numerator: Total number of all registered tuberculosis patients who are HIV positive, over a given time period Denominator: Total number of all tuberculosis patients registered over the same given time period MEASUREMENT Ideally all newly registered tuberculosis patients should be considered for HIV surveillance. However, if surveys or sentinel methods are used and resources are limited, countries may choose to focus only on adult smear positive pulmonary patients. Countries with scarce resources where the HIV epidemic state is either low or concentrated may also chose to only include patients between the ages of 15 and 59 years. Relapse cases should be excluded from surveillance systems, because of the risk of surveying the same patient twice, unless they are identified as such and the results are analysed separately. However, relapse cases may be included and need not be identified as such, if surveillance is based on survey methods and these surveys are undertaken over a short period of time, ideally less than 2 -3 months. Periodic surveys have a specific role where the prevalence of HIV among tuberculosis patients has not been previously estimated. Surveys using representative sampling methods and appropriate sample sizes can provide accurate estimates of the burden of HIV upon the tuberculosis situation and are an essential part of the initial assessment of the situation. This information may alert tuberculosis programmes to a potential HIV problem and enable action to be taken, which may include the institution of more systematic surveillance. Platform: Special surveys or sentinel surveillance Frequency: In the absence of a national recording and reporting system where data should be continuously collected and reported quarterly, data should be collected every 2-3 years. REFERENCES WHO. Revised guidelines for the surveillance of HIV among people with tuberculosis (in preparation)
66
TIMELY DETECTION AND TREATMENT Population covered by DOTS

Proportion of the population with geographic access to DOTS.
RATIONALE This indicator measures the availability of DOTS within a country. The target is to make DOTS available to 100% of the population. DEFINITION OF INDICATOR Numerator: Number of people living within the catchment area of facilities that implement DOTS as per national guidelines Denominator: Total population MEASUREMENT For a programme review, the numerator will be the number of people living within the catchment are of facilities that implement DOTS, per NTP guidelines, and the denominator will be the total population. For routine monitoring, the data should be included in quarterly reports. Platform: NTP annual report; quarterly and annual reports submitted from intermediate to central level of NTP Frequency: This indicator should be measured on an annual basis for the purposes of external monitoring, and quarterly for routine monitoring by the NTP.
REFERENCES World Health Organization (2002). Global Tuberculosis Control: WHO Report 2003. http://www.who.int/gtb/publications/globrep/index.html
67
TIMELY DETECTION AND TREATMENT Smear- positive TB cases registered under DOTS successfully treated
Percent of new smear positive pulmonary TB cases registered under DOTS in a specified period who were cured plus the percent who completed treatment.
RATIONALE The cure rate and treatment completion rate indicators should be added together in order to determine the treatment success over a specified period. Evaluation of treatment outcomes of new pulmonary smear-positive patients is used to determine the quality and effectiveness of DOTS implementation. When cure cannot be established, treatment completion is the best means of ensuring patients have been adequately treated, and in the absence of confirmatory documentation, are likely to be cured. A treatment success rate of 85 percent or greater is the global target. This indicator is useful for following trends within a country or region and for cross-country comparisons and can be used to monitor and evaluate the impact of specific interventions DEFINITION OF INDICATOR Numerator: Number of new smear positive pulmonary TB cases registered under DOTS in a specified period that were cured plus the number that completed treatment Denominator: Total number of new smear positive pulmonary TB cases registered under DOTS in the same period MEASUREMENT At the end of the treatment course, each sputum smear positive TB case is assigned a treatment outcome, which is recorded in the TB Register. The numerator for this indicator is the number of patients registered in a specified period (e.g., quarter or year) and recorded with the treatment outcome treatment complete and cured. This number can be obtained from quarterly treatment outcome reports or the TB Register. The denominator can also be obtained from quarterly treatment outcome reports or the TB Register. Platform: TB Register; quarterly reports of treatment outcomes (TB-08) Frequency: quarterly and annual basis REFERENCES
!
WHO. Compendium for monitoring TB control activities (in preparation)
68
CONTROL OF DRUG RESISTANCE New smear-positive cases who default or transfer out of treatment
Percent of new smear positive pulmonary TB cases registered in a specified period that interrupted treatment for more than two months in the past year or who were transferred to another basic management unit for which there is no treatment outcome information.
RATIONALE Evaluation of treatment outcomes of new pulmonary smear-positive patients is used to determine NTP quality and effectiveness. One of the overall objectives and the highest priority of TB control is to cure cases of infectious TB. For patients that are not classified as cured, it is essential to determine to which treatment outcome they are assigned so that appropriate interventions can be designed and implemented. DEFINITION OF INDICATOR Numerator: Number of new smear positive pulmonary TB cases registered in a specified period that interrupted treatment for more than two months in the past year or who were transferred to another basic management unit for which there is no treatment outcome information Denominator: Total number of new smear positive pulmonary TB cases registered in the same period MEASUREMENT At the end of the treatment course, each sputum smear positive TB case is assigned a treatment outcome, which is recorded in the TB Register. Patients whose treatment was interrupted for two or more consecutive months (e.g., patients that did not collect drugs for 2 or more months any time after registration) are designated as default. Patients who were transferred to another basic management unit for which there is no treatment outcome information are classified as having transferred out. Platform: Quarterly reports of treatment outcomes (TB-08) Frequency: quarterly and annual basis REFERENCES WHO. Compendium for monitoring TB control activities (in preparation)
69
MONITORING OF PERFORMANCE IN CASE MANAGEMENT Completeness of reporting to national tuberculosis programme

Proportion of units submitting case finding and treatment outcome reports to the NTP each quarter.
RATIONALE This indicator measures if the NTP receives the essential data necessary for programme management. The on-going systematic recording, analysis, interpretation and reporting of TB data will facilitate planning, implementation, and evaluation of the NTP and related public health programmes.
DEFINITION OF INDICATOR Numerator: Number of units that submitted case finding and treatment outcome reports to the NTP in the previous quarter* Denominator: Total number of units required to submit case finding and treatment outcome reports to the NTP each quarter Note: *A unit is included in the numerator only if it submits both reports to the NTP It is recommended to separate the indicator into levels of reporting MEASUREMENT This indicator measures the completeness and timeliness of TB report submission, which is essential for efficient programme management since it provides the data to evaluate TB programme targets, guide efforts to allocate staff, and to monitor results. This indicator is measured at the central health level in a country on a quarterly basis and should be collected for the most recent reporting period for monitoring purposes. Platform: NTP Statistics and Reports Frequency: quarterly, unless the NTP guidelines for recording and reporting specify another timeframe REFERENCES WHO. Compendium for monitoring TB control activities (in preparation)
70
CARE AND SUPPORT INDICATOR (CS) 1:
SUPPORT THROUGH DIRECT OBSERVATION OF TREATMENT Patients under direct observation of therapy (DOT)
Proportion of TB patients whose therapy was directly observed by a trained, regularly supervised individual according to NTP guidelines*.
RATIONALE This indicator measures an essential element of the DOTS strategy: direct observation of therapy (DOT) to ensure patient and provider adherence to treatment. WHO recommends that a health care worker or trained and regularly supervised person observe the patient swallowing each dose of medicine and record the dose on the individual treatment card throughout the intensive phase of treatment. Each facility should attempt to achieve 100 percent on this indicator in order to comply with international guidelines and prevent drug resistance. DEFINITION OF INDICATOR Numerator: Number of new pulmonary smear positive TB patients who report observation of every dose of medication per NTP guidelines Denominator: Total number of new pulmonary smear positive TB patients interviewed regarding direct observation of therapy Note: *NTP guidelines should specify direct observation of therapy for at least the first two months of treatment. In some countries, the guidelines may specify direct observation for the full course of treatment if rifampin is used in the continuation phase. MEASUREMENT It is recommended that multiple sources be used to determine the numerator value: 1) In exit interviews, the patient should be asked if a health worker or treatment supervisor has observed every dose of medication, or alternatively, if s/he can recall any time when treatment was not directly observed. 2) Observation of patient-provider interaction. 3) Review of treatment cards to verify if every dose was recorded for each patient registered for treatment during the quarter. If patients are hospitalized during the intensive phase, the same methods should be used to determine who receives DOT. Platform: Routine measurement, or as part of a special facility level survey assessing clinic performance Frequency: annual for the purposes of external monitoring, and quarterly for routine monitoring by the NTP REFERENCES ! WHO. Compendium for monitoring TB control activities (in preparation)
71
SUPPORTING ENVIRONMENT INDICATOR (SE) 1:
DRUGS AND LABORATORY SUPPLY TB drugs out of stock treatment facilities

Average percentage of time that first-line TB drugs are not available in treatment facilities.
RATIONALE The availability of medication is critical to the successful management of tuberculosis, and an uninterrupted supply of drugs at treatment centres is crucial to cure patients and to avoid the emergence of drug resistant strains of tuberculosis. This indicator measures a key DOTS strategy component, uninterrupted drug supply. DEFINITION OF INDICATOR Numerator: Total number of stock out days for all first-line drugs stocked x 100 Denominator: 365 MEASUREMENT Data should be collected from as many treatment facilities as possible. Collection from 20 established sentinel sites is ideal. To calculate this indicator record the number of days each drug was out of stock last year (or last 12 months) and sum the total number of days out of stock for all drugs. Then divide the number of days by 365 times the total number of drugs normally stocked. Multiply the fraction by 100. Platform: Health centre drug stock cards Frequency: quarterly REFERENCES
World Health Organization (1999). Indicators for Monitoring National Drug Policies. WHO/EDM/PAR/99.3.
72
SUPPORTING ENVIRONMENT INDICATOR (SE) 2:
HUMAN RESOURCE CAPACITY Health facilities and laboratories with sufficient capacity for DOTS
Proportion of diagnostic centers with at least one laboratory technician trained in acid-fast bacilli microscopy in the last three years.
RATIONALE One of the five components of DOTS is the use of smear microscopy to diagnose pulmonary TB. Trained individuals, along with adequate laboratory capacity and supplies, are critical to the provision of these services for the NTP. Thus, the indicator is important for measuring the human resources input for this critical DOTS component, hence whether or not the NTP has the minimum human resources required to carry out diagnosis by sputum microscopy throughout the network of diagnostic centers. The NTP should work towards achieving 100 percent on this indicator or at least an increasing trend over time. DEFINITION OF INDICATOR Numerator: Number of diagnostic centers with at least one laboratory technician trained in acid-fast bacilli microscopy in the last three years* Denominator: Total number of diagnostic centers providing smear microscopy in the country Note: *This number should include new technicians who received their initial training in AFB microscopy within the last three years AND technicians who received refresher training during the same period. MEASUREMENT Centers with at least one full time technician who was trained in AFB microscopy in the last three years are included in the denominator. Given that technicians may not use the necessary skills every day and often need refresher training to maintain them, the upper limit at which the most recent training should have occurred is three years. All diagnostic centers utilized by the NTP should be included in the denominator. Platform: NTP training records; list of certified laboratory technicians and laboratory of employment Frequency: annual REFERENCES World Health Organization (2003). Management of Tuberculosis Training for health facility staff, http://www.who.int/gtb/publications/training/management_of_tb/pdf/who_cds _tb_2003_314i.pdf
73
ANNEX C
Description of Malaria Indicators
74
INSECTICIDE TREATED NETS Households owning ITNs

Proportion of households with at least one insecticide-treated net.
RATIONALE ITNs have been shown to be associated with reductions in all-cause child mortality, malaria-related morbidity, and low birth weight, within malaria endemic areas of subSaharan Africa. There is also some evidence of a community effect where promptly treated ITNs are associated with reductions in all-cause child mortality and malariarelated morbidity among unprotected children within close proximity to households with ITNs. In addition, there is evidence of a correlation between ownership and usage of nets. This indicator captures household ITN possession among the general population at the national level.
DEFINITION OF INDICATOR Numerator: Number of households surveyed with at least one mosquito net, which has been treated with approved insecticide within the last 6 months Denominator: Total number of households surveyed Note: Analysis and reporting by province and according to urban/rural setting is recommended MEASUREMENT This indicator requires data collected at the household level from nationallyrepresentative sample surveys. The limited number of questions required to ascertain the data for this indicator can be easily added to any nationally-representative sample survey of households. It is important that these data be collected on a household questionnaire, rather than from an individual, as individuals may not be representative of household possession. It is also important that surveys be conducted with sufficient design and sample size to allow comparisons between provinces and urban/rural strata at the household level. The numerator for this indicator is obtained from asking household respondent if there is any mosquito net in the house that can be used to avoid being bitten while sleeping, and whether it has been treated in the last 6 months. The denominator is simply measured by the total number of surveyed households. Suggested questions: 1.1, 1.3 and 1.4 from malaria add on household questions in
75
Guidelines for core indicators for assessing malaria intervention coverage from household surveys. Platform: DHS (USAID/MACRO), MICS (UNICEF), MIS, Rider on other nationally representative surveys Frequency: every 2-3 years REFERENCES
!
! ! !
Roll Back Malaria/MEASURE. Guidelines for core indicators for assessing malaria intervention coverage from household surveys. 2004. In preparation. Roll Back Malaria. Malaria Indicator Survey (MIS). In preparation UNICEF MICS: http://www.unicef.org/reseval/micsr.html MEASURE Demographic and Health Surveys: http://www.measuredhs.com/
76
INSECTICIDE TREATED NETS Children under five using ITNs

Proportion of children <5 years old who slept under an ITN the previous night.
RATIONALE The use of ITNs within areas of intense transmission are of particular importance as their effect on reducing all-cause mortality and malaria-related morbidity is concentrated among young children. For these reasons coverage of children with ITNs is a key component of the technical strategy for transmission prevention and vector control advocated by RBM. This indicator captures the level of ITN use by children <5 years of age at the national-level. DEFINITION OF INDICATOR Numerator: Number of children under five years old who slept under a mosquito net the previous night, which has been treated with approved insecticide within the last 6 months Denominator Total number of children under five years old who slept in surveyed households the previous night Note: Analysis and reporting by province and according to urban/rural setting is recommended MEASUREMENT This indicator requires data collected from nationally-representative household sample surveys. The limited number of questions required to ascertain the data for this indicator can be easily added to any nationally-representative sample survey. However, it is important that the survey contain a household listing that captures all children under five years old within each surveyed household. Such surveys should be conducted with sufficient design and sample size to allow comparisons between provinces and urban/rural strata at the individual level. ! The data for the denominator is obtained during the household listing procedure when every child under five who slept in the house the previous night is identified. The data for the numerator is then obtained from a listing of children in the house who slept under a mosquito net the previous night, in combination with information on whether the net had been treated with insecticide within the last 6 months
Suggested questions: 1.1 and 1.3-1.6 from malaria add on household questions in Guidelines for core indicators for assessing malaria intervention coverage from household surveys
77
Platform: DHS (USAID/MACRO), MICS (UNICEF), MIS, Rider on other nationally representative surveys Frequency: every 2-3 years REFERENCES
!
! ! !
Roll Back Malaria/MEASURE. Guidelines for core indicators for assessing malaria intervention coverage from household surveys. 2004. In preparation. Roll Back Malaria. Malaria Indicator Survey (MIS). In preparation. UNICEF MICS: http://www.unicef.org/reseval/micsr.html MEASURE Demographic and Health Surveys: http://www.measuredhs.com/
78
MALARIA IN PREGNANCY Pregnant women using ITNs

Proportion of pregnant women who slept under an ITN the previous night.
RATIONALE ITN use by pregnant women has been shown to be associated with reductions in malariarelated maternal morbidity, as well as improved birth outcomes, including the reduction of low birth weight babies. For these reasons coverage of pregnant women with ITNs is a key component of the technical strategy for control and prevention of malaria in pregnancy advocated by RBM. This indicator captures the level of ITN use by pregnant women at the national-level. DEFINITION OF INDICATOR Numerator: Number of pregnant women who slept under a mosquito net the previous night, which has been treated with approved insecticide within the last 6 months Denominator: Total number of pregnant women who reside within surveyed households Note: Analysis and reporting by province and according to urban/rural setting is recommended MEASUREMENT This indicator requires data collected from nationally-representative household sample surveys. The limited number of questions required to ascertain the data for this indicator can be easily added to any nationally-representative sample survey. However, due to small number of currently pregnant women at any given time, a survey designed to collect these data should have an overall sample of 5000 women (in order to be comparable with MICS and DHS). If questions are to be added on as a rider to a survey, it is important that the survey contain a household listing that captures all women of reproductive age within each surveyed household. Such surveys should be conducted with sufficient design and sample size to allow comparisons between provinces and urban/rural strata at the individual level. The data for the denominator is obtained from a series of questions asked of all women of reproductive age in the household about their current pregnancy status. The data for the numerator is then obtained from a listing of these women that slept under a mosquito net the previous night, in combination with information on whether the net had been treated with insecticide within the last 6 months. Suggested questions: 1.1, 1.3-1.6, and 2.1 from malaria add on household questions in
79
Guidelines for core indicators for assessing malaria intervention coverage from household surveys Platform: DHS (USAID/MACRO), MICS (UNICEF), MIS, Rider on other nationally representative surveys Frequency: every 2-3 years REFERENCES
!
! ! !
80
MALARIA IN PREGNANCY Pregnant women receiving Intermittent Preventive Therapy (IPT)

Proportion of pregnant women who receive IPT as prophylaxis for malaria.
RATIONALE IPT of sulphadoxine-pyrimethamine (SP) given to pregnant women has been shown to reduce the risk of maternal anemia, placental parasitemia, and low birth-weight. IPT in pregnancy is therefore a key component of the technical strategy for control and prevention of malaria in pregnancy advocated by RBM. This indicator captures the national-level use of IPT to prevent malaria among pregnant women. DEFINITION OF INDICATOR Numerator: Number of women who took an antimalarial drug treatment to prevent malaria during their last pregnancy that led to a live birth within the last 2 years Denominator: Total number of women surveyed who delivered a live baby within the last 2 years Note: Analysis and reporting by province and according to urban/rural setting is recommended MEASUREMENT This indicator requires data collected from nationally-representative household sample surveys. The limited number of questions required to ascertain the data for this indicator can be easily added to any nationally-representative sample survey. If questions are to be added on as a rider to a survey, it is important that the survey contain a household listing that captures all women of reproductive age within each surveyed household as well as a female questionnaire to collect data on previous births and antenatal care. Additionally, due to the limited number of women who delivered a live baby within the previous 2 years, care should be taken to ensure such surveys are conducted with sufficient sample size and design to allow comparisons between provinces and urban/rural strata at the individual level. Data from the female questionnaires for all women who delivered a live baby within the last 2 years within surveyed household is used to calculate the denominator. The numerator is derived from the number of women who mention taking an antimalarial for prevention (NOT treatment) during pregnancy from among all women who have given birth in the last 2 years. It is important to differentiate between a treatment dose for prevention (as prescribed for IPT) and actual treatment of an existing malaria infection. Although it is extremely difficult to differentiate in the context of a survey interview, the latter is
81
curative care, and does not count as standard IPT procedure. Similarly, women taking weekly cholorquine prophylaxis are not considered to be covered by IPT. Suggested questions: 2.2-2.7 from malaria add on household questions in Guidelines for core indicators for assessing malaria intervention coverage from household surveys Platform: DHS (USAID/MACRO), MICS (UNICEF), MIS, Rider on other nationally representative surveys Frequency: every 2-3 years REFERENCES
!
! ! !
82
PREDICTION AND CONTAINMENT OF EPIDEMICS Malaria epidemics detected and properly controlled
Proportion of epidemics detected within 2 weeks of onset and properly controlled.
RATIONALE With an increasing occurrence of epidemics in both low-risk areas and areas of moderate transmission of malaria, the institution of special responses to epidemics on top of the regular malaria control activities is imperative. The impact of epidemics can be greatly reduced if they are timely detected or, even better, predicted, and prevention started. This indicator captures the national response to epidemics. DEFINITION OF INDICATOR Numerator: Number of epidemics detected in a specific geographical area (country, district) within two weeks during the last 12 months and for which appropriate control measures* have been initiated Denominator: Number of malaria epidemics recorded during the last 12 months within a specific geographical area Note: * Action based on preparedness plan of action, according to global WHO guidelines, where applicable MEASUREMENT The Management survey forms are designed for collating general policy and implementation guidelines with emphasis on management issues. Thy should be administered to the National Programme Officers and District Health Managers. There are several scenarios that can be adopted to facilitate easy collection of the information. The survey forms must not be sent out as questionnaires. The interview teams are expected to ask additional questions, clarifying issues during the interview. Platform: Management Survey Frequency: 2-3 years REFERENCES
!
Hook C. Field Buide for Malaria Epidemic Assessment and Reporting. DRAFT for Field Testing. World Health Organization. 2003, Available online: http://rbm.who.int/cmc upload/0/000/016/569/FTest.pdf Roll Back Malaria. Framework for Monitoring Progress and Evaluating Outcomes and Impact. 2000. Geneva.
83
http://rbm.who.int/cmc_upload/0/000/012/168/m_e_en.pdf WHO. Roll Back Malaria Initiative in the African Region. Monitoring and Evaluation Guidelines. Harare, WHO Regional Office, 2000.
84
PROMPT EFECTIVE TREATMENT Children under five years of age with access to treatment
Proportion children under five with fever in last 2 weeks who received antimalarial treatment according to national policy within 24 hours from onset of fever.
RATIONALE The majority of deaths from severe malaria in childhood are caused by the delayed administration of effective antimalarial treatment. Prompt access to effective malaria treatment among children is therefore a key component of the technical strategy for control and prevention of malaria in pregnancy advocated by RBM. This indicator captures the national-level access to prompt and effective treatment for malaria.
DEFINITION OF INDICATOR Numerator: Number of children <5 years old who had a fever in previous 2 weeks who received antimalarial treatment according to national policy <24 hours from onset of fever. Denominator: Total number of children <5 years old who had a fever in previous 2 weeks Note: Analysis and reporting by province and according to urban/rural setting is recommended MEASUREMENT In order to collect data on this indicator, the survey must be nationally representative and collect data on children <5 years old. The child-level data are obtained during the household listing procedure when every child <5 who slept in the house the previous night is identified. Questions are asked about whether the child has had a fever in the past two weeks, and if and where s/he was given antimalarial treatment. At a minimum, the following data will also need to be collected to assist with interpretation and control for potential confounding: ! Age of listed children in years ! Confirmation of type of malaria retreatment given to child ! Socioeconomic variables at the household and community levels routinely collected by DHS and MICS Platform: DHS (USAID/MACRO), MICS (UNICEF), MIS, Rider on other nationally representative surveys Frequency: 2-3 years
85
REFERENCES
!
! ! !
Roll Back Malaria/MEASURE. Guidelines for core indicators for assessing malaria intervention coverage from household surveys. 2004. In preparation. Roll Back Malaria. Malaria Indicator Survey (MIS). In preparation. UNICEF MICS: http://www.unicef.org/reseval/micsr.html MEASURE Demographic and Health Surveys: http://www.measuredhs.com
86
PROMPT EFECTIVE TREATMENT Health facilities with no reported stock out

Percentage of health facilities with no stock outs of nationally recommended antimalarial drugs continuously for one week during the last 3 months
RATIONALE The continuos supply of antimalarial drugs is key to prompt effective treatment at health facilities. This indicator captures the availability of nationally recommended antimalarial drugs in health facilities.
DEFINITION OF INDICATOR Numerator: Number of health facilities with nationally recommended antimalarial drugs on the day of survey and with no stock outs in the last 3 months Denominator: Total number of Health facilities surveyed
Note: Analysis and reporting by province and according to urban/rural setting is recommended MEASUREMENT The health facility survey forms are administered to the head of each section of the health facilities identified for the survey. They are expected to include the District hospital, and at least two other Health centres/posts serving selected communities in the district. Examination of in-patient records and an assessment of the appropriateness of treatment of severe malaria cases admitted to the District Hospital as well as observation of health care providers providing services to clients should be part of data collection. Platform: Health Facility Survey Frequency: yearly REFERENCES
!
Roll Back Malaria. Framework for Monitoring Progress and Evaluating Outcomes and Impact. 2000. Geneva. http://rbm.who.int/cmc_upload/0/000/012/168/m_e_en.pdf WHO. Roll Back Malaria Initiative in the African Region. Monitoring and Evaluation Guidelines. Harare, WHO Regional Office, 2000.
87
TREATMENT INDICATOR(TI) 3:
CASE MANAGEMENT Patients with severe malaria receiving correct treatment

Percentage of children under five admitted with severe malaria and correctly treated at health facilities.
RATIONALE Severe malaria usually occurs as a result of a delay in treating uncomplicated malaria. Sometimes, especially in children, severe malaria may develop very rapidly. Because of the often fatal outcome, the correct management is key to saving lives. This indicator captures the ability of health facilities to correctly manage severe malaria.
DEFINITION OF INDICATOR Numerator: Number of children under five and other target groups admitted with severe malaria and correctly given antimalarials and supportive treatment according to national policy Denominator: Total number of children under five and other target groups admitted with severe malaria surveyed at health facilities MEASUREMENT The health facility survey forms are administered to the head of each section of the health facilities identified for the survey. They are expected to include the District hospital, and at least two other Health centres/posts serving selected communities in the district. Examination of in-patient records and an assessment of the appropriateness of treatment of severe malaria cases admitted to the District Hospital as well as observation of health care providers providing services to clients should be part of data collection. Platform: 1) Part of routine supervision of NMCP; 2) Health facility survey (Form 7) Frequency: 1) yearly; 2) every 2-3 years REFERENCES
!
Roll Back Malaria. Framework for Monitoring Progress and Evaluating Outcomes and Impact. 2000. Geneva. http://rbm.who.int/cmc_upload/0/000/012/168/m_e_en.pdf WHO. Roll Back Malaria Initiative in the African Region. Monitoring and Evaluation Guidelines. Harare, WHO Regional Office, 2000.
88

3 M E Toolkit 2004

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

3 M E Toolkit 2004

Uploaded by

Copyright:

Available Formats

MONITORING AND EVALUATION TOOLKIT

HIV/AIDS, TUBERCULOSIS AND MALARIA

World Health Organization

M&E toolkit, Draft 14.01.04

Why this kit?

What are its contents?

Basic elements of M&E

M&E toolkit, Draft 14.01.04

DATA COLLECTION & ANALYSIS

M&E toolkit, Draft 14.01.04

Overall framework for M&E

PROCESS (human resources, training, commodities)

M&E toolkit, Draft 14.01.04

Area Drugs and commodities

Service delivery, technologies

OUTPUTS (services, numbers reached, coverage)

Knowledge, skills and practice

Target population knows about the benefit of the intervention

M&E toolkit, Draft 14.01.04

Area People on treatment, people benefiting from intervention

Key questions A majority of target population is covered by the intervention

OUTCOMES (changed behaviours, coverage)

IMPACT (biology and quality of life)

Levels of Monitoring and Evaluation

M&E toolkit, Draft 14.01.04

HIV/AIDS Information, Education, Communication (IEC) Youth Education

MALARIA Insecticide-treated nets (ITNs)

Prediction and Containment of epidemics Indoor Residual Spraying

Information, education & communication (IEC)

M&E toolkit, Draft 14.01.04

Prophylaxis and treatment for opportunistic infections HIV/TB

Monitoring of drug resistance

Home based management of malaria

Coordination and partnership development (national, community, public-private)

M&E toolkit, Draft 14.01.04

Reduced Malaria specific morbidity

M&E toolkit, Draft 14.01.04

M&E toolkit, Draft 14.01.04

Summary table for HIV/AIDS

Prevention Condom distribution Programmes for specific groups

M&E toolkit, Draft 14.01.04

Service Delivery Area

Output Mobile populations exposed to outreach programmes* (number and percentage**)

Health facilities offering minimum package of PMTCT* (PI7)

M&E toolkit, Draft 14.01.04

Service Delivery Area Antiretroviral treatment(AR T) and monitoring

Prophylaxis and treatment for opportunistic infections (OIs) HIV/TB

Provision of cotrimoxazole preventive therapy and/or ART (TI 6 and TI 7)

Support for orphans

Care and Support

Support for the chronically ill

Strengthening of Civil Society Stigma

Monitoring, evaluation, and operational research

M&E toolkit, Draft 14.01.04

Output Number of patients who are accurately referred*

Number of networks/partnerships involved*

Unit costs of drugs and commodities

M&E toolkit, Draft 14.01.04

M&E toolkit, Draft 14.01.04

M&E toolkit, Draft 14.01.04

Prevention of transmission by treating infectious cases

Prevention of TB among PLWHA

HIV seroprevalence among TB patients (PI 4)