School of Health Systems and Public Health

Monitoring & Evaluation of HIV and AIDS Programs

Data Quality
Wednesday March 2, 2011

Win Brown USAID/South Africa

Slide 1 of 18
 Objectives of the Session

• To Review and Discuss:

– A Data Quality approach to M&E

– Six important elements of data quality

– Practical applications

Slide 2 of 18
 Why Data Quality?

• Program is “evidence-based”

• Data quality  Data use

• Accountability

Slide 3 of 18
 Data Quality
Real World
In the real world, activities are
implemented in the field. These
activities are designed to produce
results that are quantifiable.
Data Management System
An information system represents
these activities by collecting the results
that were produced and mapping them
to a recording system.

Data Quality: How well the DMS represents the real world

?
Real Data
Management
World
System

Slide 4 of 18
 ? = Dimensions of Data Quality
Valid data are considered accurate: They measure
Validity what they are intended to measure.

The data are measured and collected consistently; definitions

Reliability and methodologies are the same over time.

Completely inclusive: the DMS represents the complete list of

Completeness eligible names and not a fraction of the list.

The data have sufficient detail; in this case the “accuracy” of

Precision the data refers to the fineness of measurement units.
Data are up-to-date (current), and information is available on
Timeliness time; the DMS produces reports under deadline.
The data are protected from deliberate bias or manipulation for
Integrity political or personal reasons.

Slide 5 of 18
Good Data are Valid and Reliable

≠ Valid ≠ Valid  Valid

≠ Reliable  Reliable  Reliable

X X X XXX
XX XXX
XXXX
X X
XXX
X XXXX
X XXX
X
Slide 6 of 18
What are:
– Valid data?

– Reliable data?

– Complete data?

– Precise data?

– Timely data?

– Data with integrity?

Slide 7 of 18
 Framework for Enhancing
Data Quality

Risk Verification
Data Management System Data Quality System Auditable
System
Data Management Data Quality
Processes / Processes / Document!
Procedures Procedures
Source
Collection
Collation Validity
Analysis Paper Trail
Reliability that allows
Reporting
Completeness verification of
the entire
Precision DMS and the
Timeliness data produced
Use within Slide
it 8 of 18
The South Africa Approach

• Data Quality Assessment

• Training

• Data Warehouse

• SASI Manual

• Standard M&E plan  DQ Plan

Slide 9 of 18
 Data Quality Assessment

• PMTCT Data; District focus

• Trace and Verify

• Routine Data Quality

Assessment Tool (RDQA)

Slide 10 of 18
 M&E Training?

M E
•Routine data collection •Internal validity
•Data quality •Operations research
•Results reporting •Instrument design
•Strategic planning •Survey sampling

•Data analysis for data use

•Local training partners
•Participant follow-up
•User’s groups/networks
Slide 11 of 18
 PEPFAR Reporting Issues

• Are PEPFAR’s results valid & reliable?

• How do you know?

• Are your patient numbers valid & reliable?

• How do you know?

Slide 12 of 18
 Data and Statistics are Empowering

100

# random
samples
drawn
50

0% 10% 20% 30% 40% 50%

Percent reporting: “I understand statistics.”

Slide 13 of 18
 Data Warehouse

• Online results reporting system

• Standardized data capture

• Control of data quality

• Customized reporting tool

• Online indicator guidance

Slide 14 of 18
South Africa Strategic Information
Manual (SASI Manual)

• Operational manual

• Standard definitions for

PARTNERS

• Addresses common data quality

problems

Slide 15 of 18
Try Making a Data Quality Plan

• Component of the M&E plan

• Strategically think about data

quality

Slide 16 of 18
 Measurement
With monitoring of progress in a clinic or in a
community, always try to hit the bull’s eye.

Paper Trail
Always document progress.

Data Use
Who is using the data?

Slide 17 of 18
Thank you

Slide 18 of 18