Using Data to Drive Health System

Performance

Commissioned from Ovations by the National

Primary and Care Trust Development Programme

1
 Health Care Evaluation &
Measurement for PCTs
Some Key Evaluation Terms
Acceptable Cost Quality
Accessible Data: primary and Quantity
Accomplishment secondary Recording
Accountable Goals Reliability
Accuracy Judgment Reporting
Analysis Metrics Standards
Appropriate Norms Synthesis
Available Objectives Time
Cohorts Outcomes Timelines
Comparison Outputs Validity
Content Precision Value
Context Process Weighting
Control Purpose Worth 2
 Health Care Evaluation &
Measurement for PCTs
Things to Consider
Comparing and contrasting various types of evaluation
Identification of problems that may hinder an effective
evaluation
Reasons to conduct an evaluation
The planning and conducting of an evaluation
Defining metrics and methods to measure and report
evaluation findings

3
 Health Care Evaluation
Key Ideas for PCTs
Evaluation is critical for all health care, and prevention/
health promotion initiatives.
Evaluation must be designed early in the process of health
care planning. Remember PRECEDE/PROCEED!
The process of designing an evaluation must be a
collaborative effort of all stakeholders.
Evaluation ultimately becomes “judgment.” Who has the
power to decide?
Evaluation does not need to be a formal academic study
Do you report successes, failures, strengths, weaknesses?
4
 Health Care Evaluation
Purposes for PCTs
1. To determine achievement of objectives related to
improved health status
2. To improve health program implementation
3. To provide accountability to funders
4. To increase community support
5. To contribute to the scientific base for community.
public health interventions
6. To impact policy decisions

5
 Health Care Evaluation
Definition
Evaluation:
Determining the value or worth of the health care
initiative against a standard of acceptability.
To examine or judge.

(The key is who establishes the standard and who judges!)

6
 Health Care Evaluation
Elements
Context: What, when , where, and who
Process: How care is organized and
delivered
Content: Program elements to be provided
and why - available birdseed
Output: How many times did the bird flap
its wings?
Outcome: Did the bird fly?
Impact: How high? How far? Where?

7
 Health Care Evaluation
Types

Process Evaluation: Examines the procedures

and tasks involved during the implementation of
a program.

8
 Health Care Evaluation
Types
Process Evaluation
When to use: As soon as the health initiative
begins
What it shows: How well a program is
working as it goes
Why is it useful: Identifies early problems

9
 Health Care Evaluation
Types
Outcome Evaluation: Used to obtain descriptive
data on a project and to document short-term
results. Focuses on an ultimate goal of a health
care program or treatment. Generally measured by
vital statistics in a population.

10
 Health Care Evaluation
Types
Outcome Evaluation
When to use: For ongoing programs at
appropriate intervals or for one time programs
when program is complete
What it shows: Has program reached its
ultimate goal.
Why is it useful: Learn from successes and
for future funding.
11
 Health Care Evaluation
Types

Impact Evaluation: Is the most comprehensive

type of evaluation because it focuses on the long-
range results and the resultant improvements in
health status.
Impact evaluation is the most costly.
Information obtained from an impact evaluation
can include changes in e.g.,morbidity and
mortality.
12
 Health Care Evaluation
Types
Impact Evaluation
When to use: After the health program has
made “contact” with at least one person or a
population
What it shows: Changes in knowledge,
attitudes, and beliefs
Why is it useful: Allows management to
modify resources effectively

13
 Health Care Evaluation

Lets pick a population health programme idea

for your specific PCT and consider aspects of
process, outcome and impact evaluation.

14
 Metrics and Methods
to Measure and Report
Indicators of Hospital Utilisation
Assumption:
> Hospital admission is an adverse event
that should be avoided whenever possible
> Not all admissions are preventable
Two major indicators
> Admissions per 1000
> Days per 1000
15
 Metrics and Methods
to Measure and Report
Indicators of Hospital Utilisation
Admissions per 1000
> In part measures a systems success at
preventing acute disease episodes or
caring for them in alternative systems
> Considered a prevention indicator
Days per 1000
> Reflects both frequency of admission and
length of stay
> Measure episode characteristics
16
 Metrics and Methods
to Measure and Report
Indicators of Hospital Utilisation
Both measures are typically collected separately
for:
> Adult and pediatric medical
> Surgical
> ICU
> Obstetrics
> Boarder babies (newborn days after mother’s
discharge)
> Mental health
> Substance abuse
17
 Metrics and Methods
to Measure and Report
Risk Adjustment

Prospective vs. retrospective

In general populations the ‘theoretical ceiling
to predication is between 18-22%
>This is higher in sicker, chronic populations
Demographic models
>Typically explain 2-5% of variance in
utilization

18
 Metrics and Methods
to Measure and Report
Risk Adjustment
Adding prior use increase the predicative
ability of models to between 8 – 14%
> Some antidotial evidence that adding Rx
and Lab can increase predicative ability to
the 18-22% range
Adding health status changes the weights but
does not increase the predicative ability of
models

19
 Metrics and Methods
to Measure and Report
Traditional Health Care Performance
Indicators
Finances
> Revenue and costs

Clinical Activities

Quality of Care
> Includes access
20
 Metrics and Methods
to Measure and Report
Guidelines for Selecting Indicators

The ability of systems to produce the data required to

generate the indicator
The perceived importance of the indicator to the
development and assessment of the system
The current utility of the indicator in organizations
within the system
The comparability of the indicator across systems
The uniqueness of the applicability of the indicator

21
 Metrics and Methods
to Measure and Report
Indicators of System Performance

Population Health
> SF-30, SF-12, etc
> Mental Health Index (such as the DIS)
> Disease specific incidence and prevalence
rates
> Disease specific outcomes of care (SF-36,
ADLs, IADLs)

22
 Metrics and Methods
to Measure and Report
Indicators of System Performance

Quality of Care
> Compliance with guidelines by type of
provider
> Incidence of Sentinel Events:
- Unplanned re-visits
- Avoidable hospitalisations
- Hospital re-admission
- Mortality
23
 Metrics and Methods
to Measure and Report
Indicators of System Performance

Quality of Care (cont.):

> Peer review
> Disease specific mortality
- Surgical
- Infant mortality
- Deaths due to cervical cancer
> Frequency of preventable acute episodes
for chronic conditions
24
 Metrics and Methods
to Measure and Report
Indicators of System Performance

Quality of Care (cont.):

> RX
- Inappropriate poly pharmacy
- ADRs
Episode Prevention

25
26
27
28
 Calculating Bed Days
Works for any given time period (e.g.
day, month to date, year to date)
[A/ (B/365)] /(C/1000)
> A = Gross bed days per unit of time
> B = Days per unit of time
> C = population size

29
 Knowledge Management
Planning Cycle for Driving PCT Health Care
System Performance

Social/Health
Indicators
Evaluation Data & Analysis

Implementation Useful Informatio

Operations Planning Knowledge Managem

Strategy

30