600 East Superior Street, Suite 404 I Duluth, MN 55802 I Ph. 800.997.6685 or 218.727.9390 I www.ruralcenter.

org

Assessing Behavior Change Using the

Recommendation Adoption Process
Model

Kate Stenehjem
Nat’l Rural Health Resource Center
Information Specialist II
 RHPI Serves
Rural hospitals
<199 beds that are
located in Delta
Regional Authority
(DRA) counties /
parishes in 8 state
region
 Project Development Process
• Prioritize performance improvement (PI)
needs
• Define the PI activity
• Select a consultant
• Develop a scope of work (SOW)
RHPI Selection Considerations:
• No pending or anticipated issues that would
hinder the consultation process.
• Congruence between hospital needs and
available consulting services
 Project Development Process
Scope of Work (SOW) Includes:
• Purpose of consultation
• Objectives of the consultation
• Strategies to complete objectives
• Sustainability plan
• Work plan and timeline
– Define purpose, objectives, & anticipated outcomes in
general terms
– Define timeline and pre-schedule onsite consultation
dates to prepare staff for the consultant visit.
 RHPI does NOT accept ‘date TBA’!!
Example: Business Office Assessment SOW
Purpose
The objective is to conduct
rapid and focused analyses
of targeted areas within the
business office that result in
the identification of
concrete opportunities for
operational and financial
performance gains. We
facilitate the initital
implementation of
improvements to process
with the goal of increasing
gross and net
reimbursement while
mitigating compliance
Objectives Anticipated Outcomes
1. Identify areas of compliance risk 1. Drive higher gross
due to chargemaster or business revenue and net
office operational errors. reimbursement while
2. Identify areas of lost mitigating compliance
reimbursement due to issues with concerns.
business office policies, 2. Increase the
procedures and processes. opportunity
3. Identify areas of lower departmental
reimbursement due to issues with ownership and
contract management. accountability
4. Identify areas of compliance 3. Increase business
concern and risk within the office operational
business office. Present efficiency and
opportunities, training and steps effectiveness
to mitigate the identified risks.
 Project Completion Process
Consultant final reports should include:
1. Overview of services provided
2. Findings from assessments
3. Recommendations to improve performance
4. Action plan that outlines next steps to implement
recommendations
5. Specific anticipated outcomes
6. Measures / indicators to track progress
 Metrics should be standard industry accepted financial,
operational ratios / measures and/or quality indicator(s)
 600 East Superior Street, Suite 404 I Duluth, MN 55802 I Ph. 800.997.6685 or 218.727.9390 I www.ruralcenter.org

Post-Project Evaluation:
Recommendation Adoption Progress (RAP) Reporting
 What is RAP?
Recommendation Adoption Progress
(RAP) reporting is an evaluation tool that
captures the degree to which program
activities have been implemented and/or
services utilized by the program
participants.
 RAP Goal & Purpose
• The goal is to demonstrate the hospital’s
progress over time by indicating the extent
to which the facility has implemented the
consultant recommendation.
• The purpose is to summarize the overall
impact of activities and services.
– Hospitals are able to discuss accomplishments,
identify outcomes, and determine the impact
of activities and services.
Loosely Based on Appreciative Inquiry
RAP approach is loosely based on
Appreciative Inquiry (AI) and modified to
RHPI needs
• Seeks to discover the “best of what is”
• Integrates the collection of data
concerning expected measurable
outcomes and “how has this affected the
hospital bottom line?”
Loosely Based on Appreciative Inquiry
• RAP is a coaching tool
– Encourages the hospital to stay the course
by celebrating accomplishments and
recognizing the effort already expended
– increases awareness for the CEO to embed
the PI process into the hospital culture
• RAP is an integral part of RHPI process to
support sustainability
 Application of the RAP
• RAP process can easily be modified to
accommodate other purposes.
• AI concepts can be learned by those wishing
to utilize the RAP process.
• The process should be used when gathering
strong data and information is critical and
there is a need to “dig deeper.”
• Questions can then be developed to meet
the purpose of the interviewer.
 Application of the RAP
The Interviewer:
• Does not have to be a content expert but
should have some background knowledge
• Must have good rapport building skills
and be able to fluidly facilitate a
conversation
• Must be able to “follow threads” of
possible issues and not stay tied to a set
of questions
 The RAP Interview
RAP interview is scheduled 9 – 12
months post-project
• RAP evidence indicates hospitals
require at least 1.5 – 2 years to fully
implement the consultant
recommendations
• RHPI does not contact the hospital
at 3 or 6 months
 The RAP Interview
• Interview is scheduled with the CEO
• Interview package, which is forwarded
prior to the call, includes:
– The final report and action plan
– Scope of Work
– Pre-/ Post-Project outcomes work sheet
– RAP interview questionnaire
 The RAP Interview
Why forward all the documents again?
 Prepare the CEO for the interview
 Remind the CEO of project details
 Enhance probability of obtaining post-
project values
 Simplify process to augment the
interview experience and improve
outcomes capture
 The RAP Interview
RAP interview focuses on:
 Measurable results of anticipated
outcomes
 What is going well
 Next steps
 Project impact
 RAP Questionnaire
1. On a scale of 1 – 5, rate the extent to which the hospital
has implemented consultant best practice
recommendations.
1 = none or few recommendations implemented
5 = all implemented and the project is sustained with
outcomes
 Assesses the status of the implementation phase
 Provides a score to compare projects in the ROCI
analysis and selection process
 Determines if the hospital requires more time before
supporting another RHPI project
 Evaluates if a hospital is ready and deserving of another
project
 RAP Questionnaire
2. Please refer to the “Pre/Post Project Outcomes
Worksheet” included with this email. Please complete
the worksheet and return to rbarcus@ruralcenter.org
prior to the scheduled call.
 Encourages CEO to return worksheet prior to RAP.
 Considered in selection process

2. What is your hospitals current status with regard to

implementing the performance improvement
recommendations made by your RHPI sponsored
consultant(s)?
 Focuses on all the accomplishments.
 RAP Questionnaire
4. What are your hospitals expected next steps towards
adopting your consultants’ recommendations?
 Focuses on next steps to refocus on project
 May discuss any recommendations not implemented and why

5. Aside from the measurable outcomes, what are some of

the ways this project has impacted your hospital, its
culture, and the community?
 Looks at long term impact of this project and how to sustain
the project gains
 Summarizes the call and tends to energize the team
 RAP Score and Concept
• RAP Score ranges from 1 to 5
1 = none or few recommendations were adopted by
the hospital
5 = all recommendations were adopted by the
hospital

• The higher the score, the greater the number

of recommendations implemented by the
hospital.
– The hospital should see performance improvement
and the project should be sustained.
 RAP Scores of 4 - 5
• Hospitals are able to provide detailed
evidence of performance improvement gains
– Indicates very successful projects that are
sustained
• Reasons for score of 4.0 (<5.0)
– Some recommendations take longer to implement
– Financial limitations of implementation
– Target “low hanging fruit” first for quick fix
– Target “top 20” for the “biggest bang for the buck”
– Individualize to fit their culture
 RAP Score 3
• RAP score of 3 is considered by RHPI as
very positive and indicates that:
 the project is “in process”
 leadership is committed to continuing and
sustaining projects
 Correlated with initial outcomes associated
with cost savings and/or cost containment
 RAP Score 0 - 2

0 -2 Unsuccessful Project!
 Hospital needs more time to implement
the consultant recommendations
 Hospital must take action to implement
recommendations
 It’s the Hospital’s sole responsibility to implement
the recommendations, and NOT the consultant’s!
 RAP Score 0- 2
• Hospitals with scores of <2
– <5 hospitals reported score of 2
– Only 1 hospital reported a score of 1
– Only 2 hospital reported a score of 0

• Why?
– Majority - Change in leadership during the project!
– Many -Hospitals took on too many projects at
once
– Minority – Fewer than 3 hospitals decided that ALL
the recommendations were not feasible
 Scoring System
• RAP interviews are considered in the
selection process for future projects.
• Hospitals are scored based on completion of
RAP interview and its robustness.
• Hospitals are not weighted on their RAP
scores in an effort to encourage honest
interview responses and provide an
incentive to the implementation process.
 Variability in Outcomes
• Many variables may impact performance
– Clearly defining project objectives
– Indicators to quantify progress

• Hospitals should be well-positioned to

explain outcome variables that could
enhance/impede their adoption of
recommendations, and thus, performance
 Variability in Outcomes
• External variables may impact the
performance and may be beyond the
control of the hospital
– Triple Aim and Medicare reimbursement
 Example: BO Assessment RAP Findings
RAP Anticipated
Actual Reported Outcomes Administrative Feedback and Project Impact
Score Outcomes
4 1.Drive higher gross 1. Recovered 440k cash after We found that we didn’t have a coordinated revenue cycle
revenue and net addressing observation patient team to look at nuts and bolts, now have that, doing
reimbursement billing; Use same model in ER department by department charge master review,
while mitigating
and capture $30 more per observation patients –charge master capture. Able to rebill
compliance
concerns. patient at 10,000 visits per some of those patients and 400k cash that they would have
2.Increase the year=$300k lost. Using same model with ER and increase visit capture
opportunity 2. Yes, with revenue cycle by $30 and do about 10,000 visits year so about $300,000.
departmental meetings Department managers present to revenue cycle meeting
ownership and 3. Dropped AR days from under about how they are doing
accountability
50 to 36 Impact:
3.Increase business
office operational Better efficiency in business office
efficiency and Increased revenue and charge capture
effectiveness Increased ownership of department managers for
their revenue
Improved AR dates
Taken revenue reporting to “a different level”
Increase business savvy of management
600 East Superior Street, Suite 404 I Duluth, MN 55802 I Ph. 800.997.6685 or 218.727.9390 I www.ruralcenter.org

RHPI Outreach:
Dissemination of Lessons Learned
RAP Limitations & Program Improvements
FY of RAP Interview Year Response
Evaluated Rate
FY13 FY12 93%
FY12 FY11 88%
FY11 FY10 71%
FY10 FY09 71%
FY10 FY08 67%
 RHPI ROCI External Evaluation
• RHPI utilizes a Return on Community
Investment (ROCI) Analysis for the external
evaluation, which should be completed 3
consecutive years for good representation of
actual impact.
• The evidence gathered through RAP reports
demonstrate the individual hospital project
outcomes, but also illustrate the overall
impact of the program.
 ROCI – Key to RHPI Improvements
• RAP information used for ROCI
• In the first year of completion, ROCI identified
areas of RAP that needed strengthening,
which consequently, improved outcomes
capture
• Improved RAP process led to stronger ROCI
report in its second year, which is in turn
provided new options to improving project
planning
• In each case, RAP and ROCI has improved the
project planning and/or reporting processes
 RHPI RAP Evaluation Summary Approach
Correlation
Anticipated
Purpose Objective Method Limitations Between
Outcomes
Evaluations
The purpose of the To document Data is gathered by interviewing Data may be limited RAP results RAP is primary
RAP is to gather project CEOs approximately 6 – 12 due to: represent data for ROCI
information on the outcomes, months post-project. Completion of the individual analysis.
adoption of successes, and RAP interview, project
consultant best lessons Utilizes a scoring system to availability of the data outcomes and Also provides
practice learned from determine the overall adoption from the hospital, status of background
recommendations, RHPI rate. -RAP Score ranges from 1 and robustness of the progress. information and
and ultimately sponsored to 5. The concept is that the CEO response to the possibly
project outcomes. projects. higher the score, then the data request and Assessment supporting
greater number of interview findings and evidence to the
The goal of RAP is To recommendations were questionnaire. outcomes data financial and
to demonstrate a demonstrate implemented by the hospital to are used to quality
hospital’s progress how the sustain the project, which should Accuracy of the monitor and evaluations.
over time by project improve performance. reported financial and modify internal
showing the extent impacted the outcome data is processes to
to which a facility hospital and -Progress on anticipated dependent upon the improve services
has implemented how it’s being outcomes is recorded. CEO’s knowledge at to hospitals.
recommendations. sustained Impact on the hospital is also the time of reporting.
long-term. explored as it is an indicator of
sustainability.
National Rural Health Resource Center
600 East Superior St. Suite 404
Duluth, MN 55802
(218) 727-9390

Get to know us better: