Critical Analysis of the Four Habits Coding Scheme (4HCS)

We recommend first reading the Critical Analysis of the Four Habits Coding Scheme, which
covers basic descriptive and psychometric information, including the number of items,
instrument purpose, and psychometric (i.e. validity) data and can help determine if the Four
Habits Coding Scheme meets your needs. If you are interested in getting more detailed
information about the items in the instrument or have decided to consider using the Four
Habits Coding Scheme, we recommend reviewing the Four Habits Coding Scheme instrument
file.

Educational objectives:
1. To describe the purpose and basic properties of the Four Habits Coding Scheme,
including number of items, subscales, and psychometric properties;
2. To describe the application of the Four Habits Coding Scheme to the field of health
sciences education;
3. To evaluate the relative strengths and weaknesses of the Four Habits Coding Scheme;
and
4. To provide the Four Habits Coding Scheme to aid in its administration.

Resource files:
1. Critical Analysis of the 4HCS.pdf
2. 4HCS Instrument.doc

A. Original citation: Krupat E, Frankel R, Stein T, Irish J. The Four Habits Coding Scheme:
Validation of an Instrument to Assess Clinicians’ Communication Behavior. Patient Education
and Counseling 2006; 62:38-45.

B. Brief Description/Purpose: The 4HCS measures clinician behavior based on the Four Habits
Model, a framework used for teaching communication skills that describe basic tasks and
functions of patient-centered medical interview and elements related to the interview, within
and between medical visits.1 The Four Habits Model consists of the following four tasks: Habit 1
(Invest in the Beginning), Habit 2 (Elicit the Patient’s Perspective), Habit 3 (Demonstrate
Empathy), and Habit 4 (Invest in the End). The 4HCS was developed such that its four subscales
correspond to core skills of the Four Habits Model: Habit 1 (6 items), Habit 2 (3 items), Habit 3
(4 items), and Habit 4 (10 items). Trained coders observe and rate the 4HCS. Each item is scored
on a 5-point behaviorally anchored rating scale. The instrument provides definitions for the
middle and end categories; the “2” and “4” categories are reserved only when the particular
behavior lies directly between the middle and end categories. The predominant method of
scoring each Habit is calculated by taking the mean across items.

C. Development and psychometrics: Behaviors observed in the 4HCS are broadly defined,
where the coder is required to consider and combine several features based on verbal and
nonverbal signals. Moreover, the coding processes based on descriptions of clinician behavior
have an underlying conceptual model that provides evaluative distinctions of behaviors.

1
Initial validity evidence related to scores from this instrument was based on 100 videos of
physician-patient visits from the Ambulatory Care Center of the Massachusetts General
Hospital. Videos were based on 25 male and 25 female physicians who each saw one male and
one female patient. Patients were primarily white and had an average age of 62.1

This critical analysis on the validity evidence for scores relating to the 4HCS is evaluated based
on Messick’s unified validity framework:

• Content: The conceptual framework of the 4HCS maps directly to behaviors in the Four
Habits Model.1 Items of the 4HCS correspond to each subscale that forms the Habits.

• Response process: Two health professions students with exposure to clinical practice
were trained (8 to 10 hours) to reach acceptable agreement. Scoring time ranged
between 2 and 5 minutes, in addition to the encounter. Among 100 videos, 13 videos
were used to calculate inter-rater agreement. Correlations of scores provided by the
two raters were 0.70, 0.80, 0.71, and 0.69 for each Habit, respectively; overall average
was 0.72. Descriptive statistics were presented using the full set of 100 videos. Four
items (1 item from Habit 1 and 3 items from Habit 4) had high mean scores (mean rating
about 4 or above), and 2 items (1 item from Habit 1 and 1 item from Habit 4) had
particularly low mean scores (mean rating below 2). Out of 23 items, 7 items were left
skewed (mode=5), while 10 items were right skewed (mode=1).1

• Internal structure: Cronbach’s alpha for each Habit was 0.71, 0.51, 0.81, and 0.61,
respectively.1

• Relationship to other variables: All four Habits were significantly correlated with one or
more of the Roter Interaction Analysis Scheme (RIAS) categories: Personal Talk with
Habit 1 (r=0.28); Positive/Partnership Talk with Habit 2 (r=0.29) and Habit 3(r=0.23);
Emotional Talk with Habit 2 (r =0.30) and Habit 3 (r =0.37); Psycho-social Questions
Asked with Habit 2 (r =0.37), Habit 3(r =0.27), and Habit 4 (r =0.20); Psycho-Social
Information Given with Habit 2 (r =0.20) and Habit 3 (r=0.25). Habits 2, 3, and 4 were
associated with back channel responses (brief verbal expression of sustained interest),
with correlations ranging between 0.26 and 0.41. Nonverbal behaviors (smiles, nods,
and eye-contact) were associated with the 4HCS, with correlations ranging between
0.26 and 0.42. Duration of visit was correlated with Habit 1 (r=0.25), Habit 3 (r=0.26),
and Habit 4 (r=0.28). There were no significant correlations with post-visit evaluations, a
15-item scale (scored on a 6-point Likert scale) completed by patients measuring
informativeness, respect/consideration, and competence of the physician.1

• Consequences: The 4HCS is a formative tool based on the Four Habits Model. Programs
using this model have reported significant improvement in patient satisfaction,
sustained over time. Uses of the Four Habits Model have extended to diverse areas,
such as geriatric care, training for cultural competence, and cost conversations with
patients.1
2
D. Additional Studies Reporting Validity Evidence: Six peer-reviewed papers were found
reporting additional validity evidence relating to scores on the 4HCS (see Table 1).

E. Application to Health Sciences Education and Health Sciences Education Research: Although
there was low evidence of 4HCS association with clinical skills assessment scores measured with
medical students,2 lower scores on 4HCS was linked with longer duration3 of back pain.
Communication training (20 hours) based on the Four Habits Model showed significant
improvement6 in communication skills, as measured using the 4HCS. Moreover, sufficient
training (20 hours) based on 4HCS led to improved self-efficacy7 of participating doctors.

Studies using the 4HCS have indicated validity evidence in the use of its scores. Moreover, the
4HCS has been examined in various specialties.4 Given reasonable association with scores in
other communication instruments, such as the RIAS, and its applicability in various settings, the
4HCS may be an appropriate tool to measure communication skills in various health sciences
education fields. These results indicate that additional uses of the 4HCS in various settings are
supported, as they may contribute to findings that can inform field-specific areas of health
sciences education when using this instrument.

F. Commentary: The 4HCS is a formative tool used to measure patient-centered

communication skills. Results indicate validity evidence supporting the use of scores on the
4HCS. The instrument requires shorter coding time than other instruments, such as the RIAS,
which may take from two8 to five times9 the duration of the encounter. As such, there are
benefits in the feasibility of the 4HCS, in addition to evidence supporting the validity of its
scores.

The 4HCS requires the use of coders who have undergone sufficient training and recalibration.
Users of this instrument should note that the high levels of inter-rater reliability (above 0.70)
were obtained through rigorous rater training (18 hours).4 In particular, items with high and low
values may benefit from further analysis on whether they are consistent with other findings on
physicians’ communication strengths and weaknesses. Although inter-rater agreement was
satisfactory at the instrument level, subscale agreement between raters at the Habit level or at
the item level was unsatisfactory. Internal-consistency reliability was also low for some
subscales. These results indicate that the use of mean scores across all items of the 4HCS may
be more appropriate than using subscale level scores. In addition, expanding raters to include
experienced faculty observers may offer a broader assessment of behaviors in the 4HCS;
correlation between faculty and student observers would be informative to evaluate
differences in the ability between groups of raters. Validity evidence may be needed for 4HCS
scores based on trainees at different stages of medical training.

3
Table 1. 4HCS application studies

Validity
Study Population tested n Results Comments
evidence
Scores from the 4HCS were not The local institution plans
Relationship to 3rd
year medical 30 SP correlated with clinical skills assessment to use a more
Rouf2
other variables students encounters or interpersonal and communication comprehensive and valid
skills. checklist.
Poor communication
79 clinic
Patients with low Negative correlation between 4HCS skills measured using
Relationship to encounters
Gulbrandsen3 back pain (Mean scores and duration of back pain 4HCS were associated
other variables (21
age=45) (r=–0.32) and patient age (r=–0.35). with back pain and older
clinicians)
age.
Hospital patients Inter-rater reliability following training Scoring took 1.5 times
497
in various settings, (18 hours) for 4HCS total score was the duration of the
hospital
Response involving several above 0.70 (Pearson’s correlation and encounter. This is still
Jensen4 encounters
process specialties (Mean intraclass correlation) for four raters. shorter than the RIAS,
(71
age=46); four Inter-rater reliability at the item or which can take 2 to 5
doctors)
trained raters Habit level was unsatisfactory. times the encounter.8,9
Correlations between 4HCS and Correlations between
174 family Measure of Patient-Centered two patient-centered
Relationship to Family practice
Clayton5 practice Communication (MPCC) was low coding schemes were
other variables visits
office visits (r=0.16). Subscale correlations were not examined and found to
significant. have low association.
Response Scores on the 4HCS were
Hospital doctors in Following training on the Four Habits
process & able to detect training
Jensen6 various settings 51 doctors Model (20 hours), 4HCS scores
Relationship to effects from the Four
(Mean Age=60) improved significantly (7.5 points).
other variables Habits Model.
Response Following training (20 hours), Scores on the 4HCS were
Hospital doctors in
process & correlation between self-efficacy and able to detect post-
Gulbrandsen7 various settings 62 doctors
Relationship to communication was significant (r=0.34). training effects and
(Mean Age=62)
other variables Training improved self-efficacy (d=0.27). improve self-efficacy.

4
G. Citations:

1. Krupat E, Frankel R, Stein T, Irish J. The Four Habits Coding Scheme: Validation of an
Instrument to Assess Clinicians’ Communication Behavior. Patient Education and
Counseling 2006;62:38-45.
2. Rouf E, Chumley H, Dobbie Al. Patient-centered interviewing and student performance
in a comprehensive clinical skills examination: Is there an association? Patient Education
and Counseling 2009;75:11-15.
3. Gulbrandsen P, Madsen HB, Benth JS, Lærum E. Health care providers communicate less
well with patients with chronic low back pain – A study of encounters at a back pain
clinic in Denmark. Pain 2010; 150:458-461.
4. Jensen BF, Gulbrandsen P, Benth JS, Dahl FA, Krupat E, Finset A. Interrater reliability for
the Four Habits Coding Scheme as part of a randomized controlled trial. Patient
Education and Counseling 2010;80:405-409.
5. Clayton MF, Latimer S, Dunn TW, Haas L. Assessing patient-centered communication in a
family practice setting: How do we measure it, and whose opinion matters? Patient
Education and Counseling 2011;84:294-302.
6. Jensen BF, Gulbrandsen P, Dahl FA, Krupat E, Frankel RM, Finset A. Effectiveness of a
short course in clinical communication skills for hospital doctors: Results of a crossover
randomized controlled trial. Patient Education and Counseling 2011;84:163-169.
7. Gulbrandsen P, Jensen BF, Finset A, Blanch-Hartigan D. Long-term effect of
communication training on the relationship between physicians’ self-efficacy and
performance. Patient Education and Counseling 2013;91:180-185.
8. Roter DL. The Roter Method of Interaction Process Analysis (RIAS Manual).
9. Ong LML, Visser MRM, Kruyver IPM, Bensing JM, van der Brink-Muinen A, Stouthard
JML, Lammes FB, de Haes JCJM. The Roter Interaction Analysis System (RIAS) in
oncological consultations: Psychometric properties. Psycho-oncology 1998;7:387-401.

H. Author Information:
Yoon Soo Park, PhD Ara Tekian, PhD, MHPE
Assistant Professor, Dept. of Medical Education Associate Professor, Dept. of Medical Education
University of Illinois at Chicago University of Illinois at Chicago
808 South Wood Street, 986 CMET (MC 591) 808 South Wood Street, 986 CMET (MC 591)
Chicago, IL 60612-7309 Chicago, IL 60612-7309
Office phone: 312-355-5406 Office phone: 312-996-8438
yspark2@uic.edu tekian@uic.edu