This article was downloaded by: [University Of Maryland]

On: 15 October 2014, At: 03:39

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

Psychotherapy Research
Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/tpsr20

Therapist responsiveness and patient engagement in

therapy
a b a c d
Irene Elkin , Lydia Falconnier , Yvonne Smith , Kelli E. Canada , Edward Henderson , Eric R.
e a
Brown & Benjamin M. McKay
a
School of Social Service Administration, University of Chicago, Chicago, IL, USA
b
Governors State University, University Park, PA, USA
c
School of Social Work, University of Missouri, Columbia, MO, USA
d
Corpus Christi Outpatient Clinic, Veterans Administration, Corpus Christi, TX, USA
e
Urban Education Institute, University of Chicago, Chicago, IL, USA
Published online: 01 Aug 2013.

To cite this article: Irene Elkin, Lydia Falconnier, Yvonne Smith, Kelli E. Canada, Edward Henderson, Eric R. Brown &
Benjamin M. McKay (2014) Therapist responsiveness and patient engagement in therapy, Psychotherapy Research, 24:1,
52-66, DOI: 10.1080/10503307.2013.820855
To link to this article: http://dx.doi.org/10.1080/10503307.2013.820855

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and
should be independently verified with primary sources of information. Taylor and Francis shall not be liable for
any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of
the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
 Psychotherapy Research, 2014
Vol. 24, No. 1, 5266, http://dx.doi.org/10.1080/10503307.2013.820855

Therapist responsiveness and patient engagement in therapy

IRENE ELKIN1*, LYDIA FALCONNIER2, YVONNE SMITH1, KELLI E. CANADA3,

EDWARD HENDERSON4, ERIC R. BROWN5, & BENJAMIN M. MCKAY1
1
School of Social Service Administration, University of Chicago, Chicago, IL, USA; 2Governors State University, University
Park, PA, USA; 3School of Social Work, University of Missouri, Columbia, MO, USA; 4Corpus Christi Outpatient Clinic,
Veterans Administration, Corpus Christi, TX, USA & 5Urban Education Institute, University of Chicago, Chicago, IL, USA
(Received 15 May 2012; revised 15 June 2013; accepted 26 June 2013)

Abstract
This study tests the hypothesis that therapist responsiveness in the first two sessions of therapy relates to three measures of
Downloaded by [University Of Maryland] at 03:39 15 October 2014

early patient engagement in treatment. Using videotapes and data from the NIMH Treatment of Depression Collaborative
Research Program (TDCRP), an instrument was developed to measure therapist responsiveness in the first two sessions of
Cognitive Behavior Therapy and Interpersonal Psychotherapy. A factor measuring positive therapeutic atmosphere, as well
as a global item of therapist responsiveness, predicted both the patient’s positive perception of the therapeutic relationship
after the second session and the patient’s remaining in therapy for more than four sessions. A negative therapist behavior
factor also predicted early termination. Factors measuring therapist attentiveness and early empathic responding did not
predict the engagement variables.

Keywords: therapist responsiveness; patient-therapist relationship; therapeutic alliance; early termination

There is a considerable body of literature on the
relationship of patient early engagement in treatment
to the outcome of psychotherapy (Beutler et al.,
2004; Horvath, Del Re, Fluckinger, & Symonds,
2011; Horvath & Greenberg, 1994; Suh, Strupp, &
O’Malley, 1986; Tryon & Kane, 1993). The focus of
this study is on the therapist’s behavior in the first
two sessions of treatment that may facilitate such
engagement. The study utilizes videotapes and data
from the NIMH Treatment of Depression Colla-
borative Research Program (TDCRP; Elkin, 1994).
The TDCRP was a multi-site collaborative study of
two psychotherapies, Cognitive Behavior Therapy
(CBT) and Interpersonal Psychotherapy (IPT), as
well as pharmacotherapy reference and control
conditions, in the treatment of patients with Major
Depressive Disorder.
‘‘Early engagement in treatment’’ is defined, in
this study, by the patient’s positive perception of the
relationship after the second session, the patient’s
contribution to the therapeutic alliance during the
third session, and the patient’s remaining in treat-
ment for more than four sessions. This definition of
engagement has been successfully used in a previous
study based on TDCRP data (Elkin et al., 1999). In
that study, ‘‘patient-treatment fit’’ (i.e., the congru-
ence of a patient’s treatment assignment with his or
her predilection for the treatment) was significantly
related to each of these three components of early
engagement.
Many studies support the relationship to outcome
of both the patient’s perception of the relationship
(Barrett-Lennard, 1986; Elliott, Bohart, Watson, &
Greenberg, 2011; Farber & Doolin, 2011; Gurman,
1977), generally measured by the patient’s ratings of
the Rogerian facilitative conditions (Rogers, 1957),
and various measures of the therapeutic/helping/
working alliance (Horvath et al., 2011; Lambert &
Ogles, 2004). Orlinsky and colleagues, summing up
a wide body of literature on the relationship of
process variables to outcome, write that ‘‘The
strongest evidence linking process to outcome con-
cerns the therapeutic bond or alliance . . .’’ (Orlinsky,
Ronnestad, & Willutzki, 2004, p. 323). In addition
to the literature on the relationship to outcome of the
therapeutic relationship and the alliance, we also
know that a sizable number of patients terminate
treatment before they have been able to derive much
benefit from it (Garfield, 1994; Reis & Brown, 1999;
Ogrodniczuk, Joyce, & Piper, 2005).

Correspondence concerning this article should be addressed to Irene Elkin, University of Chicago, School of Social Service Administration,
969 E. 60th Street, Chicago, IL 60637, USA. Email: ielkin@uchicago.edu

# 2013 Society for Psychotherapy Research


Most important for the current study, findings
have been reported, based on TDCRP data, for the
relationship to outcome of each of the engagement
variables in this study. Krupnick and colleagues
(Krupnick et al., 1996) found a positive relationship
with outcome, across treatments, for ratings of the
patient’s contribution to the alliance, using a mod-
ified version of the Vanderbilt Therapeutic Alliance
Scale (VTAS; Suh et al., 1986); Blatt and colleagues
(Blatt, Zuroff, Quinlan, & Pilkonis, 1996) reported a
positive relationship to outcome, across treatments,
for patient ratings on the Barrett-Lennard Relation-
ship Inventory (BLRI; Barrett-Lennard, 1986), and
Elkin and colleagues (Elkin et al., 1999) reported
much poorer outcomes for patients terminating by
the fourth week, as compared to those with more
than 4 weeks of treatment.
Although there is considerable evidence of the
Downloaded by [University Of Maryland] at 03:39 15 October 2014
relationship to outcome of measures of the thera-
peutic relationship and of the alliance, relatively little
is known about the therapist interventions during the
first two sessions of treatment that may facilitate the
early development of the relationship or alliance.
Although the terms relationship and alliance are
both used here, it should be pointed out that they are
not equivalent, especially as operationalized in this
study. The ‘‘relationship’’ is measured by the pa-
tient’s perception of the facilitative conditions pro-
vided by the therapist, using the Barrett-Lennard
Relationship Inventory (Barrett-Lennard, 1962,
1969), while the ‘‘alliance’’ is defined by an obser-
ver’s rating of the patient’s contribution to the
alliance between patient and therapist, using a
modified version of the Vanderbilt Therapeutic
Alliance Scale (Hartley & Strupp, 1983; Krupnick
et al., 1994, 1996). The two instruments differ in
both perspective and focus.
Horvath and Greenberg (1994) wrote: ‘‘It
seems . . . that all the research on the impact of
therapist activities on the alliance have investigated
the status of the alliance once it has been devel-
oped . . . less is known about the transactions that
enhance the initial buildup of the alliance (sessions 1
through 3)’’ (p. 7). Although there have been some
relevant studies since the Horvath and Greenberg
review (e.g., Sexton, Hembre, & Kvarme, 1996;
Sexton, Littauer, Sexton, & Tommeras, 2005) they
are still relatively rare. This is surprising, given the
fact that the first few sessions (or even the first few
moments) of psychotherapy are often seen as critical,
by both clinicians and researchers, for the engage-
ment of the patient in the treatment process (e.g.,
Castonguay, Constantino, & Holtforth, 2006;
Freud, 1913; Garfield, 1980; Henry & Strupp,
1994; Hubble, Duncan, Miller & Wampold, 2010;
Therapist responsiveness 53
Kanfer & Schefft, 1988; Reis & Brown, 1999;
Sullivan, 1954; Wills, 1982).
The first few sessions are the time when patients
make their initial decisions about whether to commit
to treatment and begin to develop (or not develop) a
relationship with the therapist that will allow them to
benefit from treatment. Patients may, of course,
enter treatment with a predisposition toward a
particular form of treatment (Arnkoff, Glass, Shea,
McKain, & Sydnor-Greenberg, 1987; Elkin et al.,
1999) or with other characteristics that may affect
their ability or willingness to remain in treatment and
to enter a therapeutic relationship (Clarkin & Levy,
2004; Elliot et al., 2011; Farber & Doolin, 2011;
Garfield, 1994; Horvath, 1994; Reis & Brown, 1999;
Tryon, 2002). The behavior of the therapist, in his or
her initial encounters with the patient, is also very
likely to have an effect on whether the patient begins
to feel positive about and to become involved in the
relationship with the therapist and decides whether
to remain in therapy. In the current study, it is
hypothesized that one aspect of the therapist’s
behavior that may be especially important early in
therapy, specifically in the first two sessions, is his or
her responsiveness to the patient.
‘‘Responsiveness,’’ in the recent psychotherapy
research literature, is often identified with the writ-
ings of Stiles and his colleagues (Stiles, 1988; Stiles,
Honos-Webb & Surko, 1998; Stiles & Shapiro
1994). They introduced their conceptualization in
an effort to explain the lack of consistent relation-
ships between process variables and outcome, rather
than for the purpose of measuring any specific aspect
of responsiveness. While their conceptualization
includes the therapist’s responding with specific
interventions tailored to the patient’s characteristics
and behavior in the unfolding therapeutic process,
the construct in the current study is much narrower
in scope. It refers to a general responsiveness to the
person with whom the therapist is interacting, and
the scale developed to measure it is being used to test
a very specific hypothesis about the relationship of
therapist behavior very early in therapy and the
patient’s engagement in treatment.
Responsiveness in this study is defined as ‘‘the
degree to which the therapist is attentive to the
patient; is acknowledging and attempting to under-
stand the patient’s current concerns; is clearly
interested in and responding to the patient’s com-
munication, both in terms of content and feelings;
and is caring, affirming, and respectful towards the
patient’’ (Elkin & Smith, 2007). This construct and
its accompanying scale were developed in an iterative
process over a number of years. They were devel-
oped in the context of doctoral seminars and
research groups in which psychotherapy researchers
 54 I. Elkin et al.
and doctoral students (some with considerable
clinical experience) viewed many videotapes of
treatment sessions, primarily first and second ses-
sions in the pilot phase of the TDCRP. The focus
was on identifying therapist behaviors that might
facilitate the patient’s engagement in therapy and
developing items to measure those behaviors. The
constellation of items that emerged seemed to be
best encompassed by the construct of therapist
responsiveness to the patient. (More information
on the development and final version of the scale and
its accompanying manual will be provided in the
Method section.)
No specific theory of psychotherapy guided the
development of the scale, since the responsiveness
construct is considered to be pan-theoretical. It is
probably most influenced, however, by a general
client-centered philosophy, especially in its emphasis
Downloaded by [University Of Maryland] at 03:39 15 October 2014
on the importance of empathy and positive regard
(Rogers, 1957). The aspect of empathy most rele-
vant here is the sub-type described by Bohart and
Greenberg (1997) as ‘‘empathic rapport,’’ a global
form of empathy, which they consider important for
relationship-building. The construct of responsive-
ness, as operationalized by the current scale, is also
compatible with the concept of ‘‘rapport,’’ as de-
scribed by Beck and colleagues (Beck, Rush, Shaw,
& Emery, 1979) in their discussion of the therapeutic
relationship in Cognitive Behavior Therapy, one of
the treatments in the TDCRP. Beck and colleagues’
suggestions of therapist behaviors to facilitate rap-
port, e.g., maintaining eye contact, following the
content of the patient’s talk, trying to infer and
reflect the patient’s feelings, and carefully timing
when to talk and when to listen, are also applicable
to other therapeutic approaches.
Although most studies of therapist in-session
behaviors have focused on their relationship to
outcome of treatment (see, e.g., reviews by Beutler
et al., 2004, and Orlinsky et al., 2004), a smaller
number of researchers have studied the relationship
of various therapist characteristics and techniques to
the therapeutic alliance or to patient-perceived em-
pathy, a component of the therapeutic relationship.
Ackerman and Hilsenroth (2001, 2003), in their
reviews of studies that have looked at the role of
therapist in-session variables related to the alliance,
list some that are relevant to the construct of
responsiveness that was developed in this study.
They include the therapist’s being respectful and
warm towards the patient, using techniques such as
reflection and attending to the patient’s experience
(Ackerman & Hilsenroth, 2003), and not being
critical (Ackerman & Hilsenroth, 2001). Watson
(2002) reviewed studies correlating measures of
therapist behavior to patient-perceived empathy.
Therapist behaviors relevant to the construct of
responsiveness in this study include several non-
verbal behaviors, such as eye contact, a concerned
expression, head nods, and interruptions (in a
negative direction).
In the majority of the studies investigating the
relationship of therapist interventions to the alliance,
however, these behaviors were measured in the same
sessions in which the alliance was assessed. Thus,
they may be seen as correlates rather than possible
therapist contributors to the building of the alliance.
Also, very few studies of the alliance examine
therapist behavior in the first and second sessions
of treatment.
Sexton and colleagues are among the few research-
ers who have focused on therapist behaviors in the
first session that may facilitate the development of
the alliance. They found (Sexton et al., 1996) that
patients’ alliance ratings at the end of the first session
were higher for therapists who listened more and did
not change the topic as frequently as those in low-
alliance sessions. In a more recent study, Sexton and
colleagues (Sexton et al., 2005) reported a positive
relationship between a single-item rating of ‘‘con-
nection,’’ established early in the first session, to
alliance ratings in the second session. Most relevant
for the current study are the therapist behaviors
related to higher connection scores, including thera-
pist warmth, engaged listening, and a mixture of
emotional and cognitive speech. Some of the beha-
viors identified in the Ackerman and Hilsenroth
reviews, in the Watson review, and in the Sexton
group’s studies are similar to the construct of
responsiveness and related scale items in the current
study.
There is some overlap of the scale developed in
this study to measure responsiveness and other
measures of therapist behaviors, including the Van-
derbilt Psychotherapy Process Scale (VPPS) and the
Vanderbilt Negative Indicators Scale (VNIS) (Suh et
al., 1986), the Penn Therapist Facilitative Behaviors
Scale (TFB; Alexander & Luborsky, 1986; Lubors-
ky, Crits-Christoph, Mintz, & Auerbach, 1988), the
Psychotherapy Process Q Set (Jones, 2000), and the
Measure of Expressed Empathy (Watson & Prosser,
2002). These measures are generally broader in
scope, however, than the current scale. They include
items targeted to measure therapist behaviors across
the entire course of treatment; they are rarely applied
to the first two sessions, and some of their items
would be relevant only in later stages of therapy.
Some also include a few items specific to particular
theoretical approaches and therapist techniques or
items that are more evaluative of the therapist’s
behavior. To our knowledge, no previous measure
has been developed that focuses specifically on

therapist responsiveness as it may be expressed in the
first two sessions.
The major aim of this study is to test the
hypothesis that therapist responsiveness in the first
two sessions is related to three components of the
patient’s early engagement in therapy. Since this is
the first study reported using the current measure of
responsiveness, information is also provided about
its development and psychometric characteristics.
Although therapist responsiveness is the primary
focus of the study, we must recognize that, since the
therapeutic process is an interactive one, certain
patient behaviors may influence and/or interact with
the conditions provided by the therapist (Clarkin &
Levy, 2004; Elliott et al., 2011; Farber & Doolin,
2011; Horvath & Bedi, 2002; Tryon, 2002). One
aspect of patient behavior that repeatedly appears in
both the theoretical and empirical literature is that of
Downloaded by [University Of Maryland] at 03:39 15 October 2014
the patient’s resistance (or reactance) in therapy.
Resistance and the therapist’s response to it are
sometimes seen as critical variables in both the
process and outcome of psychotherapy (see, e.g.,
Arnow et al., 2003; Beutler, Harwood, Michelson,
Song & Holman, 2011; Wachtel, 1999). For this
reason, we also present an exploratory analysis which
includes an item measuring patient resistance.
Method
Design
This study is based on data and videotapes from the
NIMH Treatment of Depression Collaborative Re-
search Program (TDCRP). The design and proce-
dures of the TDCRP have already been described in
some detail (Elkin, 1994; Elkin, Parloff, Hadley, &
Autry 1985; Elkin et al., 1989). We, therefore,
present only a brief overview here. The TDCRP
was a collaborative study in which an identical
research protocol was conducted at three research
sites (George Washington University, University of
Pittsburgh, and University of Oklahoma). The de-
sign of the TDCRP was a 3 (research sites) by 4
(treatment conditions) factorial design. The 250
patients who passed screening and rescreening and
gave informed consent (including the audio- and
video-taping of treatment sessions for research
purposes) were randomly assigned to four treatment
conditions: Cognitive Behavior Therapy (CBT),
Interpersonal Psychotherapy (IPT), Imipramine
plus Clinical Management (IMI-CM) and Placebo
plus Clinical Management (PLA-CM). Only the two
psychotherapies are included in the current study.
(Institutional Review Board [IRB] approval for the
TDCRP was obtained at each of the research sites.
Approval of the current research was obtained from
Therapist responsiveness 55
the IRB at the School of Social Service Administra-
tion, University of Chicago.)
Patients
To be included in the study, patients had to meet
Research Diagnostic Criteria (Spitzer, Endicott &
Robins, 1978) for a current episode of Major
Depressive Disorder and have a score of 14 or
greater on an amended version of the 17-item
Hamilton Rating Scale for Depression (HRSD;
Hamilton, 1967). These criteria had to be met
both at screening and at rescreening 1 to 2 weeks
later. Exclusion criteria are detailed in the previous
publications. Of the 250 patients randomly assigned,
239 actually began treatment, including 120 in the
CBT and IPT conditions. Seventy-two patients (36
each in CBT and IPT) are included in the current
study. Fourteen of the 120 patients were excluded
because tapes were missing or had inadequate audio
quality, and 18 were excluded because they had been
seen during training or pre-study reliability checks.
Of the remaining 88 patients, 72 were selected to
approximate the original sample of 120 patients on
five variables: Gender, marital status, completer
status, and pre-treatment scores on the HRSD and
the Global Assessment Scale (GAS; Endicott, Spit-
zer, Fleiss, & Cohen, 1976). Seventy-four percent of
the patients in the final sample are female, 83% are
white, 69% are treatment completers, 46% were
married or cohabiting, 25% single, and 29% sepa-
rated, divorced or widowed. Their average pre-
treatment scores were 19.61 on the HRSD and
52.15 on the GAS.
Treatments and Therapists
Treatments were conducted in accord with manuals
prepared by the developers of each of the psy-
chotherapies (Beck et al., 1979, for CBT; Klerman,
Weissman, Rounsaville, & Chevron, 1984, for IPT).
Both CBT and IPT were planned to be 16 weeks
long, with 20 sessions for CBT and 16 individual
sessions, plus the option of up to four additional
sessions with a significant other, for IPT. A different
group of experienced psychotherapists conducted
each of the treatments. The therapists were carefully
selected, received further training in their respective
treatment approaches and had to meet criteria for
conducting the treatment in a competent fashion
(see Elkin et al., 1985, for more details on selection
and training of therapists and Hill, O’Grady, &
Elkin, 1992, for therapists’ adherence to their
respective treatment protocols). There are eight
therapists in the CBT condition (two psychiatrists
and six psychologists) and nine therapists (six
 56 I. Elkin et al.
psychiatrists and three psychologists) in IPT. An
effort was made to keep the number of patients per
therapist in the current study in relative proportion
to the number they saw in the total sample.
Engagement Measures
Barrett-Lennard Relationship Inventory
(BLRI)
The BLRI (Barrett-Lennard, 1962, 1969) was
developed to measure the ‘‘necessary and sufficient
conditions’’ for therapeutic change postulated by
Carl Rogers (1957): The patient’s perception of the
therapist’s Level of Regard, Empathic Understand-
ing, Congruence, and Unconditionality of Regard.
There is substantial evidence for the reliability and
validity of the BLRI (Barrett-Lennard, 1986; Gur-
Downloaded by [University Of Maryland] at 03:39 15 October 2014
man, 1977). In the TDCRP, the alpha coefficient for
a composite of the totals of the first three scales was
.93, but was lowered to .85 when the fourth scale
was added. For this reason, the BLRI score used in
the current analyses was derived by totaling a
patient’s scores on the first three scales, Level of
Regard, Empathic Understanding, and Congruence.
This measure was used in a previous study by the
Elkin and colleagues (Elkin et al., 1999). Blatt et al.
(1996), using data from the TDCRP, made a similar
decision, based on their factor analysis of the four
scales, in which the first three shared high loadings
on a single factor.
Modified Vanderbilt Therapeutic Alliance Scale
(VTAS)
The therapeutic alliance measure used in this study
is a modification of the VTAS (Hartley & Strupp,
1983). The original 44-item scale was revised (and
seven items dropped) by Krupnick and associates
(Krupnick et al., 1994, 1996) to make it more
applicable to the different treatments in the TDCRP.
Ratings on the scale, made for the studies reported
by Krupnick and colleagues, were carried out by
trained raters after viewing a complete videotape of a
session. Three sessions (generally, 3, 9, and 15) were
rated for most patients, but only the Session 3
ratings were used in the current study, as a measure
of early engagement in therapy. (For patients who
completed only two sessions, hour 2 had been
substituted for hour 3.) A factor analysis by Krup-
nick et al. (1996) identified two factors: a Patient
factor and a Therapist factor. Only the Patient factor
was used in the current study, since it was most likely
to reflect patient engagement in treatment. In addi-
tion to 13 items related to the patient’s behavior
(e.g., patient acknowledges problems therapist could
help him/her with), this factor includes seven items
related to the interaction between patient and
therapist (e.g., patient and therapist work in joint
effort). The inter-rater reliability (ICC) reported by
Krupnick et al. (1996) for the Patient factor was .75
and the alpha coefficient for the score based on third
session ratings was .92.
Early Termination
For early termination analyses, patients were divided
into those with more than four sessions and those
with four or fewer sessions. The assumption was
that, if a patient continued for more than four
sessions, at least some initial engagement had
occurred. Reasons for termination were obtained
from the TDCRP Termination Report completed by
the research assistant, supplemented by information
in the clinical evaluator and therapist Case Summa-
ries and raters’ notes on sessions. Although most of
the nine patients who had four or fewer sessions
terminated (and in one case, was terminated by the
research staff) for treatment-related reasons, one
terminated because of improvement, and two or
three others may have terminated for other than
treatment-related reasons or for a combination of
treatment and non-treatment related reasons.
Responsiveness Scale
Development of the Scale
As discussed in the introduction, the Therapist
Responsiveness Scale was developed to test the
hypotheses in the current study, using videotapes
collected in the TDCRP. (At all stages of this
research, anyone viewing the tapes was required to
sign a confidentiality agreement.) A preliminary
version of the scale was used in the last stage of
development, prior to training raters for this study.
In this stage, four clinical researchers, two PhDs
(one a psychologist, the other a social worker) and
two MSWs, all with considerable clinical experience,
spent 7 months, meeting bi-weekly, to view and rate
videotapes, check discrepancies, and assess reliabil-
ity. The researchers added and subtracted items,
refined definitions and examples for each of the
items, and developed a detailed manual to be used in
training raters. (A few further changes were made to
the manual during the training process, prior to the
actual rating of tapes for this study.) The manual
(Elkin & Smith, 2007) provided general instructions
for raters, as well as definitions and examples for the
items.
There are three components of the scale. Part I
consists of 11 items that are rated, on a scale from 0

to 4, at 5-minute intervals. These include eight
positive items (makes eye contact, uses minimum
encouragers, focuses on and demonstrates interest in
the patient, makes an effort to understand from
patient’s perspective, on topic, responds to verbally
expressed feelings, makes inferences regarding un-
expressed content or feelings, and makes affirming,
validating, normalizing, and/or optimistic state-
ments) and three negative items to reflect the relative
absence of responsiveness (disrupts the flow of the
session, lectures patient, and makes critical, judg-
mental, countering, minimizing, or invalidating
comments). The decision to rate these items every
5 minutes was based on our experimenting with
different length intervals and deeming the 5-minute
segments most compatible with the concepts under-
lying the items (see Hill & Lambert, 2004, on choice
of units).
Downloaded by [University Of Maryland] at 03:39 15 October 2014
Part I was the primary focus of our original efforts
in developing the scale. After further viewing and
rating of videotapes and further consideration of the
clinical literature, it became clear that there might be
limitations in restricting the scale to these fairly
specific items, rated on the basis of short intervals of
time. Perhaps we would be missing important
contributions of the therapist to the global atmo-
sphere of these early sessions, which could reflect a
more general responsiveness of the therapist to the
person entering treatment and his/her communica-
tion. We, therefore, added Part II of the scale, which
consists of items that are rated globally (on a scale
from 0 to 4) and are based on the entire session.
Four of these items (compatible level of discourse,
appropriate level of emotional quality and intensity,
caring/compassionate, and respectful) are specifi-
cally focused on the overall therapeutic atmosphere,
and are included in the analyses in this study. We also
included a few items in the scale to measure patient
behaviors that might serve as control and/or moder-
ating variables. Of particular interest in the current
study is a global item ‘‘patient exhibits resistant and/
or hostile behavior’’ (also rated on a scale from 0 to
4), and it is included in some exploratory analyses.
Finally, a third section was added to the scale, in
an effort to tap the rater’s global impressions of the
therapist and patient. Only one therapist item in Part
III is particularly relevant for the current study, a
global rating of therapist responsiveness. The defini-
tion of this item is the same as that presented for the
construct of responsiveness: ‘‘The degree to which
the therapist is attentive to the patient; is acknowl-
edging and attempting to understand the patient’s
current concerns; is clearly interested in and re-
sponding to the patient’s communication, both in
terms of content and feeling; and is caring, affirming,
Therapist responsiveness 57
and respectful towards the patient’’ (Elkin & Smith,
2007).
Selection and Training of Raters
Eleven master’s level social work students responded
to a posting for raters in this study. Individual
interviews were held with each of them. Six candi-
dates were selected for additional screening, based
on whether they had had at least some previous
clinical experience, demonstrated interest in and
understanding of clinical research, and were avail-
able to participate fully in the study. They received
background information on the TDCRP, were
introduced to the responsiveness instrument and
manual, viewed and rated one videotape of a
psychotherapy session, and discussed their ratings.
Based both on their aptitude for the rating task and
their ability to discuss discrepancies in ratings, four
raters were selected for the study.
The four raters participated in 100 hours of
training, meeting for 4 hours bi-weekly over a period
of 3 months. (Trainers were the first and third
authors of this article and had been heavily involved
in the development of the instrument.) Although the
raters were trained extensively in the use of the
responsiveness instrument and had basic knowledge
of the TDCRP, they were not aware of the specific
hypotheses of this study. Training consisted of
studying the rating manual, rating sessions (largely
from the pilot phase of the TDCRP), and discussing
ratings in group meetings with the trainers, to clarify
misunderstandings and improve reliability. Since
factor analyses were not carried out until a sufficient
N was available in the study itself, the major inter-
rater reliabilities calculated prior to beginning the
study were those for two a priori scales based on the
eight positive and three negative items in Part I of the
scale. ICCs (intraclass correlations), using a two-way
random effects model for the average ratings of two
raters, were .88 for the positive scale and .67 for the
negative scale.
The original design called for each rater to rate 36
patients, paired an equal number of times with every
other rater. Following training, one of the raters was
no longer able to take part in the study. Of the 36
patients originally assigned to this rater, 12 were
rated by one of the three remaining raters to whom
these patients had not previously been assigned, and
24 were rated by a PhD student who had taken part
in the last stage of developing the scale, prior to the
training of raters. Before rating tapes in the current
study, she received brief retraining on the final
version of the scale and manual, using videotapes
of the first and second sessions of six of the patients
in the rater reliability study, followed by a reliability
 58 I. Elkin et al.
check on the remaining six patients. The final
average inter-rater reliabilities, prior to the rating of
study tapes, including her ratings, were .86 for the
positive a priori scale and .68 for the negative scale.
Procedures
The order of rating the videotapes of the 72 patients
was randomized, with two restrictions: That every
successive group of four patients include two CBT
and two IPT patients and that no therapist succeed
him/herself. Groups of four were then randomly
assigned to pairs of raters, ensuring that each rater
would rate an equal number of CBT and IPT
patients and would not rate the same therapist twice
in a row. The two raters assigned to each patient
rated the videotapes independently, viewing the first
and second sessions consecutively. Raters took
Downloaded by [University Of Maryland] at 03:39 15 October 2014
detailed notes while watching a videotape and were
encouraged to refer to these notes while filling out
the rating form. They were also instructed to refer to
the manual whenever there was a question about the
definition or anchors for an item. The items vary in
terms of the amount of inference involved, but raters
were asked to rely as much as possible on the
therapist’s actual behavior.
As in the training phase, the videotape was
stopped every 5 minutes, in order to rate the items
in Part I of the scale. At the end of the videotape,
ratings were made on all of the global items. During
the course of rating the study videotapes, two
reliability checks were carried out. Following each
of these, a brief retraining was held in order to
minimize rater drift. This consisted of two 4-hour
sessions, in which videotapes previously rated were
viewed and reasons for discrepancies in ratings
discussed.
When all videotapes had been rated, the scores for
each item were averaged across two raters and across
both sessions. (In the case of five patients who did
not have a second session, the first session score was
averaged across the two raters.) These average scores
were then submitted to a principal axis factor (PAF)
analysis with varimax rotation. A separate PAF
analysis with varimax rotation was carried out for
the 11 items in Part I, which were rated every 5
minutes, and for the four global items, measuring the
therapeutic atmosphere, in Part II. The four factors
identified in these analyses and their respective items
are presented in Table I. Also presented, for each of
the factors, are the inter-rater reliabilities (ICCs) and
measures of internal consistency (alphas). Since the
major analyses in the study include an item rating
global responsiveness, its ICC is also included in
Table I.
Table I. Reliabilities of responsiveness factors (and item)
Factors with items ICC Alpha
Attentiveness .898 .751
Makes eye contact
Minimum encouragers
Focus/demonstrates interest
On topic
Early empathic responding .667 .726
Understands from patient’s perspective
Responds to expressed feelings
Inferences re. Unexpressed feelings
Negative therapist behavior .737 .697
Disrupts flow
Lectures
Critical/judgmental
Positive therapeutic atmosphere .737 .897
Caring/compassionate
Respectful
Compatible level of discourse
Appropriate emotional quality/intensity
Global Responsiveness Item .696
Note. ICC Intraclass correlation.
The first two factors, labeled ‘‘Attentiveness’’
(eigenvalue of 3.55, accounting for 44% of the
variance) and ‘‘Early Empathic Responding’’ (eigen-
value of 1.35, accounting for 17% of the variance),
were derived from the positive 5-minute items. The
four items on the Attentiveness factor, in the order of
their factor loadings, are: Makes eye contact, focuses
on and demonstrates interest in the patient, uses
minimum encouragers, and on topic. The Early
Empathic Responding factor consists of three items:
Makes an effort to understand from the patient’s
perspective, makes inferences regarding unexpressed
content or feelings, and responds to verbally ex-
pressed feelings. (The ‘‘affirming’’ item did not load
on either of these factors.)
The third factor (eigenvalue of 1.06, accounting
for 13% of the variance), composed of the three
negative 5-minute items (makes critical, judgmental,
countering, minimizing or invalidating comments,
lectures patient, and disrupts the flow of session), is
labeled ‘‘Negative Therapist Behavior.’’ All four of
the global items relating to the therapeutic atmo-
sphere (caring/compassionate, appropriate level of
emotional quality and intensity, respectful, and
compatible level of discourse) loaded on a single
factor, labeled ‘‘Positive Therapeutic Atmosphere’’
(eigenvalue of 2.88, accounting for 72% of the
variance). The items included on all four of the
factors had loadings .50 on their respective factors.
Table II presents the inter-correlations of the four
factors and the global responsiveness item and the
three engagement variables. The Negative Therapist
Behavior factor had a large positive skew (coefficient
of variation 1.38). It was therefore transformed
 Therapist responsiveness 59
Table II. Correlations of responsiveness factors (and item) with engagement variables

1 2 3 4 5 6 7 18

1. Attentiveness _
2. Early empathic responding .38*** _
3. Negative therapist behaviora .31** .14 _
4. Positive therapeutic atmosphere .52*** .19 .71*** _
5. Global responsiveness item .57*** .45*** .61*** .85*** _
6. VTAS .24 .06 .16 .21 .15 _
7. BLRI .17 .05 .11 .28* .22 .01 _
8. Early terminationb .07 .05 .31** .30* .34** .22 .20 _

Note. VTAS Vanderbilt Therapeutic Alliance Scale (Patient Factor); BLRI Barrett-Lennard Relationship Inventory.
N 72 but is reduced to 67 for correlations with the VTAS and the BLRI.
a
The negative therapist behavior factor was transformed using a reciprocal transformation, which reverses the direction of the correlation
coefficient.
b
Correlations with the early termination dichotomy are point-biserial correlations.
*pB.05 (two-tailed); **p B.01 (two-tailed); ***p B.001 (two-tailed).

using a reciprocal transformation (transformed coef- includes an eight-item measure of ‘‘Facilitative

ficient of variation .14). It should be noted that the Conditions’’ (FC). To test the convergent validity
Downloaded by [University Of Maryland] at 03:39 15 October 2014

reciprocal transformation reverses the order of scores
on a measure, so that the largest score becomes the
smallest score. All results using the Negative Thera-
pist Behavior factor must therefore be interpreted
accordingly.
In order to get some indication of the construct
validity of the Therapist Responsiveness Scale,
scores on each of the three positive factors and the
overall responsiveness item were correlated with
relevant scales on the Collaborative Study Psy-
chotherapy Rating Scale (CSPRS). The CSPRS
was developed by Hollon and colleagues (Hollon et
al., 1988), in the context of the TDCRP, primarily to
measure therapists’ adherence to their respective
treatment modalities. The scale was developed using
audio-tapes from the pilot/training phase of the
TDCRP, and it was applied to the audio-tapes in
the current study by Hill et al. (1992). In addition to
scales measuring CBT, IPT, and pharmacotherapy
(clinical management) techniques, the CSPRS also
Table III. Correlations of positive responsiveness factors and
CSPRS scores (Session 1)
CSPRS CSPRS
FC CB
Responsiveness Factor (N 70) (N 35)
of the Therapist Responsiveness Scale factors, they
were related to the FC scale. Of particular interest
was the relationship of the ‘‘Positive Therapeutic
Atmosphere’’ factor to the FC scale, which includes
such items as ‘‘supportive encouragement’’ and
‘‘involvement.’’ The responsiveness factors were not
expected to be as strongly related to the adherence
scales for CBT and IPT, and these relationships were
considered an indication of discriminant validity.
The relationships of the positive responsiveness
factors and the overall responsiveness item to the
CSPRS scales are shown in Table III. The correla-
tions in the table are based on first session ratings,
since there were no second session ratings made on
the CSPRS. As anticipated, the three positive
responsiveness factors (and the global responsiveness
item) were all significantly related to the Facilitative
Conditions scale, providing some support for con-
vergent validity. The highest correlation was with the
Positive Therapeutic Atmosphere factor. Correla-
tions with the adherence scales (the CB scale for
patients in Cognitive Therapy and the IPT scale for
patients in Interpersonal Psychotherapy) were lower
than those with FC and none reached significance,
CSPRS providing some support for discriminant validity.
IPT
(N35)

Attentiveness .25* .08 .13 Results

Early empathic responding .38*** .11 .14
Positive therapeutic .44*** .17 .19
Major Analyses
atmosphere To test the major hypothesis that therapist respon-
Global responsiveness Item .41*** .18 .22
siveness would predict patient early engagement in
Note. CSPRS FC Collaborative Study Psychotherapy Rating treatment, separate regression analyses were carried
Scale, Facilitative Conditions scale; CSPRS CB Collaborative out, relating each of the four responsiveness factors
Study Psychotherapy Rating Scale, Cognitive Therapy scale;
to each of the engagement variables. The analyses
CSPRS IPT Collaborative Study Psychotherapy Rating Scale,
Interpersonal Psychotherapy scale.
with the VTAS and the BLRI as dependent variables
*p B.05 (two-tailed); **p B.01 (two-tailed); ***p B.001 (two- were two-level hierarchical linear regression models
tailed). (HLM; Raudenbush & Bryk, 2002), with the patient
 60 I. Elkin et al.

at level one, and the therapist at level two. A two-
level analysis was chosen in order to partition off the
variance in the engagement variables due to therapist
variation in these variables. All predictors were
entered at level one. In the case of early attrition,
analyses were simple logistic regression models.
These models did not include a second level to
partition off therapist variance, since there were only
nine early terminators and 17 therapists in the study.
In each model considered, severity of depressive
symptomatology, as measured by the pre-treatment
HRSD, as well as the treatment variable, were
included as controls. The findings in the literature
on the relationship of initial severity of symptoma-
tology to the alliance have been mixed (Gibbons et
al., 2003). Because of the importance of initial
severity in previous TDCRP findings, however
(e.g., Elkin et al., 1995), it is being used as a control
Downloaded by [University Of Maryland] at 03:39 15 October 2014
in each model. (Interactions are not reported for the
early termination variable, since there were only nine
early terminators.) The results of these analyses are
presented in Table IV. Table IV also includes the
results of analyses with the global overall responsive-
ness item in place of a responsiveness factor.
In the analyses using the VTAS as the engagement
variable, there was no main effect for any of the
responsiveness factors, and no interaction effects
with treatment.
In the analyses with the BLRI as the dependent
variable, there was a significant main effect for
severity (p.017), with higher initial scores on the
HRSD related to lower scores on the BLRI, but no
effect for treatment. The positive therapeutic atmo-
sphere factor had a significant effect (p .025), with
higher positive therapeutic atmosphere scores related
to higher scores on the BLRI. The global respon-

variable in these analyses. Although responsiveness is
conceptualized as a common factor that should have
effects across treatments, the possibility that there
could be differences due to treatment led to its
inclusion in the analyses. Finally, the individual
responsiveness factor and the interaction of the
factor with treatment, to test for a possible moderat-
ing effect of treatment on the relationship of respon-
siveness to the engagement variables, were included
siveness item, which is highly correlated with the
positive therapeutic atmosphere factor, was also
significantly positively related to the BLRI (p
.034). All of the effects on the BLRI were over and
above those explained by treatment and severity of
depression. There were no interaction effects with
treatment. Pre-treatment severity was significantly
related to early termination (p .005; OR1.265),
with more severely depressed patients more likely to

Table IV. Regression analyses predicting engagement from therapist responsiveness

Engagement variables
Predictor
a
VTAS BLRIa Early terminationb

B SE P B SE p x2 for change OR p

Attentiveness .07 .05 .23 1.64 1.16 .16 1.15 .71 .28
Early empathic .003 .06 .96 1.51 1.36 .27 .26 .81 .61
responding
Negative therapist .42 .55 .45 11.53 11.92 .34 6.65 .00 .01
behaviorc
Positive .03 .03 .32 1.53 .67 .025 6.48 .63 .01
therapeutic
atmosphere
Global responsiveness item .07 .10 .52 4.85 1.84 .034 11.46 .09 .001
N 67 67 72

Therapist variation in engagement variablese

Outcome measure SD Variance df x2 p
component

VTAS .127 .016 15 16.31 .36

BLRI 2.60 6.77 15 16.04 .38

Note. VTAS Vanderbilt Therapeutic Alliance Scale; BLRI Barrett-Lennard Relationship Inventory; OR odds ratio.
a
VTAS and BLRI analyses are two-level linear regression models.
b
Early termination analyses are logistic regressions.
c
A separate regression analysis was carried out for each predictor, with pretreatment severity (HRSD; Hamilton Rating Scale for
Depression) and treatment condition, CBT (Cognitive Behavior Therapy) or IPT (Interpersonal Psychotherapy), included as controls.
Results in the table are main effects (no significant interactions with treatment were found).
d
The negative therapist behavior factor was transformed using a reciprocal transformation which reverses the direction of the finding.
e
Each equation includes control variables for initial severity and treatment modality.

be early terminators. The positive therapeutic atmo-
sphere factor and the global responsiveness rating
were both significantly related to early termination
(p.011; OR.628 for PTA and p .001; OR 
.085 for global responsiveness), with patients with
higher positive therapeutic atmosphere and global
responsiveness scores less likely to terminate early.
The Negative Therapist Behavior factor was also
significantly related to early termination (p .010,
OR.001), with higher scores for patients who
terminated early. All of the effects on early termina-
tion were over and above those explained by treat-
ment and severity of depression.
There were no significant effects for either the
Attentiveness or Early Empathic Responding factors
on any of the engagement variables.
Downloaded by [University Of Maryland] at 03:39 15 October 2014
Exploratory Analyses
As mentioned in the introduction, we also carried
out exploratory analyses using an item labeled
‘‘patient exhibits resistant and/or hostile behavior,’’
which was rated globally on a 0 to 4 scale at the end
of viewing each session. Behaviors rated as resistant
ranged from skepticism and pessimism about the
treatment, the therapist’s ability to help, and/or the
program, through problems with specific aspects of
treatment (e.g., homework, focus on cognitions, not
focusing on the past), to hostility toward the
therapist, treatment, or program. Since the resis-
tance item is not a pre-treatment variable, it cannot
technically be considered a ‘‘moderator.’’ We were,
nevertheless, particularly interested in the interac-
tion of this item with the responsiveness factors:
Might they be differentially related to the engage-
ment variables, depending on the degree to which
the patient’s behavior was rated as ‘‘resistant?’’.
Examination of the resistance data revealed an
unusual distribution of scores. Only 24 of the 72
patients had any rating above zero on this item.
Accordingly, the sample was divided into the 24
patients with and the 48 patients without resistance
ratings, and this dichotomy was used in the analyses.
The kappa for the dichotomized ratings is .58.
Analyses were carried out separately for each of the
factors and the overall responsiveness item, using
regression models similar to those described above
for the primary analyses. Each model considered
included the dichotomized resistance item, a respon-
siveness factor, and the resistance by responsiveness
factor interaction, which was the main focus of these
analyses. Severity was again included as a control
variable, as in the primary analyses. (Treatment was
not included in these analyses, since it had not had a
significant effect in the major analyses, and adding
Therapist responsiveness 61
Figure 1. Interaction of patient resistance and positive therapeu-
tic atmosphere.
another variable would have reduced the statistical
power of these analyses.)
In the analyses with the VTAS as the dependent
variable, the resistance item, itself, was a strong
predictor of the VTAS scores (B.45, SE.11,
p .000), with lower scores for patients rated as
resistant. There were no main effects for any of the
responsiveness factors, but there was a significant
interaction with resistance for the Positive Thera-
peutic Atmosphere factor (B.15, SE.05,
p .006) and an interaction at a trend level (B 
.35, SE.18, p .056) for the global responsiveness
item. Figure 1 illustrates the nature of the interaction
for the positive therapeutic factor, suggesting that
this factor has a positive relationship to the VTAS
only for those patients rated as resistant. The
interaction for the global responsiveness item has a
similar trajectory.
In contrast to the findings for the VTAS, resis-
tance did not have a significant effect on the BLRI or
early termination, nor were there significant interac-
tions, on the BLRI, of resistance with any of the
responsiveness factors.
Discussion
In the major analyses in this study, the Positive
Therapeutic Atmosphere factor appears to be the
most important component of therapist responsive-
ness in the first two sessions of therapy. It is a
significant predictor both of the patient’s positive
assessment of the therapeutic relationship on the
BLRI and of the patient’s remaining for more than
 62 I. Elkin et al.
four sessions. In the exploratory analyses discussed
below, it also predicts the VTAS for patients rated as
resistant to treatment. Thus, it may play an impor-
tant role for all three of the engagement variables.
The global responsiveness item, which is highly
correlated with the positive therapeutic atmosphere
factor, also has a significant relationship with both
the BLRI and early termination. The fact that there
was no significant interaction between the positive
therapeutic atmosphere factor and treatment pro-
vides some support for the positive therapeutic
atmosphere in early sessions to be considered a
‘‘common factor’’ in therapy process.
This factor draws together, in a comprehensive
measure, aspects of the therapist’s behavior*caring/
compassion, respect, and attunement to the patient’s
affect*which serve to create a positive atmosphere in
which therapeutic work can begin. Such a positive
Downloaded by [University Of Maryland] at 03:39 15 October 2014
atmosphere is probably conducive to patient progress
throughout the course of therapy (as has been posited
by theorists ranging from Rogers to Beck). What is
most important in the current study is the finding
that this atmosphere can be reliably measured in the
first two sessions and that it is related to ensuing
indicators of patient early engagement in treatment.
In contrast to the positive findings for the global
positive therapeutic atmosphere factor, the results for
the attentiveness and early empathic responding
factors, based on items rated at 5-minute intervals,
were disappointing; they did not have a significant
relationship with any of the engagement variables.
Although these results may be due, in part, to
measurement issues (to be addressed below), it is
important to recognize that it may be precisely the
more general therapeutic atmosphere that is most
important for patients when they first enter the
treatment situation, rather than the specific nature
of therapist responses, even those that convey the
therapist’s moment-to-moment attentiveness or early
empathic responding. The global items composing
the positive therapeutic atmosphere factor were, in
fact, added during scale development, due to the
impression that the more specific items, rated every 5
minutes, were missing important contributions of the
therapist in creating a positive atmosphere in which
the patient could feel truly responded to as a person.
Although they did not predict the engagement
variables, both the attentiveness and early empathic
responding factors were significantly correlated with
the global responsiveness item and attentiveness was
significantly correlated with positive therapeutic
atmosphere. Raters may, thus, have taken their
cumulative impression of the more specific items in
these factors into account in making their global
ratings at the conclusion of the session. It would be
of interest to see whether more global ratings of
attentiveness and early empathic responding might
themselves be related to the engagement variables.
Global ratings of the ‘‘affirming’’ item, which did not
load on any of the 5-minute factors, would also be of
interest.
Although the attentiveness and early empathic
responding factors were not related to any of the
engagement variables, the negative therapist
behavior factor, also rated at 5-minute intervals,
was significantly related to early termination. Thus,
it seems that some patients whose therapists engage
in negative behaviors respond by leaving at an early
point in therapy. As Garfield wrote some years ago, in
discussing the importance of initial therapist-patient
interactions, ‘‘if these are quite negative the relation-
ship may be terminated before it has a chance to
develop’’ (Garfield, 1980, p. 96). In general, thera-
pists in this study did not receive high scores on the
negative therapist behavior factor; even for the early
terminators, the mean for negative therapist behavior
ratings was only .33 on a scale of 0 to 4 (compared to
.14 for the remainers). But, as Binder and Strupp
(1997) have pointed out ‘‘the presence of even
relatively low levels of negative therapist behavior
may be sufficient to prevent change’’ (p. 126), or, in
this case, may lead to very early termination. The low
frequency of negative therapist behaviors may make it
necessary to measure them at frequent intervals,
such as the 5-minute segments in this study.
In turning to the results of analyses including the
resistance item, we must again emphasize that the
resistance analyses are exploratory, especially since
the rating of resistance is based on a single item. In
the VTAS analyses, as mentioned above, the ex-
pected interaction between resistance and a respon-
siveness factor did occur, but only for the positive
therapeutic atmosphere factor. There was also a
trend for an interaction with the global responsive-
ness item, but, again, there were no findings for
attentiveness or early empathic responding. Figure 1
illustrates that the positive therapeutic atmosphere
factor had an effect on the VTAS only for those
patients who were seen as exhibiting resistant and/or
hostile behavior. Thus, the provision of a positive
therapeutic atmosphere may help to mitigate the
patient’s ‘‘negative attitude’’ in regard to the thera-
pist, the treatment, or the program.
The impact of patient resistance on the VTAS is
further underscored by the fact that the resistance
item itself accounts for over 20% of the variance on
the VTAS. In understanding the effects of the patient
resistance item on the VTAS, it is important to note
that both of these measures are based on observers’
ratings (albeit different groups of observers).
Further, there are even a few items on the patient
alliance measure which overlap with the construct of

resistance, e.g., ‘‘patient acts in hostile, attacking, or
critical manner.’’ Thus, we may, in part, be predict-
ing the more negative component of the VTAS
patient factor in the third session from similar
behaviors in the first and second sessions. What
seems most important in the current findings is the
fact that this behavior can be seen very early, from
the very beginning of therapy.
The major findings in this study and the explora-
tory findings demonstrate that it is possible to
measure therapist behaviors in the first two sessions
that are related to ensuing measures of patient
engagement in treatment. This has implications for
both researchers and clinicians. Although, as indi-
cated in the introduction, there has been a great deal
of literature on the relationship of measures of the
therapeutic relationship and the therapeutic alliance
early in treatment to outcome, few researchers have
Downloaded by [University Of Maryland] at 03:39 15 October 2014
attempted to identify therapist behaviors related to
the initial development of the relationship and
alliance. The current study and its findings will
hopefully encourage others to pursue such research,
in order to increase our understanding of this very
early phase of treatment.
The importance of therapist behaviors in early
sessions that may contribute to patient engagement
should also be of interest to therapists and those who
train them. Drawing on the literature relating the
therapeutic alliance to outcome, Crits-Christoph
and colleagues (Crits-Christoph et al., 2006) have
begun to train therapists in ‘‘alliance-fostering’’
techniques. Some of the techniques they consider
‘‘alliance-fostering’’*a sense of caring, respect, ac-
ceptance, warmth, and positive regard*are similar
to aspects of the positive therapeutic atmosphere
factor in the current study. Although these research-
ers focused on techniques used throughout the
course of therapy, such alliance-building would
seem to be especially important in the very early
sessions in order to engage patients in the treatment
process.
If the findings in regard to the importance of the
patient’s ‘‘resistant’’ behavior are replicated, they
would have clear implications for the clinician: To be
particularly alert to such behavior, even in the first
session, and to try to be responsive to the patient’s
concerns. Wachtel (1999) has emphasized that it is
not helpful to ‘‘blame’’ the patient, but that, in fact,
what is considered ‘‘resistant’’ behavior can be ‘‘a
very useful phenomenon for the therapeutic effort if
properly understood and worked with’’ (p. 106). It
might also be helpful to consider the use of
adjunctive procedures. Westra (2004), for example,
has suggested the use of Motivational Interviewing as
an adjunct to traditional CBT for patients ambiva-
lent about or resistant to the therapy. It is interesting
Therapist responsiveness 63
to note that, in the current study, two-thirds of the
patients receiving resistance ratings were in CBT.
This seemed to be due, at least in part, to patients’
problems with aspects of the CBT condition that
were not present in IPT (e.g., homework assign-
ments, twice a week sessions in the early part of
treatment, and the focus on negative cognitions).
Although the exploratory findings with the resis-
tant/hostile behavior item were promising, it would
clearly have been preferable to have a multi-item
measure of this variable, as well as measures of other
relevant patient behaviors. The Responsiveness Scale
was developed to test the hypotheses in this study,
and this is the first report of findings using the scale.
Further work would be needed to improve this
instrument if it were to be used in other studies.
This could include adding or subtracting items,
refining the definition of current items, and includ-
ing more examples in the manual. As suggested
above, it would also be of interest to include global
ratings of ‘‘attentiveness’’ and ‘‘early empathic re-
sponding.’’ Another approach to studying the impact
of responsiveness would be to provide more specifi-
city to the construct, e.g., by focusing on therapists’
responsiveness to particular patient communica-
tions, as in the study by Falconnier and Elkin
(2008), in which the focus was on therapists’
responsiveness to patients’ communication about
economic stressors.
In addition to possible limitations in this study due
to the instrument used to measure responsiveness,
there is a limit to the generalizability of the results on
one of the engagement variables due to the small
number of early terminators (nine out of 72 patients).
If this finding were to be replicated, however, it
would have implications for the measurement of
patient engagement in therapy. Early termination
was, in fact, the most predictable of the engagement
variables, having a significant relationship to positive
therapeutic atmosphere, negative therapist beha-
viors, and the global responsiveness item. Since
four patients terminated before we could obtain their
VTAS or BLRI scores, they could not even be
included in the analyses of those variables. Most
studies of the relationship between therapist beha-
viors and the therapeutic alliance or relationship do
not consider the fact that some patients may have
already ‘‘voted with their feet’’ and left therapy before
alliance or relationship measures could be obtained.
Measuring three different components of patient
engagement allowed us to get a more nuanced
picture of the relationship of therapist behaviors to
engagement than we might otherwise have had. The
positive therapeutic atmosphere factor and global
responsiveness item were more highly related to the
BLRI and early termination than to observors’
 64 I. Elkin et al.
ratings on the VTAS. For the patients rated as
resistant, however, the positive therapeutic atmo-
sphere was related to the VTAS. Therapist behaviors
in early sessions may thus have different impacts on
how the patient feels about the therapy and therapist
and on the extent to which the patient is seen as
contributing to the therapeutic alliance. The differ-
ent findings for the three engagement variables, and
the non-significant correlations among them, lead to
the suggestion to include all three components in
future studies of early patient engagement in treat-
ment.
Acknowledgements
The NIMH Treatment of Depression Collaborative
Research Program was initiated and supported by
the National Institute of Mental Health. The pro-
Downloaded by [University Of Maryland] at 03:39 15 October 2014
gram was funded by cooperative agreements to six
participating sites. The three research sites and their
Principal Investigators and Project Coordinators
were: George Washington University (MH 33762),
Stuart M. Sotsky and David R.Glass; University of
Pittsburgh (MH33753), Stanley D. Imber and Paul
A. Pilkonis, and University of Oklahoma (MH
33760), John T. Watkins and William R. Leber.
The three sites responsible for training therapists and
their Principal Investigators and Project Coordina-
tors were Yale University (MH 33827), Myrna M.
Weissman, Eve S. Chevron, and Bruce J. Rounsa-
ville; Clarke Institute of Psychiatry (MH 38231),
Brian F. Shaw and T. Michael Vallis; and Rush
Presbyterian-St. Luke’s Medical Center (MH
35017), Jan A. Fawcett and Phillip Epstein. The
principal NIMH collaborators were Irene Elkin,
Coordinator, M. Tracie Shea, Associate Coordina-
tor, Morris B. Parloff, and John P. Docherty.
Collaborators in data management and data analysis
were C. James Klett, Joseph F. Collins, and Roderic
Gillis, V.A. Cooperative Studies Program, Perry
Point, MD.
The authors thank Janice Krupnick and Stuart
Sotsky for access to the VTAS data and Clara Hill for
access to the CSPRS data. They also acknowledge
the contribution of Susan Connors-Herrera, Kelly
Gartland, Jessie Beebe, and David Orlinsky at
various stages of this research.
References
Ackerman, S. J., & Hilsenroth, M. J. (2001). A review of therapist
characteristics and techniques negatively impacting the ther-
apeutic alliance. Psychotherapy: Theory, Research, Practice,
Training, 38, 171185. doi:10.1037/0033-3204.38.2.171
Ackerman, S. J., & Hilsenroth, M. J. (2003). A review of therapist
characteristics and techniques positively impacting the ther-
apeutic alliance. Clinical Psychology Review, 23, 133. doi:10.
1016/S0272-7358(02)00146-0
Alexander, L. B., & Luborsky, L. (1986). The Penn Helping
Alliance scales. In L. S. Greenberg & W. M. Pinsof (Eds.) The
psychotherapeutic process: A research handbook (pp. 325366).
New York, NY: Guilford Press.
Arnkoff, D. B., Glass, C. R., Shea, C. A., McKain, T. L., &
Sydnor-Greenberg, J. M. (1987). Client predisposition towards
cognitive and social skills treatments for shyness. Journal of
Integrative and Eclectic Psychotherapy, 6, 154164.
Arnow, B. A., Manber, R., Blasey, C., Klein, D. N., Blalock, J. A.,
Markowitz, J. C., Rothbaum, B. O., . . . Keller, M. B. (2003).
Therapeutic reactance as a predictor of outcome in the treat-
ment of chronic depression. Journal of Consulting and Clinical
Psychology, 71, 10251035. doi:10.1037/0022-006X.71.6.1025
Barrett-Lennard, G. T. (1962). Dimensions of therapist response
as causal factors in therapeutic change. Psychological Mono-
graphs, 76 (43, Whole No. 562).
Barrett-Lennard, G. T. (1969). Technical note on the 64-item
revision of the Relationship Inventory. Unpublished Manu-
script. University of Waterloo, Waterloo, Ontario.
Barrett-Lennard, G. T. (1986). The relationship inventory now:
Issues and advances in theory, method and use. In L. S.
Greenberg & W. M. Pinsof (Eds.) The psychotherapeutic process:
A research handbook (pp. 439476). New York, NY: Guilford
Press.
Beck, A. T., Rush, A. J., Shaw, B. F., & Emery, G. (1979).
Cognitive therapy of depression. New York, NY: Guilford Press.
Beutler, L. E., Malik, M., Alimohamed, S., Harwood, T. M.,
Talebi, H., Noble, S., & Wong, E. (2004). Therapist variables.
In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of
psychotherapy and behavior change (pp. 227306). New York,
NY: John Wiley & Sons.
Beutler, L. E., Harwood, T. M., Michelson, A., Song, X., &
Holman, J. (2011). Reactance/Resistance Level. In J. C.
Norcross (Ed.), Psychotherapy relationships that work: Evidence-
based responsiveness (pp. 129143). New York, NY: Oxford
Universisty Press.
Binder, J. L., & Strupp, H. H. (1997). Negative process: A
recurrently discovered and underestimated facet of therapeutic
process and outcome in the individual psychotherapy of adults.
Clinical Psychology: Science and Practice, 4, 121139. doi:10.
1111/j.1468-2850.1997.tb00105.x
Blatt, S. J., Zuroff, D. C., Quinlan, D. M., & Pilkonis, P. A.
(1996). Interpersonal factors in brief treatment of depression:
Further analyses of the National Institute of Mental Health
Treatment of Depression Collaborative Research Program.
Journal of Consulting and Clinical Psychology, 64, 162171.
doi:10.1037/0022-006X.64.1.162
Bohart, A. C., & Greenberg, L. S. (1997). Empathy and
psychotherapy: An introductory overview. In A. C. Bohart &
L. S. Greenberg (Eds.), Empathy reconsidered: New directions in
psychotherapy (pp. 331). Washington DC: American Psycho-
logical Association.
Castonguay, L. G., Constantino, M. J., & Holtforth, M. G.
(2006). The working alliance: Where are we and where should
we go? Psychotherapy: Theory, Research, Practice, Training, 43,
271279. doi:10.1037/0033-3204.43.3.271
Clarkin, J. F., & Levy, K. N. (2004). The influence of client
variables on psychotherapy. In M. J. Lambert (Ed.), Bergin and
Garfield’s handbook of psychotherapy and behavior change (pp.
194226). New York, NY: John Wiley & Sons.
Crits-Christoph, P., Connolly-Gibbons, M.B.C., Crits-Christoph,
K., Narducci, J., Schamberger, M., & Gallop, R. (2006). Can
therapists be trained to improve their alliances? A preliminary
study of alliance-fostering psychotherapy. Psychotherapy
Research, 16, 268281. doi:10.1080/10503300500268557

Elkin, I. (1994). The NIMH treatment of depression collaborative
research program: Where we began and where we are. In A. E.
Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and
behavior change (pp. 114149). New York, NY: John Wiley &
Sons.
Elkin, I., Gibbons, R. D., Shea, M. T., Sotsky, S. M., Watkins, J. T.,
Pilkonis, P. A., & Hedeker, D. (1995). Initial severity and
differential treatment outcome in the National Institute of
Mental Health Treatment of Depression Collaborative Research
Program. Journal of Consulting and Clinical Psychology, 63, 841
847. doi:10.1037/0022-006X.63.5.841
Elkin, I., Parloff, M. B., Hadley, S. W., & Autry, J. H. (1985).
NIMH Treatment of Depression Collaborative Research Pro-
gram: Background and research plan. Archives of General
Psychiatry, 42, 305316. doi:10.1001/archpsyc.1985.01790
260103013
Elkin, I., Shea, M. T., Watkins, J. T., Imber, S. D., Sotsky, S. M.,
Collins, J. F, . . . Parloff, M. B. (1989). National Institute of
Mental Health Treatment of Depression Collaborative Re-
search Program: General effectiveness of treatments. Archives of
General Psychiatry, 46, 971982. doi:10.1001/archpsyc.1989.
01810110013002
Downloaded by [University Of Maryland] at 03:39 15 October 2014
Elkin, I., & Smith, Y. (2007). Therapist responsiveness study
rating scale. Unpublished manuscript, School of Social Service
Administration, University of Chicago, Chicago, Illinois.
Elkin, I., Yamaguchi, J. L., Arnkoff, D. B., Glass, C. R., Sotsky, S.
M., & Krupnick, J. L. (1999). ‘‘Patient-treatment fit’’ and early
engagement in therapy. Psychotherapy Research, 9, 437451.
Elliott, R., Bohart, A. C., Watson, J. C., & Greenberg, L. S.
(2011). Empathy. In J. C. Norcross (Ed.), Psychotherapy
relationships that work: Evidence-based responsiveness (pp. 132
152). New York, NY: Oxford University Press.
Endicott, J., Spitzer, R. L., Fleiss, J. L., & Cohen, J. (1976). The
Global Assessment Rating Scale: A procedure for measuring
overall severity of psychiatric disturbance. Archives of General
Psychiatry, 33, 766771. doi:10.1001/archpsyc.1976.017700
60086012
Falconnier, L., & Elkin, I. (2008). Addressing economic stress in
the treatment of depression. American Journal of Orthopsy-
chiatry, 78, 3746. doi:10.1037/0002-9432.78.1.37
Farber, B. A., & Doolin, E. M. (2011). Positive regard and
affirmation. In J. C. Norcross (Ed.), Psychotherapy relationships
that work: Evidence-based responsiveness (pp. 168186). New
York, NY: Oxford University Press.
Freud, S. (1913). On beginning the treatment. Standard Edition,
12, 123144.
Garfield, S. L. (1980). Psychotherapy: An eclectic approach. New
York, NY: John Wiley & Sons.
Garfield, S. L. (1994). Research on client variables in psychother-
apy. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of
psychotherapy and behavior change (pp. 72113). New York, NY:
John Wiley & Sons.
Gibbons, M.B.C., Crits-Christoph, P., de la Cruz, C., Barber, J.
P., Siqueland, L., & Gladis, L. (2003). Pretreatment expecta-
tions, interpersonal functioning, and symptoms in prediction of
the therapeutic alliance across supportive-expressive psy-
chotherapy and cognitive therapy. Psychotherapy Research, 13,
5976. doi:10.1093/ptr/kpg007
Gurman, A. S. (1977). The client’s perception of the therapeutic
relationship. In A. Gurman & A. Razin (Eds.), Effective
psychotherapy: A handbook of research (pp. 503543). New York,
NY: Pergamon Press.
Hamilton, M. A. (1967). Development of a rating scale for primary
depressive illness. British Journal of Social and Clinical Psychology,
6, 278286. doi:10.1111/j.2044-8260.1967.tb00530.x
Hartley, D. E., & Strupp, H. H. (1983). The therapeutic alliance:
Its relationship to outcome in brief psychotherapy. In J. M.
Therapist responsiveness 65
Masling (Ed.), Empirical studies of psychoanalytic theories (pp. 1
37). Hillsdale, NJ: Erlbaum.
Henry, W. P., & Strupp, H. H. (1994). The therapeutic alliance as
interpersonal process. In A. O. Horvath & L. S. Greenberg
(Eds.), The working alliance: Theory research and practice (pp.
5184). New York, NY: John Wiley & Sons.
Hill, C. E., & Lambert, M. J. (2004). Methodological issues in
studying psychotherapy processes and outcomes. In M. J.
Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy
and behavior change (pp. 307389). New York, NY: John Wiley
& Sons.
Hill, C. E., O’Grady, K. E., & Elkin, I. (1992). Applying The
Collaborative Study Psychotherapy Rating Scale to rate thera-
pist adherence in cognitive-behavior therapy, interpersonal
therapy, and clinical management. Journal of Consulting and
Clinical Psychology, 60, 7379. doi:10.1037/0022-006X.60.1.73
Hollon, S. D., Evans, M. D., Auerbach, A., DeRubeis, R. J.,
Elkin, I., Lowery, A., . . . Piasecki, J. (1988). Development of a
system for rating therapies for depression: Differentiating
cognitive therapy, interpersonal psychotherapy and clinical
management pharmacotherapy. Unpublished manuscript, Van-
derbilt University, Nashville, TN.
Horvath, A. O. (1994). Research on the alliance. In A. O. Horvath
& L. S. Greenberg (Eds.), The working alliance: Theory, research
and practice (pp. 259286). New York, NY: John Wiley & Sons.
Horvath, A. O., & Bedi, R. P. (2002). The alliance. In J. C.
Norcross (Ed.), Psychotherapy relationships that work: Therapist
contributions and responsiveness to patients (pp. 3769). New
York, NY: Oxford University Press.
Horvath, A. O., Del Re, A. C., Fluckinger, C., & Symonds, D.
(2011). Alliance in individual psychotherapy. In J. C. Norcross
(Ed.), Psychotherapy relationships that work: Evidence-based
responsiveness (pp. 2569). New York, NY: Oxford University
Press.
Horvath, A. O., & Greenberg, L. S. (1994). Introduction. In A. O.
Horvath & L. S. Greenberg (Eds.), The working alliance: Theory,
research and practice (pp. 19). New York, NY: John Wiley &
Sons.
Hubble, M. A., Duncan, B. L., Miller, S. D., & Wampold, B. E.
(2010). Introduction. In B. L. Duncan, S. D. Miller, B. E.
Wampold & M. A. Hubble (Eds.), Heart and soul of change:
Delivering what works in therapy (pp. 2346). Washington DC:
American Psychological Association.
Jones, E. E. (2000). Therapeutic action: A guide to psychoanalytic
therapy. Northvale, NJ: Jason Aronson.
Kanfer, F. H., & Schefft, B. K. (1988). Guiding the process of
therapeutic change. Champaign, IL: Research Press.
Klerman, G. L., Weissman, M. M., Rounsaville, B. J., & Chevron,
E. S. (1984). Interpersonal psychotherapy of depression. New York,
NY: Basic Books.
Krupnick, J. L., Elkin, I., Collins, J., Simmens, S., Sotsky, S. M.,
Pilkonis, P. A., & Watkins, J. T. (1994). Therapeutic alliance
and clinical outcome in the NIMH Treatment of Depression
Collaborative Research Program: Preliminary findings.
Psychotherapy, 31, 2835. doi:10.1037/0033-3204.31.1.28
Krupnick, J. L., Sotsky, S. M., Simmens, S., Moyer, J., Elkin, I.,
Watkins, J., & Pilkonis, P. A. (1996). The role of the
therapeutic alliance in psychotherapy and pharmacotherapy
outcome: Findings in the National Institute of Mental Health
Treatment of Depression Collaborative Research Program.
Journal of Consulting and Clinical Psychology, 64, 532539.
doi:10.1037/0022-006X.64.3.532
Lambert, M. J., & Ogles, B. M. (2004). The efficacy and
effectiveness of psychotherapy. In M. J. Lambert (Ed.),
Bergin and Garfield’s handbook of psychotherapy and behavior
change (pp. 139193). New York, NY: John Wiley & Sons.
 66 I. Elkin et al.
Luborsky, L., Crits-Christoph, P., Mintz, J., & Auerbach, A.
(1988). Who will benefit from psychotherapy? Predicting treatment
outcomes. New York, NY: Basic Books.
Ogrodniczuk, J. S., Joyce, A. S., & Piper, W. E. (2005). Strategies
for reducing patient-initiated premature termination of psy-
chotherapy. Harvard Review of Psychiatry, 13, 5770. doi:10.
1080/10673220590956429
Orlinsky, D. E., Ronnestad, M. H., & Willutzki, U. (2004). Fifty
years of psychotherapy process-outcome research: Continuity
and change. In M. J. Lambert (Ed.), Bergin and Garfield’s
handbook of psychotherapy and behavior change (pp. 307389).
New York, NY: John Wiley & Sons.
Raudenbush, S. W., and Bryk, A. S. (2002). Hierarchical Linear
Models (2nd ed.). Thousand Oaks, CA: Sage.
Reis, B. F., & Brown, L. G. (1999). Reducing therapeutic
dropouts: Maximizing perspective convergences in the psy-
chotherapy dyad. Psychotherapy, 36, 123136. doi:10.1037/
h0087822
Rogers, C. R. (1957). The necessary and sufficient conditions of
therapeutic personality change. Journal of Consulting Psychology,
21, 95103. doi:10.1037/h0045357
Sexton, H. C., Hembre, K., & Kvarme, G. (1996). The
Downloaded by [University Of Maryland] at 03:39 15 October 2014
interaction of the alliance and therapy microprocess: A
sequential analysis. Journal of Consulting and Clinical Psychology,
64, 471480. doi:10.1037/0022-006X.64.3.471
Sexton, H., Littauer, H., Sexton, A., & Tømmerås, E. (2005).
Building an alliance: Early therapy process and the client-
therapist connection. Psychotherapy Research, 15, 103116.
doi:10.1080/10503300512331327083
Spitzer, R. L., Endicott, J., & Robins, E. (1978). Research
diagnostic criteria: Rationale and reliability. Archives of General
Psychiatry, 35, 773782. doi:10.1001/archpsyc.1978.01770300
115013
Stiles, W. B. (1988). Psychotherapy process-outcome correlations
may be misleading. Psychotherapy, 25, 2735. doi:10.1037/
h0085320
Stiles, W. B., Honos-Webb, L., & Surko, M. (1998). Responsive-
ness in psychotherapy. Clinical Psychology: Science and Practice,
5, 439458. doi:10.1111/j.1468-2850.1998.tb00166.x
Stiles, W. B., & Shapiro, D. A. (1994). Disabuse of the drug
metaphor: Psychotherapy process-outcome correlations.
Journal of Consulting and Clinical Psychology, 62, 942948.
doi:10.1037/0022-006X.62.5.942
Suh, C. S., Strupp, H. H., & O’Malley, S. S. (1986). The
Vanderbilt process measures: The Psychotherapy Process Scale
(VPPS) and the Negative Indicators Scale (VNIS). In L. S.
Greenberg & W. M. Pinsof (Eds.), The psychotherapeutic process:
A research handbook (pp. 285323). New York, NY: Guilford
Press.
Sullivan, H. S. (1954). The psychiatric interview. New York, NY:
W.W. Norton.
Tryon, G. S. (2002). Engagement in counseling. In G. S. Tryon
(Ed.), Counseling based on process research: Applying what we
know (pp. 125). Boston, MA: Allyn & Bacon.
Tryon, G. S., & Kane, A. S. (1993). Relationship of working
alliance to mutual and unilateral termination. Journal of
Counseling Psychology, 40, 3336. doi:10.1037/0022-0167.40.
1.33
Wachtel, P. L. (1999). Resistance as a problem for practice and
theory. Journal of Psychotherapy Integration, 9, 103117.
doi:10.1023/A:1023262928748
Watson, J. C. (2002). Revisioning empathy. In D. J. Cain & J.
Seeman (Eds.), Humanistic psychotherapies: Handbook of research
and practice (pp. 445472). Washington DC: American Psy-
chological Association.
Watson, J. C., & Prosser, M. (2002). Development of an observer-
rated measure of therapist empathy. In J. C. Watson, R. N.
Goldman & M. S. Warner (Eds.), Client-centered and experiential
psychotherapy in the 21st century: Advances in theory, research and
practice (pp. 303314). Ross-on-Wye: PCCS Books.
Westra, H. (2004). Managing resistance in cognitive behavioural
therapy: The application of Motivational Interviewing in mixed
anxiety and depression. Cognitive Behaviour Therapy, 33, 161
175. doi:10.1080/16506070410026426
Wills, T. A. (1982). Directions for research on helping relation-
ships. In T. A. Wills (Ed.), Basic processes in helping relationships
(pp. 479496). New York, NY: Academic Press.