Psychotherapy Research

ISSN: 1050-3307 (Print) 1468-4381 (Online) Journal homepage: http://www.tandfonline.com/loi/tpsr20

Therapist competence in case conceptualization

and outcome in CBT for depression

Michael H. Easden & Richard B. Fletcher

To cite this article: Michael H. Easden & Richard B. Fletcher (2018): Therapist competence
in case conceptualization and outcome in CBT for depression, Psychotherapy Research, DOI:
10.1080/10503307.2018.1540895

To link to this article: https://doi.org/10.1080/10503307.2018.1540895

Published online: 03 Nov 2018.

Submit your article to this journal

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=tpsr20
Psychotherapy Research, 2018
https://doi.org/10.1080/10503307.2018.1540895

EMPIRICAL PAPER

Therapist competence in case conceptualization and outcome in CBT

for depression

MICHAEL H. EASDEN & RICHARD B. FLETCHER

School of Psychology, Massey University, Albany, New Zealand

(Received 4 December 2017; revised 10 October 2018; accepted 12 October 2018)

Abstract
Objective: This study aimed to investigate the relationship between therapist competence in case conceptualization and
outcome in psychotherapy. Method: Twenty-eight adults received Cognitive Behavioral Therapy (CBT) for depression.
The Conceptualization Rating Scale (CRS) was used to systematically evaluate therapist competence in case
conceptualization using ratings from 225 DVD recordings of live therapy in real time across the first ten sessions of
treatment. Results: Multilevel modelling (MLM) analysis revealed that after controlling for time and other pertinent
therapy influencing variables, therapist competence in case conceptualization explained 40% of within patient variance
and 19% of between patient variance associated with significant and positive change on the BDI-II. Conclusions: This
study provided evidence that increased therapist competence in using case conceptualization in CBT is associated with
greater reductions in depressive symptomology. The implications for supervision, training, and practice are discussed.

Keywords: CBT; case conceptualization; formulation; therapist competence; depression

Clinical or Methodological Significance Summary: Case conceptualization is considered to be an essential component

of psychotherapy training and practice. However, there is a lack of empirical evidence in support of this assertion, particularly
in CBT. This research focuses on therapist competence in case conceptualization in the process of psychotherapy as a means
to investigate therapy outcome-relations and towards establishing empirically based guidelines.

Cognitive Behavioral Therapy (CBT) as espoused

by Aaron T. Beck in the treatment of depression
Case conceptualization is fundamental to the theory
(Beck et al., 1979) is an empirically supported treat-
and practice of psychotherapy (Eells, 2007).
ment for depression as well as a range of other
Researchers and clinicians have referred to case con-
psychological disorders. The central importance of
ceptualization as the “first principle,” an “essential
CBT-specific case conceptualization is widely
component,” the “cornerstone,” or the “heart” of
accepted in practice, although there is a striking gap
psychotherapy (Beck, Rush, Shaw, & Emery, 1979;
in the research to support the validity of CBT case
Bieling & Kuyken, 2003; Chadwick, Williams, &
conceptualization in relation to treatment outcomes
Mackenzie, 2003; Persons, 2001). Case conceptual-
and to better inform evidence-based practice
ization (synonymous with case formulation) is a
guidelines.
“process whereby therapist and patient work colla-
Cognitive–behavioural theory exists as a frame-
boratively first to describe and then to explain the
work for understanding the patient’s presenting pro-
issue a patient presents in therapy. Its primary func-
blems and informs intervention (Beck, 1995;
tion is to guide therapy in order to relieve patient dis-
Needleman, 1999; Persons, 1989, 2008). Interven-
tress and build patient resistance” (Kuyken, Padesky,
tion, in turn, re-informs the evolving case conceptu-
& Dudley, 2009, p. 3).
alization process over the course of therapy, with

Correspondence concerning this article should be addressed to Michael H. Easden, School of Psychology, Massey University, Albany, New
Zealand. Email: michaeleasden@gmail.com

© 2018 Society for Psychotherapy Research

2 M. H. Easden and R. B. Fletcher
the intention of patient mastery and generalisation of
cognitive–behavioural skills (Dozois, Covin, &
Brinker, 2003). One such intervention is “home-
work” which exists as an exception to the dearth of
research into central CBT processes. Homework is
broadly defined as collaborative and planned
between-session activities tailored to meet a patient’s
therapeutic goals; these tasks are based on an indivi-
dualized case conceptualization consistent with the
empirically supported CBT model (Kazantzis,
Deane, Ronan, & L’Abate, 2005). As such home-
work is intrinsically linked with case conceptualiz-
ation as during therapy sessions the therapist guides
the collaborative selection of in-session and
between-session homework tasks or experiments
based on individualized hypotheses for each patient
for what triggers, maintains, and interrupts their
pattern of difficulties (Kuyken, Padesky, & Dudley,
2008). The use of homework in psychotherapy has
increasingly been related to treatment benefits (for
reviews see Kazantzis, Deane, & Ronan, 2000;
Kazantzis, Whittington, & Dattilio, 2010; Kazantzis
et al., 2016; Mausbach, Moore, Roesch, Cardenas,
& Patterson, 2010).
A Theoretical Model of Case
Conceptualization in CBT
Case conceptualization can be undertaken and applied
at many different levels and in many different forms
(see Easden & Kazantzis, 2018). For instance, case
conceptualization might occur at a descriptive level
or be more in-depth and explanatory; it might take
place at the level of a problem, strength or diagnosis.
Case conceptualization may be cross-sectional (i.e.,
in the moment or “mini”-conceptualization) of a par-
ticular situation, or a longitudinal integration of infor-
mation. Given the diversity of applications, a number
of commonly used theoretical—often pictorial and
diagrammatic—frameworks exist emanating from a
CBT paradigm (e.g., A. Beck’s depression triad of
self, world/future and others, 1976; J. Beck’s case con-
ceptualization diagram, 1995; Clark’s cognitive model
of panic, 1986; Padesky and Mooney’s 5-part model,
1990; Person’s problem list, 2001). These provide a
bridge between the theoretical and practical appli-
cation of case conceptualization as an inherent
process within CBT.
The “Case Conceptualization Crucible”
(Kuyken et al., 2008, 2009) was proposed as an
overarching/guiding model of case conceptualiz-
ation in CBT. Three principles guide the approach
(1) collaborative empiricism, (2) levels of concep-
tualization, and (3) incorporation of patient
strengths. The model asserts that at each
developing level of case conceptualization (i.e.,
descriptive, cross-sectional, longitudinal) colla-
borative empiricism is the catalyst or “heat” that
facilitates understanding and change. This colla-
borative empiricism is the process whereby thera-
pist and patient work together to integrate new
information over time, aligning the lived experi-
ence of the patient with CBT-specific theoretical
knowledge and expertise of the therapist. Essen-
tially, the patient and therapist develop hypotheses
together, transparently, and within a CBT frame,
towards a shared understanding of presenting
issues, triggers (precipitating factors) and mainten-
ance (perpetuating factors), and protective and
predisposing factors (i.e., in fitting with and build-
ing upon what has previously been described as the
generic 5’Ps; Weerasekera, 1996).
The Empirical Basis of Case
Conceptualization in CBT
CBT is generally considered to be an empirically
based treatment. The question has been asked in
relation to case conceptualization, “is the emperor
clothed?” to which the conclusion has been “very
sparsely” (Kuyken, 2006). Positive implications of a
well-constructed and integrated case conceptualiz-
ation are that they facilitate the linking of theory,
research and practice, normalise problems and
increase empathy, organise large amounts of
complex information, and enable high quality super-
vision among other benefits. Case conceptualization
is also suggested to guide treatment intervention by
selecting, focusing and sequencing the order of treat-
ment intervention, facilitating a collaborative
approach, pre-empting possible therapy interfering
behaviours, and enabling the simplest most cost
effective interventions.
While other psychotherapeutic orientations have
made headway into empirical demonstration that
case conceptualization enhances therapeutic out-
comes within their respective paradigm (for an over-
view of structured case conceptualization methods
from different therapeutic orientations see Eells,
2007), the importance of case conceptualization in
CBT remains less clear. The vast majority of case
conceptualization research in CBT has focused on
written case conceptualizations based on discrete
portions of therapy. The broad implication is that
there is no empirical basis for the clinical utility of
case conceptualizations as they apply to a naturalistic
course of CBT; particularly components pertinent to
cognitive specificity which sets CBT apart from
other therapeutic orientations (Easden & Kazantzis,
2018).
 Psychotherapy Research 3

Re-evaluating the Measurement of Case provides a method of measuring the relationship

Conceptualization: Therapist Competence between therapist use of case conceptualization and
outcomes in CBT. However, surprisingly little
A significant gap in the research concerns how cogni-
research exists bearing on the validity of the case con-
tive therapists conduct case conceptualizations in
ceptualization process in CBT (Bieling & Kuyken,
real-world practice (Kuyken et al., 2009). Prior
2003; Easden & Kazantzis, 2018).
research and measurement tools have tended to
There are a number of measures designed to
focus on the quality of written case conceptualizations
measure therapist competence across broad domains
(Bucci, French, & Berry, 2016). Specifically, the
of therapist skill in CBT (i.e., global competence).
organisation and integration of the information
For example, the Cognitive Therapy Scale (CTS;
from a patient’s presentation and the quality of resul-
Young & Beck, 1980) and the revision CTS-R (Black-
tant written case conceptualizations have traditionally
burn et al., 2001) are commonly used in supervision
been considered to impact how therapists approach a
and training to assess therapist competence in cogni-
patient’s case and implement a treatment plan (Crits-
tive–behavioural oriented therapies; the measure is
Christoph, Cooper, & Luborsky, 1988; Kuyken,
rated based on live or recorded therapy sessions by
Fothergill, Musa, & Chadwick, 2005). However, it
an independent observer or supervisor. Broad judge-
has been noted that written records may not encapsu-
ments are made about the presence and quality of
late a therapist’s full understanding of their patient’s
common features of therapy (e.g., agenda setting,
case conceptualization (Eells, Kendjelic, & Lucas,
case conceptualization, and homework use).
1998). Conceivably, therapists could hold alternative
However, limited research suggests that clinicians are
or divergent hypotheses from what is written in early
unable to reliably agree on the extent to which thera-
case conceptualizations as their therapy sessions pro-
pists demonstrate competence in case conceptualiz-
gress with patients.
ation. In examining the psychometric properties of
However, therapist competence (i.e., the level of skill
the CTS-R Blackburn et al. (2001) found that inter-
in using interventions) has been related to treatment
rater reliability for individual items produced the
benefits in a range of evidence-based psychotherapies
lowest score for the “conceptualization” item (r =
(for a review see Webb, DeRubeis, & Barber, 2010)
0.42, df = 49, p < .01) while the level of competence
and in CBT-specific research (for a review see Zarafo-
in “conceptualization” was one of the few items that
nitis-Müller, Kuhr, & Bechdolf, 2014; see also David-
appeared to improve over the course of training. As a
son et al., 2004; Kuyken & Tsivrikos, 2009; Shaw
global measure of competence, the CTS does not sys-
et al., 1999; Strunk, Adler, & Hollars, 2013; Trepka,
tematically assess different components of therapist
Rees, Shapiro, Hardy, & Barkham, 2004). It is noted
competence in case conceptualization and instead col-
in this research that greater effects were observed for
lapses this judgement into a single broad rating; as
studies targeting depression compared with other
such a high level of inference is required on the part
psychological disorders.
of the rater making the judgements. Kuyken et al.
Barber, Sharpless, Klostermann, and McCarthy
(2016) in a sample of depressed patients (i.e., 2–3
(2007) differentiated and defined two levels of
clients from each of the nine therapists) provided pre-
enquiry into therapist competency, namely; global
liminary data showing a moderate correlation (r = .54,
competence (i.e., as a broad focus on therapist skill
p < .01) between scores on the Collaborative Case
across a range of clinically relevant domains) and
Conceptualization Rating Scale (CCC-RS) and the
limited-domain competence (i.e., a subset of global
CTS-R. Such research acknowledges the importance
competence assessing particular practices for particu-
of therapist competence in case conceptualization in
lar interventions and often within the context of a
relation to broader therapist competencies by addres-
specific psychotherapeutic orientation). Consistent
sing the dearth empirical support for case conceptual-
with previous studies overall therapist competence
ization in CBT. However, present research is yet to
possessed a positive association with therapy out-
provide empirical evidence in regards to therapist
comes. However, the authors recommended further
competence in case conceptualization and therapy
measures of limited-domain competence be devel-
outcome relations (i.e., symptom improvement /
oped to further understand the weaker than expected
recovery).
relationship between therapist competence and treat-
ment outcomes. Consistent with previous research
recommendations and in order to directly address
therapist moment-to-moment use of case conceptu- The Current Study
alization (as opposed to examination of static
written formulations) an operationalized definition The overarching aim of current study was to investi-
of therapist competence in case conceptualization gate the relationship between case conceptualization
4 M. H. Easden and R. B. Fletcher
and therapy outcome. In order to achieve this aim the
Conceptualization Rating Scale (CRS) was devel-
oped and utilized as a new measure of limited-
domain competence and we present preliminary
data on the psychometric properties. The CRS was
designed to be an operationalization of in-session
(i.e., observer rated based on live or video recorded
therapy sessions) case conceptualization competen-
cies in CBT (Beck et al., 1979). The purpose of the
CRS was both as, (1) a supervision tool to provide
feedback to therapists regarding their competence
in case conceptualization in CBT, and as (2) a
research tool hoped to provide data that more
closely captures real-world practice towards provid-
ing empirical support for processes and outcomes
related to case conceptualization in CBT.
Although research into limited-domain compe-
tence in CBT is sparse, there is a relatively broad
base of research into homework and the importance
of between-session tasks. Headway into the measure-
ment of therapist competence in homework use using
the Homework Adherence and Competence Scale
(HAACS) provided an opportunity to investigate
the relationship between two areas of limited-
domain competence considered to be important for
positive therapy outcomes.
It follows that our primary hypothesis was that: (1)
greater therapist competence in case conceptualization
would predict improvement in depressive symptoms as
measured on the BDI-II (hypothesis 1), and (2) the
impact of therapist competence in homework use
(which is theorised to be heavily informed be individua-
lised case conceptualization) would also be associated
with CBT outcomes (hypothesis 2). Finally, we con-
sidered that (3) pertinent patient attributes (i.e.,
beliefs about homework, symptom severity and com-
plexity) would moderate the relationship between
therapist competence and outcome (hypothesis 3).
Method
Participants
Patients. For inclusion in the study patients were
required to be experiencing a major depressive
episode (DSM-IV-TR) for the first time. Patients
were required to be between 18 and 65 years old, pro-
ficient in reading, writing, and conversing in English.
Patients were not eligible if they were taking central
nervous system (CNS) acting drugs, or if they met
criteria for substance abuse, psychosis, or borderline
personality disorder. In total 28 patients met the
DSM-IV-TR (APA, 2000) criteria for Major
Depressive Disorder (MDD) and received treatment.
The sample consisted of 10 male (36%) and 18
female (64%) patients of average age 44.75 years
(SD = 11.51; range from 20 to 62 years). The partici-
pants identified their ethnicity as predominantly
European / Caucasian (n = 24; 86%).
Therapists. Seven female therapists of average age
36.86 years (SD = 11.60; range from 23 to 50 years)
provided the course of CBT; all were intern psychol-
ogists in their second to last or final year of training
towards a doctoral qualification fulfilling the require-
ments for registration as a clinical psychologist. Each
therapist had undertaken two advanced courses in the
theory and practice of CBT for depression. Each
course was a 5-day intensive postgraduate applied
training course involving role-plays, supervision in
techniques, and other active learning activities.
Raters. Raters were five females and one male of
average age 31 years old (SD = 12.80; range from
22 to 50 years). The raters were all postgraduate psy-
chology students with a minimum of one to two years
postgraduate experience. Raters were trained at two
sites at Massey University in Auckland, New
Zealand and La Trobe University in Melbourne,
Australia. Raters at both sites received up to 13 h of
training broken up into three days. Training was
delivered by the primary author and a senior clinical
psychologist with expert knowledge in the field.
Raters were given an overview of case conceptualiz-
ation in CBT and orientation to the CRS use and
development. Short clips of therapy sessions were
then used to represent more competent and less com-
petent use of different components of CBT case con-
ceptualization (e.g., therapist and patient linking
thoughts, emotion and behaviour in a collaborative
or non-collaborative manner; therapist incorporating
patient strengths). Lastly raters viewed three full
DVD recorded therapy sessions independently then
discussed and calibrated scores against each other
and benchmark criterion scores assigned by the
trainers.
Measures
Conceptualization Rating Scale (CRS). The
CRS is an observer rated measure of therapist skills
in eliciting information used to develop comprehen-
sive case conceptualizations based on live therapy
recordings. There are four domains for each item
that include the (1) integration, (2) importance, (3)
therapist competence, and (4) fit / match of the in-
session use of case conceptualization with the
written case conceptualization (i.e., to assist in deter-
mining whether content relevant to important aspects
of case conceptualization discussed during a therapy

session are then captured in written case conceptual-
izations produced by therapists). Each of the four
domains are applied to 12 items considered most rel-
evant to discussion and development of case concep-
tualizations, namely, (1) identification of a situation,
(2) automatic thoughts, (3) meaning of automatic
thoughts, (4) emotion, (5) behaviour, (6) relevant
background data, (7) core beliefs, (8) conditional
assumptions, (9) compensatory strategies, (10) a dis-
order-specific model, (11) client resilience/strengths,
and (12) sharing the overall case conceptualization.
For the purposes of this study therapist compe-
tence across these items was the focus as the
primary predictor variable. The integration and
importance domains are simple dichotomous scales
included primarily to cue the rater to the presence
of relevant case conceptualization components
during the course of a session, in order to then gener-
ate ratings of therapist competence in the use of these
components. Therapist competence in case concep-
tualization was measured on a 7-point sub-scale (0
= less competent to 6 = more competent) with descriptive
anchors. While the stand alone tick-box for inte-
gration and importance draws the rater’s attention
the presence or absence of particular conceptualiz-
ation components (and might be used to readily
provide feedback to therapists neglecting to discuss,
for example emotion or client strengths, or to focus
on less important or relevant components of the con-
ceptualization) the therapist competence domain
incorporates whether or not a component of case
conceptualization was judged to be integrated skil-
fully (or competently omitted) and thus considered
important / relevant. For instance, to obtain a more
competent score therapists were required to identify
and integrate one or more important components of
the case conceptualization into the session (e.g.,
select a specific and relevant situation, explicitly ask
about thoughts, meanings and/or conditional
assumptions attributed, emotion and behaviour).
This was required to be done with appropriate
depth of discussion also fostering links between
different components, and eliciting patient feedback
intermittently (i.e., about relevance, acceptance and
their understanding of session content).
Conceptual and theoretical basis. It was
intended that the different items and domains on
the CRS would reflect cognitive behavioural theory
(Beck, 1976) and be acceptable within the framework
of competent day-to-day practice of CBT. In turn,
the J. Beck Case Conceptualization Diagram
(CCD; Beck, 1995, 2011) was used to provide a fra-
mework for items on the CRS. This decision was pri-
marily made due to the cognitive behavioural
Psychotherapy Research 5
theoretical foundations on which the CCD was
created. Although other written formats for case con-
ceptualizations in CBT exist, the continued popular-
ity and common usage of the CCD in clinical practice
after over 20 years of existence provided a sound
rationale for investigating assumptions regarding the
empirical utility of the CCD. As such each item on
the CRS pertained to components of the CCD
(e.g., relevant situation, automatic thoughts,
emotion, behaviour, core beliefs, etc). In additional
to items mirroring the different components of the
CCD the items “disorder specific model,” “resili-
ence/strength,” and the “sharing of the overall con-
ceptualization” were included in the CRS.
In constructing item scales, the theoretical prop-
osition and “metacompetencies” identified by
Kuyken et al. (2008, 2009) were adopted which sup-
ported a focus on three interrelated areas: (1) colla-
borative empiricism (i.e., working together to
develop mutual understanding, consistently sharing
aspects of the case conceptualization and seeking
patient feedback on the case conceptualization as it
is develops and evolves in session, etc), (2) depth of
discussion (i.e., level of cognitive / behavioural
exploration, hypothesising and testing links between
different situations and components of the case con-
ceptualization), and (3) patient strength/resilience
(i.e., explicit focus on existing coping or adaptive
qualities consistent with holistic and comprehensive
case conceptualization in modern CBT practice).
Expert feedback and face validity. Feedback
on draft versions of the CRS measure was sought
by clinicians and researchers considered to be
experts, including a panel of clinicians (n = 8) with
peer-reviewed publications in the field of psychother-
apy case conceptualization. Some notable and con-
sistent feedback that resulted in changes to early
drafts included ensuring the CRS had the flexibility
to assess therapists as competent when particular
areas of the conceptualization were not integrated
or deeply explored (e.g., the therapist appropriately
judges that it is inadvisable to activate a core belief
as in the case of a fragile patient, weak therapeutic
alliance, early in treatment, or when activation of a
core belief might significantly increase depression or
anger). A second round of feedback from experts
reflected good content validity with a general consen-
sus that the CRS was a comprehensive case concep-
tualization measure appropriate for use in CBT.
Beck Depression Inventory-II (BDI-II; Beck
et al., 1979; Beck, Steer, & Brown, 1996). The
BDI-II is a 21-item patient self-report measure
widely used to assess depression severity. Each item
6 M. H. Easden and R. B. Fletcher
is rated on a “0” to “3” point scale with possible
scores ranging from 0 to 63. The BDI-II has been
extensively researched and is a commonly used
measure in the treatment of depression (Dozois,
Dobson, & Ahnberg, 1998; Schotte, Maes, Cluydts,
De Doncker, & Cosyns, 1997; Steer, Ball, Ranieri,
& Beck, 1999). It has demonstrated excellent internal
consistency (α = .92) among depressed outpatients
(Beck et al., 1996) and good validity across a range
of domains (i.e., content, concurrent, factorial com-
position). In the current study internal consistency
for the BDI-II was excellent with values all equal to
or greater than α = .90.
Homework Adherence and Competence Scale
(HAACS; Kazantzis, Wedge, & Dobson, 2005).
The HAACS is an observer-rated 19-item measure
rated on a 7-point Likert scale with descriptive
anchors, designed to measure adherence and compe-
tence in the review, design and assigning of homework
tasks (in the current study scored by raters based on
DVD recordings of therapy sessions). Total adherence
percentage agreement has been demonstrated to range
from 74% to 85% and competence scales have pro-
duced ICCs ranging from .79 to .83 (Wedge, 2005).
Internal consistency has been demonstrated to be
excellent (α = .80) for the competence domain
(Kazantzis et al., 2006). In a further study (Munro,
2006) Cronbach’s alpha (α) were .77 for adherence
and .81 for competence. Internal consistency scores
for each section review (α = .70), assign (α = .82),
and design (α = .80) were produced in a similar
range. Concurrent validity was also demonstrated
with 20% of shared variance (r = .45) with a thera-
pist-completed measure of homework adherence and
competence. Internal consistency for the therapist
competence scale in the current study ranged from
α = .76 to α = .89. Intraclass correlation coefficients
reflected high rates of agreement (ICC = .91).
Homework Rating Scale (HRS-II; Kazantzis
et al., 2005). The HRS-II client-version is a 12-
item questionnaire rated on a 5-point Likert scale
(0 = “not at all” to 4 = “extremely”) used to rate
patient beliefs about between-session activities (i.e.,
homework). Chronbach alpha (α) coefficients have
been reported to range from .71 to .91 across sessions
(Sachsenweger, Fletcher, & Clarke, 2015) for the
client version. A three factor model was produced
(Bjornholdt, 2006) consisting of items pertaining to
benefits and completion (Factor 1), costs and com-
pletion (Factor 2), and client beliefs (Factor 3). The
HRS-II has demonstrated concurrent validity with
37% of shared variance (r = .61) with a patient-
completed measure of homework compliance
(Munro, 2006).
Personality Beliefs Questionnaire (PBQ;
Beck et al., 2001; Beck & Beck, 1991; Butler,
Brown, Beck, & Grisham, 2002). The PBQ is a
126-item self-report measure rated on a 5-point
Likert scale from “0” to “4” (0 = “I don’t believe it
at all” to 4 = “I believe it totally”). Patient’s endorse-
ments of dysfunctional personality belief statements
generate nine categories including Avoidant, Depen-
dent, Obsessive Compulsive, Histrionic, Passive-
Aggressive, Narcissistic, Paranoid, Schizoid, and
Antisocial. The PBQ categories provide an indication
of the presence of personality disorders where a link
between these categories and associated personality
disorder diagnoses has been demonstrated (Beck
et al., 2001; Butler et al., 2002). Cronbach’s alpha
(α) coefficients for individual scales on the PBQ
ranged from .79 to .94 across two studies (Beck
et al., 2001; Butler et al., 2002). Beck et al. (2001)
reported the median inter-correlations between
scales were moderate (r = .51) and Pearson test-
retest correlations for the individual PBQ categories
were adequate ranging from r = .57 to r = .93.
Composite Diagnostic Interview (CIDI)
version 2.1 (WHO, 1997). The CIDI is a structured
computerised assessment tool designed to assist clin-
icians in providing diagnoses for a range of mental
disorders consistent with the DSM-IV. Of particular
relevance to the current study the CIDI was used to
generate a potential and preliminary diagnosis and
severity of first episode Major Depressive Disorder
(MDD). The CIDI Version 2.1 has been demon-
strated to have adequate validity (Andrews &
Peters, 1998). Kessler et al. (2003) in a large
sample (N = 9090) door to door survey found a con-
cordance (Cohen’s k = .40) between the CIDI and
the structured clinical interview for DSM-IV
(SCID) for those diagnosed with experiencing a
major depressive episode over the last 12 months.
Procedure
Screening. Clients either self-referred or were
referred by a third party (e.g., clinical agency,
family, GP). Of the 251 individuals who made
initial phone contact to the study, 186 were not
found to meet inclusion criteria after an initial 30-
minute structured telephone interview by study
coordinators. The 65 remaining met preliminary
criteria for MDD and attended appointments for
further screening on the CIDI and an initial inter-
view with a therapist. Of these 37 did not meet

inclusion criteria. Each of the 28 patients accepted
were allocated to a new therapist at random and
offered up to 20 CBT sessions (average of 17.79
sessions) with an additional two booster sessions
offered at two months and six months following
therapy.
Rating procedure for process measures. All
therapy sessions were video recorded on DVD. For
the 28 patients receiving therapy, DVD recordings
of 225 individual CBT sessions were observed in
their entirety and rated using the CRS and the
HAACS. The HAACS adherence and competence
scale ratings were based solely on observations of
the DVD recorded sessions. The CRS competence
scale was the primary predictor variable of interest
in the current study. The CRS integration, impor-
tance, and competence scales were also based on
DVD recorded sessions. Of note, the CRS fit /
match scale ratings were made by raters comparing
the case conceptualization content observed to be
discussed in-session with available written case con-
ceptualization data; this was done to examine if thera-
pists were able to capture the content of in-session
case conceptualizations in their static / written case
conceptualizations subsequently generated. To facili-
tate this aspect of the rating procedure therapists
completed J. Beck CCDs (Beck, 1995) systematically
at five sampling points (Sessions 1, 3, 5, 8, and 10).
Information included in the J. Beck CCDs was gener-
ated collaboratively with the patient during each
session and the J. Beck CCDs were formally written
up by therapists between sessions. Inter-rater agree-
ment was based on ratings by pairs of independent
raters. DVD-recorded sessions were randomly allo-
cated to pairs of raters; randomisation and allocation
began before all therapy sessions were complete,
some DVD-recordings were rated when available.
All ratings were double coded and completed for
over 80% of the total available sessions. Ratings
were made for session one through to session ten as
the focus of this study.
Client-rated outcomes. The BDI-II was the
primary dependent (outcome) variable and was com-
pleted at every session. The HRS-II (client-rated
version) was used to assess client beliefs about home-
work and was also completed after every session. The
CIDI was used as a measure of symptom severity and
completed at intake. The PBQ was used as a measure
of patient complexity and was completed once at
session eight, where changes in personality beliefs
have been suggested to be more measurable from
session eight onwards (Beck & Freeman, 1990),
and was intended to further inform development
Psychotherapy Research 7
and sharing of the case conceptualization. The
HRS-II, CIDI and PBQ were used primarily as con-
trols to reduce confounds associated with patient who
might hold strongly negative views about homework
(potentially influencing the impact of therapist com-
petence in homework use) and where the symptom
severity and complexity of a patient might account
for change that would otherwise be attributed to
therapist effects. Thus where possible, the study
adopted a process-oriented design, rather than pre–
post, to facilitate more in-depth analysis.
Treatment fidelity. Regular fidelity checks and
weekly supervision sessions with therapists were
carried out over the course of therapy by a senior
registered clinical CBT psychologist. Ratings were
made based on DVD recorded therapy sessions for
each therapist using the Cognitive Therapy Scale
(CTS; Young & Beck, 1980) to provide an indication
of the level of therapist global competence in the car-
rying out the treatment protocol as intended. Thera-
pists were required to achieve a score of at least 40 in
three CTS assessments or were required to demon-
strate increased competence in further CTS assess-
ments until this level of competence was attained.
CTS scoring was carried out and reviewed by the
therapist’s clinical supervisor, an experienced regis-
tered clinical psychologist; her role was independent
to other aspects of the current study.
Data Analytic Approach and Procedure
All analyses were undertaken using SPSS version
17.0; SPSS MIXED procedure was used for multile-
vel modelling analysis (MLM; Raudenbush & Bryk,
2002; Singer & Willett, 2003). We used the Full
Maximum Likelihood (FML) which provides a
description of fit of the model for both fixed and
random parameters and allows for comparison of
goodness of fit across models. Intercepts and slopes
were modelled as random effects in the analyses as
we would expect variation in scores. We used an
unstructured covariance matrix which allowed us to
estimate a correlation between intercepts and
slopes. Data were centred consistent with recommen-
dations for MLM analyses (Peugh & Enders, 2005;
Singer & Willett, 2003). Akaike Information Cri-
terion (AIC) was used as an indication of model fit
given the number of parameters included. Pseudo
R 2 was used as a measure of effect size to determine
within patient and between patient variance
explained consistent with recommendations by
Singer and Willett (2003).
The data were structured as person-period data that
examines both between person (i.e., comparison of
8 M. H. Easden and R. B. Fletcher
average change trajectories comparing one patient to
another) and within person variation (i.e., over time
across individual patients). In the current study each
successive measurement of depression severity
(Level 1) was nested within each patient (Level 2).
Rather than treating patients as nested within thera-
pists (i.e., clustered data structure, often but not
always measured at one time point), in order to
examine change across sessions we examined how
change in depressive symptoms differ as a function
of pertinent patient characteristics. Initially at the
within person level (Level 1) we introduce therapist
competence (in both case conceptualization and
homework use) as predictor variables which might
be associated with observed depressive symptom
change. We then introduce pertinent patient charac-
teristics that might moderate the strength of this
association at the between person level (Level 2).
At present when using MLM analysis there is no
consensus in regards to appropriate sample size
required to produce stable and precise estimates.
Rules of thumb are inconsistent across studies
although generally recommend a minimum of 10,
20 or 30 observations/units at Level 2 while larger
samples are always advisable (McNeish & Stapleton,
2016; Snijders & Bosker, 2012). These studies do not
take into account the number of observations per
variable (i.e., at successive time-points) which can
be used to guide sample size decisions in clustered
data. Kwok et al. (2008) noted that a large number
of waves/repeated measures data units at Level 1
were often not applicable in psychological research,
although reliable estimates for the individual growth
models can be obtained with 8 or more measurement
waves. Wynants et al. (2015) recommended at least
10 observation points for predefined models. In the
current study the use of 10 observation points (i.e.,
10 sessions yielding n = 225 data units) and 28
patients (despite only 7 therapists providing
therapy) would suggest that this present analysis
should lead to unbiased estimates able to produce
meaningful results.
For coding data we determined which variables will
be modelled as “random” effects or as “fixed” effects.
Essentially “random” effects were obtained when the
coefficients for variables were allowed to vary over the
course of therapy (i.e., time-variant, continuous data)
sessions within patients and “fixed” effects were
obtained when coefficients for variables were con-
stant (i.e., time-invariant, categorical data) for indi-
vidual patients (Raudenbush & Bryk, 2002; Snijders
& Bosker, 2012). Therapist competence in both
case conceptualization and homework in its raw
form was continuous data that are variable over the
course of therapy for individual patients and was
therefore modelled as a random effect at Level 1.
For observational interrater data (i.e., the CRS and
HAACS) two raters generated independent scores for
each observation. The average score for each pair of
ratings was selected to represent the patients’ score.
This was done in consideration of high interrater
reliability scores obtained. All remaining data were
treated as continuous time series data within the
data set to allow for the possibility of differences in
the level of variables over time (i.e., Level 1). The
exception to this was the treatment of symptom sever-
ity (i.e., CIDI) and personality beliefs (i.e., PBQ)
which were treated as constant over time and
divided into categories of severity (i.e., Level 2).
Specifically, the CIDI generates four possibilities for
indications of symptom severity associated with
depression: “no diagnosis” and “mild” (coded as
“0”), “moderate” (coded as “1”) and “severe”
(coded as “2”). The PBQ generates values associated
with lesser or greater degrees of disordered personal-
ity beliefs and patient complexity. Total scale person-
ality beliefs scores were divided evenly between the
lower and higher 50th percentile (coded as “0” and
“1” respectively) splitting patients into one of two
categories (i.e., high or low personality beliefs).
This splitting of the total scale PBQ score also
coincided with recommendations for measuring
observable differences in personality beliefs in
patients with features of personality disordered
beliefs (Beck et al., 2001).
Results
Preliminary Analyses
Patients presented with a range of severity of depressive
symptoms and generally improved over the course of
the study. Mean baseline BDI-II score was 31.03
(SD = 10.87; range from 13 to 53). Mean post treat-
ment BDI-II score was 16.80 (SD = 11.12; range
from 2 to 45). The results of OLS regression estimates
(see Table I) showed that the average patient had an
initial change in depressive symptoms of 8.47 points
which increased by .16 points each session. On
average, for example, by session 10 BDI-II scores
Table I. Descriptive statistics for the individual growth parameters
obtained by fitting separate within-person OLS regression models
for BDI Change as a function of curvilinear time.
Initial status Rate of change
(intercept) (slope)
Mean 8.47∗∗∗ 0.16∗∗∗
Standard error 1.06 0.04
Bivariate 0.34∗
correlation
Note. ∗ p < .05, ∗∗ p < .01, ∗∗∗ p < .001.

improved by 10.07 (8.47 + (10 × .16)). These results
showed that improvement (i.e., change trajectory) in
depressive symptom change was maintained as
increases over the course of therapy for the average
patient. The correlation coefficient of r = .34 revealed
a positive relationship between time (session) and
depressive symptom change.
Initially, we were able to visually observe raw
levels of therapist competence in case conceptualiz-
ation for each session for individual patients (see
Figure 1). Table II provides descriptive character-
istics of domains on the CRS. Competence scores
on the CRS ranged from 0 to 55 (M = 25.14, SD
= 9.78). Not surprisingly bivariate correlation
revealed a negative association between fitted
initial status and fitted slope (rate of change) of
CRS competence (r = −.82, p < .01), where for
example those therapists with higher initial compe-
tence had a slower rate of improvement in compe-
tence scores, where those with lower competence
scores had greater room for more rapid improve-
ment over time.
Intercorrelations were calculated (see Table III) in
order to provide an indication of the direction (i.e.,
positive or negative association) of relationships
hypothesised between variables in the present study,
which were generally as expected.
Internal Consistency
Cronbach’s alpha (α) coefficients were computed to
determine the internal consistency of each domain
Figure 1. Therapist competence scores in case conceptualization for each patient on the CRS by session (over time).
Psychotherapy Research 9
Table II. Mean ratings on conceptualization rating scale (CRS)
Sub-scales.
CRS sub-scales Score
Integration (0–12) 2.39 + 2.36 (0 − 12)
Importance (0–12) 9.00 + 1.79 (1.5 − 13.5)
Competence (0–72) 25.14 + 9.78 (0 to 55)
Fit / Match (0–72) 29.58 + 11.36 (8.5 − 61.5)
Note. Values are given as means ± SD, with ranges in parentheses.
on the CRS to provide an indication of the full
scale reliability of the overall measure. Cronbach’s
alpha coefficients for integration, importance, com-
petence and fit / match were .72, .66, .79 and .87
respectively. High Cronbach’s alpha scores
suggested that total scale scores for each domain
were a good indicator of individual items in each
domain.
Interrater Reliability
To establish interrater reliability Cohen’s Kappa (k)
coefficients were used for dichotomous data
domains (i.e., Integration and Importance) and Intra-
class Correlation Coefficients (ICCs) were used for
continuous data domains (i.e., Competence and Fit/
Match). Total score reliability estimates (see
Table IV) for Integration (k = .83) and Importance (k
= .65) were excellent and adequate respectively.
Total score reliability estimates for Competence
(ICC = .93) and Fit/Match (ICC = .86) were both
10 M. H. Easden and R. B. Fletcher
Table III. Intercorrelations among predictor, control and dependent variables.

1 2 3 4 5 6 7 8 9 10 11

1. CRS integration –
2. CRS importance .769∗∗ –
3. CRS competence .807∗∗ .699∗∗ –
4. CRS fit / match .271∗∗ .272∗∗ .455∗∗ –
5. HAACS adherence .041 −.032 .106 .131 –
6. HAACS competence .084 −.002 .186∗∗ .193∗∗ .803∗∗ –
7. PBQ (complexity) −.061 −.064 −.025 .078 −.049 −.062 –
8. HRS-II (patient beliefs) −.043 .030 −.058 .000 −.057 −.031 −.255∗∗ –
9. CIDI (symptom severity) −.049 −.015 −.059 −.043 −.008 .041 .176∗∗ .162∗ –
10. BDI raw −.149∗ −.142∗ −.165∗ −.166∗ −.044 −.166 .335∗∗ .036 .240∗∗ –
11. BDI change .138∗ .173∗∗ .142∗ .067 .062 .075 −.061 −.080 .440∗∗ −.440∗∗ –

Note. ∗ p < .05, ∗∗ p < .01.

Table IV. Individual item and total scale reliability estimates for the CRS.

Integration Importance Competence Fit / Match

Situation 0.79 0.32 0.91 0.87

Automatic thoughts 0.82 0.45 0.89 0.91
Meaning of ATs 0.66 0.46 0.81 0.77
Emotion 0.80 0.17 0.88 0.69
Behaviour 0.73 0.29 0.82 0.76
Relevant childhood data 0.80 0.59 0.90 0.85
Core beliefs 0.71 0.60 0.91 0.85
Conditional assumptions 0.77 0.66 0.90 0.77
Compensatory strategies 0.73 0.42 0.76 0.69
Disorder-specific model 0.79 0.15 0.94 0.68
Resilience / strengths 0.66 0.12 0.88 0.89
Sharing of overall CC 0.87 0.40 0.99 0.96
Total estimate 0.83 0.65 0.93 (0.87) 0.86 (0.75)

Note. Integration and Importance are reported as Cohen’s Kappa, Competence and Fit / match reported as ICCs. Individual item estimates
are reported as average measure ICC (2, k), total ICCs are reported as average measure ICC (2, k) and single measure ICC (2, 1) estimates.

excellent. As with total scale scores, individual item
reliability estimates for Competence were the most
robust (ICC = .76–.99). Interrater reliability was
demonstrated to be fairly consistent over time (i.e.,
at each individual time-point for sessions one to
ten) for Integration (ICC range .78 to .85), Importance
(ICC range .60 to .71), Competence (ICC range .91 to
.96) and Fit/Match (ICC range .77 to .90). One
exception to this was Importance at session 10 (ICC
= .36) which was significantly lower than other indi-
vidual session estimates. This was likely as a result
of session 10 being a unique session in that the proto-
col intended for therapists to formally share verbal
and/or written case conceptualizations with the
patient, likely causing disagreement between raters.
Lastly, Competence at session 8 was adequate but
lower than other estimates (ICC = .64), also likely
due to a divergence in the protocol, whereby thera-
pists were asked to complete the PBQ with clients
systematically at this session which interfered with
therapists routine therapy agenda.
Multilevel Model Analysis
Prior to conducting the MLM analysis data were
screened for each variable and assumptions sur-
rounding normality and homoscedacity were met.
To ensure there was no biases to due to any systema-
tic pattern of missing data, Little’s Missing Comple-
tely At Random (MCAR) analysis was performed.
The non-significant result of the MCAR analysis
(χ 2 = 27.095, df = 28, p = .51) revealed that there
was no indication of non-random missingness.
MLM analysis was then conducted to test if thera-
pist competence in case conceptualization and/or
homework use was related to more positive outcomes
in consideration of other salient therapy variables (see
Table V). Analysis of fixed effects in Model 1 (i.e.,
unconditional means model or “null” baseline)
found that patients had a non-zero positive BDI-II
score at intake (γ00 = 10.78, p < .01; SE = 1.33) and
the introduction of fixed effects of time in Model 2
(i.e., unconditional growth model incorporating
 Psychotherapy Research 11
Table V. Uncontrolled and controlled effects of therapist competence in case conceptualization with therapist competence in homework,
patient beliefs about homework, symptoms severity and personality complexity on depressive symptom change.

Fixed effects Parameter Model 1 Model 2 Model 3 Model 4 Model 5 Model 6

Initial status
π0i Intercept γ00 10.78∗∗∗ 8.47∗∗∗ 5.73∗∗∗ 9.65∗∗∗ 7.40∗∗∗ 6.16
1.33 1.06 1.42 3.05 2.48 4.60
CRS (competence) γ01 0.11∗ 0.12∗ 0.13∗∗
0.05 0.06 0.05
HAACS (competence) γ02 −0.03 −0.04 −0.04
0.04 0.04 0.03
HRS (patient beliefs) γ03 −0.01
0.07
CIDI (symptom severity) γ04 2.49∼
1.22
PBQ (complexity) γ05 −2.32
1.94
Variance components Parameter Model 1 Model 2 Model 3 Model 4 Model 5 Model 6

Level 1 Within person s2e 37.91∗∗∗ 28.60∗∗∗ 16.77∗∗∗ 28.41∗∗∗ 16.66∗∗∗ 17.09∗∗∗
3.78 3.02 1.81 3.03 1.81 1.90
Level 2 In initial status s20 42.65∗∗∗ 23.39∗∗∗ 23.52∗∗∗ 23.84∗∗∗ 24.20∗∗∗ 18.99∗∗
12.89 8.35 7.71 8.51 7.93 6.93
In rate of change s21 0.61∗∗∗ 0.51∗∗ 0.58∗∗∗ 0.48∗∗ 0.36∗∗
0.20 0.20 0.21 0.20 0.14
Covariance σe 0.00∗ 0.02∗ 0.02∼ 0.02∼ 0.01
0.01 0.01 0.01 0.01 0.01
Psuedo R 2 statistics and goodness of fit
R2e 0.25 0.41 0.00 0.42 0.40
R20 0.45 0.00 −0.02 −0.03 0.19
R21 0.16 0.11 0.11 0.20

Deviance 1537.64 1483.91 1349.76 1465.32 1330.55 1132.13

AIC 1543.64 1495.98 1365.76 1481.32 1350.55 1157.13
BIC 1553.93 1516.56 1392.99 1508.65 1384.44 1208.66
Δ AIC 47.02 123.57 7.91 132.12 181.81

Note. ∼p < .10, ∗ p < .05, ∗∗ p < .01, ∗∗∗ p < .001.

time) showed that BDI-II scores continued to have a
non-zero positive change over time (γ02 = 8.47,
p < .01; SE = 1.06). The analysis of random effects
showed that a quarter of within patient variance in
depressive symptoms is explained by time (R2e =
25%, p < .001) and almost half of between patient
variance (R20 = 45%, p < .001). The significant
results of variance components suggest that further
variance is able to be explained by the addition of
further predictor variables to the model at both
Level 1 and Level 2. Models 1 and 2 effectively estab-
lished a baseline of positive treatment outcomes over
time (i.e., reduction in depressive symptoms) and
demonstrated that there was sufficient variation
between individual patients to justify further com-
parison using additional substantive predictors.
Hypothesis 1. Model 3 (i.e., the conditional
growth model) introduced therapist competence in
case conceptualization (CRS Competence) into the
model. Analysis of fixed effects revealed that for
every one unit change on the BDI-II, therapist com-
petence in case conceptualization is estimated to
change by 0.11 units (p < .01; SE = 0.05). Analysis
of random effects revealed that inclusion of therapist
competence in case conceptualization accounted for
a 41% reduction in within patient variance. The
within-patient variance (i.e., continuous variable,
random effects) refers to how much patients tended
to change in depression severity over time, referring
to the mean of the change for the average individual
in our sample. Thus, without giving due consider-
ation to pertinent patient variables considered to
influence the therapy outcome (i.e., symptom sever-
ity at intake, dysfunctional personality) therapist
competence in case conceptualization showed a posi-
tive association with treatment outcomes (i.e.,
reduction in depressive symptoms over time) for the
average patient.
Hypothesis 2. Model 4 introduced therapist com-
petence in homework use (HAACS Competence) into
the model independently of therapist competence in
case conceptualization (CRS Competence). Analysis
of fixed effects revealed no direct significant associ-
ation (γ02 = −.03, NS; SE = .04) between therapist
12 M. H. Easden and R. B. Fletcher
competence in homework use and positive change in
depressive symptoms and no reduction in within
patient or between patient variance was observed.
Although therapist competence in homework did
not appear to be a strong predictor of positive
change in depressive symptoms, the reduction in
the Akaike Information Criterion (Δ AIC = 7.91)
from Model 3 to Model 4 suggested that therapist
competence in homework use provided a better
overall fit for in the prediction of positive change in
depressive symptoms and should be retained in the
subsequent models. As such therapist competence
in homework did not reveal a strong association
with treatment outcomes independently, although
may provide some greater impact when we simul-
taneously considering the therapist’s competence in
case conceptualization (e.g., if a therapist has linked
aspects of a shared case conceptualization with
between session tasks or experiments hypothesised
to ultimately lead to patient skill acquisition and
self-awareness towards depressive symptom
reduction).
Hypotheses 1 and 2. Model 5 represented the
combination of therapist competence in case concep-
tualization and in homework use investigated indivi-
dually in Models 3 and 4. Analysis of fixed effects
for therapist competence in homework use in
Model 5 showed that patients had a non-zero positive
BDI-II score at intake which remained significant
(γ00 = 7.40, p < .01; SE = 2.48) suggesting that the
inclusion of additional predictors may be useful in
further explaining variance in the final model. Analy-
sis of fixed effects revealed that the inclusion of thera-
pist competence in homework use in the model
marginally increased the magnitude of therapist com-
petence in case conceptualization (γ01 = 0.12, p < .01;
SE = 0.06). Analysis of random effects explained
more variance at Level 1 than previous models
(R2e = 41%, p < .01). Overall, the reduction in the
Akaike Information Criterion (Δ AIC = 132.12) in
Model 5 produced the largest reduction in compari-
son to previous models suggesting that a combined
model of therapist competence in case conceptualiz-
ation and homework use should be retained for
further models.
Hypotheses 3. Having entered each variable sys-
tematically towards a “final model” therapist compe-
tence in both case conceptualization (CRS
Competence) and homework use (HAACS Compe-
tence) were retained as predictor variables. After con-
sideration of variance explained both individually and
in combination, homework beliefs (HRS Beliefs), per-
sonality beliefs (PBQ Complexity) and symptom
severity (CIDI Symptom Severity) each were retained,
all of which provided significant reductions in var-
iance associated with a relationship with change in
depressive symptoms (BDI Change). Thus, analysis
of fixed effects of the final model (Model 6) demon-
strated that taken together, after controlling for
salient variables that impact the ability of a therapist
to display competence in case conceptualization and
homework use during therapy, that for every one
unit change on the BDI-II, therapist competence in
case conceptualization is estimated to change by
0.13 units (p < .01; SE = 0.05). Model 6 explained
the greatest amount of variance at Level 1 (R2e =
40%, p < .01) and Level 2 (R20 = 19%, p < .001).
Thus, the introduction of Level 2 predictors (i.e., cat-
egorical variables, fixed effects) begins to account for
between-patient variance; that is, that the individual
patient would tend to show more or less change in
depressive symptoms associated with either severe,
moderate or mild initial symptom severity, and
whether there were lesser or greater complexity of
personality features respectively. The variance
explained by Level 1 predictors (i.e., continuous vari-
ables, random effects), however, remained consistent
and suggests stability of the model. The reduction in
the Akaike Information Criterion (Δ AIC = 181.81)
from the “baseline” or unconditional growth model
(Model 3) to Model 6 represent a “very strong”
overall fit of the final model.
Discussion
This is the first study to investigate therapist compe-
tence in case conceptualization as it relates to out-
comes in CBT for depressed patients (i.e.,
symptom change). In order to examine the hypoth-
esised relationship between case conceptualization
and symptom change, the CRS was designed and
developed. The CRS was found to be a reliable
measure with acceptable face and content validity.
The therapist competence scale which formed the
basis of the present investigation was found to have
the most robust reliability overall compared with
other sub-domains on the CRS. Internal consistency
was at least adequate across domains, and good for
therapist competence (α = .79). Consistent with con-
ventional guidelines (Cicchetti, 1994) interrater
reliability was good across domains and excellent
for the therapist competence scale (ICC = .93). Inter-
rater reliability for therapist competence was excel-
lent at the individual item level (ICC = .76–.99) and
at the level of the individual therapy session across
ten therapy sessions (ICC = .91–.96); this was with
the exception of lower, although still good interrater
reliability at session eight (ICC = .64).
High interrater reliability for the integration sub-
domain (k = .83) indicated that raters were able to
agree as to whether or not therapists made explicitly

discussed different components of case conceptualiz-
ation in session irrespective of the level of compe-
tence in doing so (i.e., did the therapist select a
situation and enquire about automatic thoughts,
emotions, behaviour, attempt to share aspects of the
case conceptualization, etc). Similarly high interrater
reliability for the fit/match sub-domain (ICC = .93)
suggested raters were able to agree when information
relevant to the case conceptualization was captured in
written conceptualizations (i.e., J. Beck Case Con-
ceptualization Diagram or other relevant written
format selected by therapists). However, interrater
reliability for the importance sub-domain (k = .65)
was relatively low; this suggests that raters were less
able to agree on what aspects of the case conceptual-
ization should be prioritised for discussion, and con-
versely when aspects of the case conceptualization
were less relevant to the particular session (e.g., expli-
cit discussion and identification of negative core
beliefs would be generally be inadvisable in the first
session of therapy). Of note, raters had the poorest
agreement (k = .12) for the importance of integrating
client resilience / strengths as an explicit focus of dis-
cussion in session. This might reflect a need for
further development of the importance item and
rater training to correctly identify the relative impor-
tance and timing of different aspects of case concep-
tualization during the course of therapy. This might
further reflect a lack of existing guidelines in
regards as to how a therapist can determine the
importance of different aspects of the developing
case conceptualization over time. Few studies have
directly investigated clinicians’ perceptions as to
what is important in case conceptualization. Existing
research suggests a lack of complete consensus
among clinicians and a need for improved clinical
training to address this need (Flitcroft, James, Free-
ston, & Wood-Mitchell, 2007; Huisman & Kangas,
2018).
Internal consistency and total scale interrater
reliability of the CRS were comparable to levels
established for well-trained raters using the CTS
(Vallis et al., 1986) and the CTS-R (Blackburn
et al., 2001), while individual item and individual
session level estimates were more robust with less
variability observed on the CRS. Having demon-
strated that the CRS was a fairly reliable measure in
the current sample in relation to accepted conven-
tions and established measures of therapist compe-
tence we proceeded to test our primary hypotheses.
Hypothesis 1. We examined the validity of the
CRS competence scale as the primary predictor to
determine therapy outcome (i.e., depressive symp-
toms change on the BDI-II). After controlling for
time, there was a positive association between thera-
pist competence in case conceptualization and
Psychotherapy Research 13
outcome (γ01 = .11, p < .01; SE = 0.05) which pro-
vided support for this hypothesis. The results
showed that across models between 40% to 42% of
within patient variance in depression scores (i.e., a
measure of how much on average the individual
patient tended to change or vary over time) was
attributed to therapist competence in case conceptu-
alization, and that after controlling for time and other
pertinent patient characteristics this result remained
stable at 40% in the final model. These results
suggest that the CRS would be a useful measure of
therapist competence in case conceptualization in
CBT to assist in enhancing resolution of depressive
symptomology.
Hypothesis 2. Hypothesis two was not strongly
supported. Therapist competence in homework use
as measured by the HAACS was not associated
with depressive symptom change in our sample
(γ02 = −.03, NS; SE = .04). However, therapist com-
petence in homework and in case conceptualization
as measured on the HAACS and CRS respectively
has a small positive correlation (r = .186; p < .01).
As such, the association between therapist compe-
tence in homework use and therapist competence in
case conceptualization as this related to outcomes
was less clear and would warrant further investi-
gation. One meta-analysis suggested that using
ratings incorporating both patient and therapist
experiences, consideration of timing (i.e., retrospec-
tive or anticipatory) and types of homework tasks
(e.g., thought records, specific behaviour exper-
iments) are more likely to more accurately account
for the impact of homework on treatment outcomes
(Mausbach et al., 2010). Consistent with this
finding, individually, patient beliefs about homework
(i.e., measured on the HRS-II) were found to explain
a greater amount of variance in depression scores
than therapist competence in homework use. Tenta-
tively, the results might suggest that supervisors and
clinicians should consider which measures are most
likely to capture patient and therapist experiences
and the idiosyncrasies of unique homework tasks as
these link to individualised between session home-
work tasks.
Hypothesis 3. The introduction of pertinent
patient attributes in the final combined model
accounted for 19% of between patient variance,
where consideration of personality complexity,
symptom severity and factors related to homework
(i.e., therapist competence and patient beliefs) are
likely to further moderate the extent to which thera-
pists are able to demonstrate competence in case con-
ceptualization and how this might influence symptom
change between patients. However, interpreting the
relative contribution of covariates (i.e., PBQ, CIDI,
and HRS-II) was less readily interpretable. The
14 M. H. Easden and R. B. Fletcher
PBQ was used as a proxy for patient complexity (i.e.,
due to the presence of features of dysfunctional
beliefs) in conjunction with the CIDI which was
used as a proxy of symptom severity. The HRS-II
was used to consider if beliefs about homework
(e.g., that monitoring thoughts / beliefs, emotion
and behaviour or engaging in behavioural exper-
iments in problem situations, etc) separate to the
competence of the therapist conceptualizing and
planning between session activities. However, none
of these measures independently contributed signifi-
cantly to the final model. Consistent with these find-
ings Kuyken and Tsivrikos (2009) found that more
competent therapists achieved improved outcomes
regardless of patient comorbidity and/ presumed
complexity. Strunk, Brotman, DeRubeies, and
Hollon (2010) found that competence was more
highly related to outcomes in patient with higher
symptom severity, but not associated with whether
clients met criteria for personality disorders pre-
sumed to be higher in complexity. In practice the
implication is that matching the most competent or
experienced therapists with patients deemed to be
most difficult may be inconsequential or less impor-
tant that might be presumed during a triage or refer-
ral process.
Limitations and Future Research
The results of the current study support the over-
arching hypothesis that therapist competence in
case conceptualization is associated with positive
outcomes in CBT in the context of depression.
However, it must be noted that this study does
not directly assess therapist competence as a com-
plete construct (Barber et al., 2007). More specifi-
cally, the results support the systematic approach
taken to demonstrate therapist competence in
case conceptualization as defined and operationa-
lised in the CRS measure as a whole. Future
research could provide a closer analysis of relative
contribution of therapist competence on different
components of case conceptualization as measured
by the individual items within the CRS (i.e., behav-
ioural, emotional, the importance of systematic
sharing of the case conceptualization, focus on
client strengths, etc).
The present study was limited by the modest
number of patients (N = 28) included in the sample.
In contrast, the number of individual session ratings
(N = 225) generated using therapist competence
measures was relatively large. Overall, the sample
was comparable or larger than previous studies con-
ducted using independently rated measures of thera-
pist competence to evaluate treatment-outcome
relations (see a meta-analytical review by Webb
et al., 2010 and accompanying supplementary
data). Furthermore strict inclusion criteria meant
patients in this study would not be representative of
the range depressed patients in a naturalistic out-
patient setting likely to have multiple comorbid diag-
noses. However, even within this fairly homogeneous
sample patient complexity (e.g., personality beliefs,
symptom severity) contributed to the ability to
explain between patient differences in our model.
Similarly, the study was unique in that clients
taking CNS acting medication were excluded.
Although in practice clients are routinely prescribed
antidepressant medication concurrently while under-
going CBT for depression, the outcomes and results
of the current study cannot be considered attribu-
table to the effect of medication, while nor can they
be considered superior. Further investigation of
therapist competence in case conceptualization in a
range of clients groups (e.g., personality disorders,
psychotic spectrum disorders), different settings
(e.g., inpatient, forensic, naturalistic) and with
larger sample sizes of patients and particularly thera-
pists is recommended.
In the present study the Cognitive Therapy Scale
(CTS; Young and Beck, 1980) was not able to be
included in analyses due to insufficient data, nor
were there any other available measures of therapist
competence in case conceptualization at the incep-
tion of the study to provide further examination of
concurrent validity. Future research would ideally
compare global domain and limited domain
measures of therapist competence as these become
available.
Once sufficient reliability is established, among
measures of global and limited domain competence,
therapy outcome-relations can begin to be eluci-
dated. One relatively large study of 43 therapists pro-
vided CBT to 1247 patients with anxiety and/or
depression. In general therapist competence on the
CTS-R was not associated with improved patient
outcomes. However, there was some association
between the most competent (top 10%) and least
competent (bottom 10%) therapists with patient
improvement and patient deterioration respectively
(Branson, Shafran, & Myles, 2015). While the
authors suggested that measures of global compe-
tence such as the CTS-R may be limited in their
ability to detect specific therapist effects, the research
also speaks to the complex relationship between
different aspects of therapist competence and
therapy outcome. Future research might adopt such
an approach alongside comparison of measures of
therapist competence that seek to directly measure
or tap into case conceptualization as a process and
central focus of psychotherapy.
 Psychotherapy Research 15

Implications for Practice challenging or upsetting cognitions. Furthermore,

discussing the most important or clinically relevant
This study presents several implications relevant to
content in depth, as agreed upon by both the therapist
the practice of CBT. Firstly, the present research
and patient, and routinely helping to patient to notice
has identified particular areas of therapist compe-
how different aspects of their case conceptualization
tence in case conceptualization hypothesised to
are linked to both individual weaknesses and
account for change in CBT for depression. Data
strengths is likely to improve treatment outcomes.
from this study support therapists’ systematic inte-
In terms of implications for training, the present
gration of metacompetencies outlined in the CRS
study involved postgraduate-level independent
considered pertinent to comprehensive cognitive be-
observers and clinicians. Research has demon-
havioural case conceptualization. Table VI summar-
strated that expert therapists are generally more
ises the key features of case conceptualization
skilled in constructing case conceptualizations and
included in the CRS that have demonstrated clinical
perhaps better able to make judgements regarding
utility as a whole (while future studies with larger
what makes a quality case conceptualization than
samples might investigate the relative contribution
experienced or novice therapists (Eells, Lombart,
of each feature). Notwithstanding limitations of the
Kendjelic, Turner, & Lucas, 2005). Regardless,
study, systematic attention to these key areas by the
the use of well-trained raters and a structured and
therapist is suggested to be of benefit to the course
systematic approach to case conceptualization (as
of treatment for depressed patients. The features
we adopted in the current study) has been advocated
are intended to strengthen empirically, and provide
as a potential means to evaluate case conceptualiz-
guidelines for the process of developing and integrat-
ation in CBT (Bieling & Kuyken, 2003; Kuyken
ing case conceptualization as intended in standard
et al., 2005). Our findings suggest that through suf-
CBT for depression (Beck, 1976; Beck et al.,
ficient training (i.e., thirteen hours or the equivalent
1979). For example, attention has been drawn to
of two days of training covering an introduction to
the importance of developing an individualized case
case conceptualization theory and practice in CBT
conceptualization in CBT (Beck, 1995; Needleman,
and the use of the CRS) independent raters con-
1999; Persons, 2001). The guidelines in Table VI
sidered to possess “novice” level experience are
provide some structure towards achieving an indivi-
able to reach high levels of agreement using the
dualized and clinically valid case conceptualization.
CRS.
The results of the present study begin to support
A third implication applies not just to the develop-
the consistent and systematic integration of different
ment of competence in delivering evidence-based
components of case conceptualization as outlined on
interventions consistent with training provided, but
the J. Beck Case Conceptualization Diagram (CCD
also to the maintenance of therapist competence
1995, 2011) written format. Put another way, the
overtime, and as this applies to use and implemen-
research supports the need to discuss each part of
tation of standardised treatment manuals (Waller
the case conceptualization with the patient systemati-
& Turner, 2016). Varying levels of therapist compe-
cally in therapy sessions, but also to do this in a timely
tence in case conceptualization have been observed
manner as the patient is able to cope with increasingly
between therapists and across time, supporting the

Table VI. Common features of in-session therapist competence in CBT case conceptualization.

1. The therapist should consistently discuss in-depth all aspects of a CBT case conceptualization (e.g., automatic thoughts, core
beliefs) apart from when clinically inadvisable (i.e., would cause undue distress without future benefit)
2. The therapist should also make judgements about the timing of discussions of “deeper-level” more inferential (e.g., or potentially
traumatic or difficult) aspects of the conceptualization and gradually increase the depth of discussion
3. The therapist should foster discussions to link different areas of case conceptualization
4. The therapist should seek regular patient feedback (i.e., collaborative empiricism) at each level of conceptualization including the
tentative overall conceptualization, mini-conceptualizations, and relevant disorder-specific models during the session in regards to
the patient’s:
a) Acceptance of the conceptualization
b) Understanding of the conceptualization
c) Assessment of clinical relevance (i.e., importance) of the conceptualization to the resolution of presenting problems and
building of patient strengths
5. The therapist should conceptualize patient strengths as well as weakness as routine practice (NB: The same attribute may severe to
function and both strength and weakness in different situations, and provides opportunities to validate a patient’s efforts to cope)

Note. Summarised from key features of the therapist competence scale of the Conceptualization Rating Scale (CRS). The CRS is available by
request from the primary author.
16 M. H. Easden and R. B. Fletcher
notion that regardless of the validity of the treatment
manual that the ability of the therapist to demon-
strate competence in using that manual will directly
impact the resultant treatment utility of the manual
for an individual patient. It has been asserted that
“the interventionist does not always equal the inter-
vention” (Nezu & Nezu, 2005). As such integrity
checks and systematic feedback are ideally offered
to therapists regarding their levels of competence
in different components of treatment based on live
observation or ratings of therapy sessions (e.g.,
using observational measures such as the CTS or
the CRS in the context of CBT). It has been
suggested that systematic “spot checks” and
regular supervisory feedback be provided to thera-
pists in consideration of the cost of detailed and
time consuming adherence and competence
ratings. In our sample therapists demonstrating
high levels of competence in early sessions were
likely to remain competent. One meta-analysis con-
cluded that the impact of therapist competence on
psychotherapy outcomes was heavily moderated by
the therapeutic relationship (Webb et al., 2010).
As such, to some extent the increase in competence
might be explained by the level of collaboration
intrinsic to high ratings of therapist competence in
case conceptualization on the CRS (as per guide-
lines outlined in Table VI). Based on our findings
taking successive measures of therapist competence
in case conceptualization early in the therapy
process, until a therapist has demonstrated a
minimum level of competence, would be sufficient
to begin to infer future performance with an individ-
ual patient. However the current research has yet to
elucidate more fully under what conditions therapist
competence is likely to change over time. For
example, one study reported levels of therapist com-
petence trending towards decreasing over time; this
was despite showing an association between
increased therapist competence and improved
overall outcomes of a composite score based on
measures of anxiety, depression and functioning
(Brown et al., 2013).
Alternatively, clinicians have suggested methods
of systematic and ongoing self-monitoring of an
individual therapist’s own performance (for
reviews of established methods see Duncan &
Reese, 2015; Lambert & Shimokawa, 2011;
Miller, Duncan, Sorrell, & Brown, 2005;
Østergård, Randa, Hilde & Hougaard, 2018),
self-reflective practice (Bennett-Levy, Thwaites,
Haarhoff, & Perry, 2015; Haarhoff, Gibson, &
Flett, 2011) or suggested the examination of indi-
vidual therapist’s levels of performance at the
level of the clinical practice (Okiishi, Lambert,
Nielsen, & Ogles, 2003; Persons, 2006; Persons,
Bostrom, & Bertagnolli, 1999; Persons, Roberts,
Zalecki, & Brechwald, 2006). The broad impli-
cation is that measures and methods suitable for
use in routine psychotherapy practice must be
developed, validated, and shown to be effective
before definitive conclusions can be drawn in
regards to empirically-based guidelines for
routine clinical practice (Castonguay, Barkham,
Lutz, & McAleavey, 2013).
Conclusion
This research is the first to investigate how thera-
pist competence relates to depressive symptom
change using time-series data, with a focus on
case conceptualization as a measurable and valid
area of central therapist competencies in CBT.
Case conceptualization research in CBT has
tended to focus on examination of inter-clinician
reliability in constructing case conceptualizations,
utilizing expert benchmarks of “gold standard”,
“good enough” or idealised prototypes of case con-
ceptualization content (Easden & Kazantzis,
2018). The present study adopted a novel
approach and more directly considered how case
conceptualization is used in practice when thera-
pists are provided minimally sufficient training in
CBT and case conceptualization. By observing
therapist use of case conceptualization in-vivo the
results suggest that with sufficient training thera-
pists can agree on what is important and relevant
in a case conceptualization. They can recognise
different components and can agree on how well
or how skilfully a therapist has integrated these
(or chosen not to integrate these) into a therapy
session. By utilizing the CRS, we provide prelimi-
nary evidence to support the assertion that cogni-
tive case conceptualization is associated with
improved treatment outcomes (i.e., treatment
utility) in CBT. Although further validation is
required, the CRS (1) provides a valid and reliable
structure for assessing case conceptualization com-
petencies in CBT during therapist sessions, (2) is
suitable for use as a training instrument in the
practice of case conceptualization in CBT, (3)
can be used to inform future attempts to operatio-
nalize and isolate aspects of case conceptualization,
and (4) is based on cognitive behavioural theory.
Finally, the present research has added to the
basis of empirical support for both cognitive behav-
ioural theory and case conceptualization as a
central therapeutic process in psychotherapy and
encourages extension of this research in future
process-oriented clinical trials and naturalistic
research.

Acknowledgement
The authors thank the experts who kindly provided
feedback for revision and development of drafts of
the Conceptualization Rating Scale (CRS) particu-
larly Judith Beck, Willem Kuyken, Keith Dobson,
and Beverly Haarhoff. The authors also acknowl-
edge Nikolaos Kazantzis for his contribution as
the primary investigator in the “CBT Depression
Study” along with Paul Merrick and Janet
Leathem.
Funding
This research was supported by the Massey Univer-
sity School of Psychology, and partially funded by
Albany Strategic Research and Lottery Health
Research Council grants. We wish to thank Margo
Munro for her assistance with the research design
and grant applications.
ORCID
Michael H. Easden http://orcid.org/0000-0002-
2577-2352
References
American Psychiatric Association. (2000). Diagnostic and statistical
manual of mental disorders (4th ed., text rev). Washington, DC:
Author.
Andrews, G., & Peters, L. (1998). The psychometric properties of
the composite international diagnostic interview. Social
Psychiatry and Psychiatric Epidemiology, 33, 80–88. doi:10.
1007/s001270050026
Barber, J. P., Sharpless, B. A., Klostermann, S., & McCarthy, K. S.
(2007). Assessing intervention competence and its relation to
therapy outcome: A selected review derived from the outcome
literature. Professional Psychology: Research and Practice, 38(5),
493–500. doi:10.1037/0735-7028.38.5.493
Beck, A. T. (1976). Cognitive therapy and the emotional disorders.
New York: International Universities Press.
Beck, J. (1995). Cognitive therapy: Basics and beyond. New York:
Guilford Press.
Beck, J. (2011). Cognitive therapy: Basics and beyond (2nd ed.).
New York: Guilford Press.
Beck, A. T., & Beck, J. S. (1991). The personality belief questionnaire
(Unpublished manuscript). University of Pennsylvania.
Beck, A. T., Butler, A. T., Brown, G. K., Dahlsgaard, K. K.,
Newman, C. F., & Beck, J. S. (2001). Dysfunctional beliefs dis-
criminate personality disorders. Behaviour Research and
Therapy, 39, 1213–1225. doi:10.1016/S0005-7967(00)00099-1
Beck, A. T., & Freeman, A. (1990). Cognitive therapy of personality
disorders. New York: Guilford.
Beck, A. T., Rush, A., Shaw, B., & Emery, G. (1979). Cognitive
therapy of depression. New York: The Guilford Press.
Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Beck depression
inventory manual (2nd ed.). San Antonio, TX: Psychological
Corporation.
Psychotherapy Research 17
Bennett-Levy, J., Thwaites, R., Haarhoff, B., & Perry, H. (2015).
Experiencing CBT from the inside out: A self-practice/self-reflection
workbook for therapists. New York: The Guildford Press.
Bieling, P., & Kuyken, W. (2003). Is cognitive case formulation
science or science fiction? Clinical Psychology: Science and
Practice, 10(1), 52–69. doi:10.1093/clipsy.10.1.52
Bjornholdt, A. (2006). A preliminary psychometric investigation of the
homework rating scale-II (Unpublished master’s thesis). Albany:
Massey University.
Blackburn, I., James, I. A., Milne, D., Baker, C., Standart, S.,
Garland, A., & Reichelt, F. (2001). The revised cognitive
therapy scale (CTS-R): Psychometric properties. Behavioural
and Cognitive Psychotherapy, 29, 431–446. doi:10.1017/
S1352465801004040
Branson, A., Shafran, R., & Myles, P. (2015). Investigating the
relationship between competence and patient outcome with
CBT. Behaviour Research and Therapy, 68, 19–26. doi:10.
1016/j.brat.2015.03.002
Brown, L., Craske, M., Glenn, D., Stein, M., Sullivan, G.,
Sherbourne, C., … Rose, R. (2013). CBT competence in
novice therapists improves anxiety outcomes. Depression and
Anxiety, 30(2), 97–115. doi:10.1002/da.22027
Bucci, S., French, L., & Berry, K. (2016). Measures assessing the
quality of case conceptualization: A systematic review. Journal of
Clinical Psychology, 72(6), 517–533. doi:10.1002/jclp.22280
Butler, A. C., Brown, G. K., Beck, A. T., & Grisham, J. R. (2002).
Assessment of dysfunctional beliefs in borderline personality
disorder. Behaviour Research and Therapy, 40, 1231–1240.
doi:10.1016/S0005-7967(02)00031-1
Castonguay, L., Barkham, M., Lutz, W., & McAleavey, A. (2013).
Practice-oriented research: Approaches and applications. In M.
J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy
and behaviour change (6th ed., pp. 85–133). Hoboken, NJ: Wiley.
Chadwick, P., Williams, C., & Mackenzie, J. (2003). Impact of
case formulation in cognitive behaviour therapy for psychosis.
Behaviour Research and Therapy, 41, 671–680. doi:10.1016/
S0005-7967(02)00033-5
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for
evaluating normed and standardized assessment instruments in
psychology. Psychological Assessment, 6(4), 284–290. doi:10.
1037/1040-3590.6.4.284
Clark, D. M. (1986). A cognitive approach to panic. Behaviour
Research and Therapy, 24, 461–470. doi:10.1016/0005-7967
(86)90011-2
Crits-Christoph, P., Cooper, A., & Luborsky, L. (1988). The accu-
racy of therapists’ interpretations and the outcome of dynamic
psychotherapy. Journal of Consulting and Clinical Psychology,
56, 490–495. doi:10.1037/0022-006X.56.4.490
Davidson, K., Scott, J., Schmidt, U., Tata, P., Thornton, S., &
Tyrer, P. (2004). Therapist competence and clinical outcome
in the prevention of parasuicide by manual assisted cognitive be-
haviour therapy trial: The POPMACT study. Psychological
Medicine, 34, 855–863. doi:10.1017/S0033291703001855
Dozois, D. J. A., Covin, R., & Brinker, J. (2003). Normative data
on cognitive measures of depression. Journal of Consulting and
Clinical Psychology, 71(1), 71–80. doi:10.1037/0022-006X.71.
1.71
Dozois, D. J. A., Dobson, K. S., & Ahnberg, J. L. (1998). A psy-
chometric evaluation of the Beck depression inventory-II.
Psychological Assessment, 10, 83–89. doi:10.1037/1040-3590.
10.2.83
Duncan, B. L., & Reese, R. J. (2015). The partners for change
outcome management system (PCOMS) revisiting the client’s
frame of reference. Psychotherapy, 52(4), 391–401. doi:10.
1037/pst0000026
Easden, M. H., & Kazantzis, N. (2018). Case conceptualization
research in cognitive behavior therapy: A state of the science
18 M. H. Easden and R. B. Fletcher
review. Journal of Clinical Psychology, 74(3), 356–384. doi:10.
1002/jclp.22516
Eells, T. (2007). Handbook of psychotherapy case formulation (2nd
ed.). New York: Guilford Press.
Eells, T., Kendjelic, E., & Lucas, C. (1998). What’s in a case for-
mulation? Development and use of a content coding manual.
Journal of Psychotherapy Practice and Research, 7(2), 144–153.
Eells, T., Lombart, K. G., Kendjelic, E. M., Turner, L., & Lucas,
C. P. (2005). The quality of psychotherapy case formulations: A
comparison of expert, experienced, and novice cognitive-behav-
ioral and psychodynamic therapists. Journal of Consulting and
Clinical Psychology, 73(4), 579–589. doi:10.1037/0022-006X.
73.4.579
Flitcroft, A., James, I. A., Freeston, M., & Wood-Mitchell, A.
(2007). Determining what is important in a good formulation.
Behavioural and Cognitive Psychotherapy, 35, 325–333. doi:10.
1017/S135246580600350X
Haarhoff, B., Gibson, K., & Flett, R. (2011). Improving the quality
of cognitive behaviour therapy case conceptualization: The role
of self-practice/self-reflection. Behavioural and Cognitive
Psychotherapy, 39, 323–339. doi:10.1017/S1352465810000871
Huisman, P., & Kangas, M. (2018). Evidence-based practices in
cognitive behaviour therapy (CBT) case formulation: What do
practitioners believe is important, and what do they do?
Behaviour Change, 35(1), 1–21. doi:10.1017/bec.2018.5
Kazantzis, N., Bjornholdt, A., Munro, M., Dobson, K., Merrick,
P., Fetcher, R., & Jones, D. (2006). Development of the homework
rating scale: A measure of patients’ beliefs about homework in cogni-
tive therapy. Poster presented at the 40th annual convention of
the association for behavioural and cognitive therapies
(ABCT), Chicago.
Kazantzis, N., Deane, F. P., & Ronan, K. R. (2000). Homework
assignments in cognitive and behavioral therapy: A meta-analy-
sis. Clinical Psychology: Science and Practice, 7, 189–202. doi:10.
1093/clipsy.7.2.189
Kazantzis, N., Deane, F. P., Ronan, K. R., & L’Abate, L. (Eds.).
(2005). Using homework assignments in cognitive behavior
therapy. New York: Routledge.
Kazantzis, N., Wedge, P., & Dobson, K. (2005). Homework adher-
ence and competence scale (HAACS).
Kazantzis, N., Whittington, C., & Dattilio, F. (2010). Meta-analysis
of homework effects in cognitive and behavioral therapy: A repli-
cation and extension. Clinical Psychology: Science and Practice, 17
(2), 144–156. doi:10.1111/j.1468-2850.2010.01204.x
Kazantzis, N, Whittington, C, Zelencich, L, Kyrios, M, Norton, P,
& Hofmann, S, G. (2016). Quantity and quality to homework
compliance: A meta-analysis of relations with outcome in
Cognitive Behavior Therapy. Behavior Therapy, 47(5), 755–
772. doi:10.1016/j.beth.2016.05.002
Kessler, R., Berglund, P., Demler, O., Jin, R., Koretz, D.,
Merikangas, K., … Wang, P. (2003). The epidemiology of
major depressive disorder: Results from the national comorbid-
ity survey replication (NCS-R). Journal of the American Medical
Association, 289(23), 3095–3105. doi:10.1001/jama.289.23.
3095
Kuyken, W. (2006). Evidence-based case formulation: Is the
emperor clothed? In N. Tarrier (Ed.), Case formulation in cogni-
tive behaviour therapy: The treatment of challenging and complex
clinical cases (pp. 12–35). London: Brunner-Routledge.
Kuyken, W, Beshai, S, Dudley, R, Abel, A, Görg, N, Gower, P …
McManus, F. (2016). Assessing competence in collaborative
case conceptualization: Development and preliminary
psychometric properties of the Collaborative Case
Conceptualization Rating Scale (CCC-RS). Behavioural and
Cognitive Psychotherapy, 44(2), 179–192. doi:10.1017/
S1352465814000691
Kuyken, W., Fothergill, C., Musa, M., & Chadwick, P. (2005).
The reliability and quality of cognitive case formulation.
Behaviour Research and Therapy, 43, 1187–1201. doi:10.1016/
j.brat.2004.08.007
Kuyken, W., Padesky, C., & Dudley, R. (2008). The science and
practice of case conceptualization. Behavioural and Cognitive
Psychotherapy, 36, 757–768. doi:10.1017/S1352465808004815
Kuyken, W., Padesky, C., & Dudley, R. (2009). Collaborative case
conceptualization: Working effectively with clients in cognitive-be-
havior therapy. New York: The Guilford Press.
Kuyken, W., & Tsivrikos, D. (2009). Therapist competence, co-
morbidity and cognitive-behavioral therapy for depression.
Psychotherapy and Psychosomatics, 78, 42–48. doi:10.1159/
000172619
Kwok, O., Underhill, A., Berry, J., Luo, W., Elliot, T., & Yoon, M.
(2008). Analyzing longitudinal data with multilevel models: An
example with individuals living with lower extremity intra-
articular fractures. Rehabilitation Psychology, 53(3), 370–386.
doi:10.1037/a0012765
Lambert, M. J., & Shimokawa, K. (2011). Collecting client feed-
back. Psychotherapy, 48, 72–79. doi:10.1037/a0022238
Mausbach, B. T., Moore, R., Roesch, S., Cardenas, V., &
Patterson, T. L. (2010). The relationship between homework
compliance and therapy outcomes: An updated meta-analysis.
Cognitive Therapy and Research, 34(5), 429–438. doi:10.1007/
s10608-010-9297-z
McNeish, D., & Stapleton, L. (2016). The effect of small sample
size on two-level model estimates: A review and illustration.
Educational Psychology Review, 28(2), 295–314. doi:10.1007/
s10648-014-9287-x
Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, G. S. (2005).
The partners for change outcome management system. Journal
of Clinical Psychology, 61(2), 199–208. doi:10.1002/jclp.20111
Munro, M. D. (2006). Client and therapist variability with psy-
chotherapy homework: A preliminary psychometric evaluation of
two scales (Unpublished master’s thesis). Albany: Massey
University.
Needleman, L. (1999). Cognitive case conceptualization: A guidebook
for practitioners. Mahwah, NJ: Lawrence Erlbaum Associates.
Nezu, A. M., & Nezu, C. M. (2005). Comments on “evidence-
based behavioral medicine: What is it and how do we achieve
it?”: The interventionist does not always equal the interven-
tion—the role of therapist competence. Annals of Behavioral
Medicine, 29(1), 80–80. doi:10.1207/s15324796abm2901_11
Okiishi, J., Lambert, M. J., Nielsen, S. L., & Ogles, B. M. (2003).
Waiting for supershrink: An empirical analysis of therapist
effects. Clinical Psychology and Psychotherapy, 10, 361–373.
doi:10.1002/cpp.383
Østergård, O. K, Randa, H., & Hougaard, E. (2018). The effect of
using the partners for change outcome management system as
feedback tool in psychotherapy: A systematic review and
meta-analysis. Psychotherapy Research. doi:10.1080/10503307.
2018.1517949
Padesky, C. A., & Mooney, K. A. (1990). Presenting the cognitive
model to clients. International Cognitive Therapy Newsletter, 6,
13–14.
Persons, J. (1989). Cognitive therapy in practice: A case formulation
approach. New York: Norton.
Persons, J. (2001). Individualized formulation and treatment plan-
ning. In J. Persons, J. Davidson, & M. Tompkins (Eds.),
Essential components of cognitive-behavior therapy for depression
(pp. 22–55). Washington, DC: American Psychological
Association.
Persons, J. (2006). Case formulation-driven psychotherapy.
Clinical Psychology: Science and Practice, 13(2), 167–170.
doi:10.1111/j.1468-2850.2006.00019.x

Persons, J. (2008). A case formulation approach to cognitive-behavior
therapy. New York: Guilford.
Persons, J., Bostrom, A., & Bertagnolli, A. (1999). Results of ran-
domized controlled trials of cognitive therapy for depression
generalize to private practice. Cognitive Therapy and Research,
23(5), 535–548. doi:10.1023/A:1018724505659
Persons, J., Roberts, N., Zalecki, C., & Brechwald, W. (2006).
Naturalistic outcome of case formulation-driven cognitive-be-
havior therapy for anxious depressed outpatients. Behaviour
Research and Therapy, 44(7), 1041–1051. doi:10.1016/j.brat.
2005.08.005
Peugh, J. L., & Enders, C. K. (2005). Using the SPSS mixed pro-
cedure to fit cross-sectional and longitudinal multilevel models.
Educational and Psychological Measurement, 65(5), 717–741.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear
models: Applications and data analysis methods (2nd ed.).
Thousand Oaks, CA: Sage.
Sachsenweger, M. A., Fletcher, R. B., & Clarke, D. (2015).
Pessimism and homework in CBT for depression. Journal of
Clinical Psychology, 71(12), 1153–1172. doi:10.1002/jclp.22227
Schotte, C. K. W., Maes, M., Cluydts, R., De Doncker, D., &
Cosyns, P. (1997). Construct validity of the Beck depression
inventory in a depressive population. Journal of Affective
Disorders, 46(2), 115–125. doi:10.1016/S0165-0327(97)00094-3
Shaw, B., Olmsted, M., Dobson, K., Sotsky, S., Elkin, I.,
Yamaguchi, J., … Imber, S. (1999). Therapist competence
ratings in relation to clinical outcome in cognitive therapy of
depression. Journal of Consulting and Clinical Psychology, 67(6),
837–846. doi:10.1037/0022-006X.67.6.837
Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analy-
sis. New York: Oxford University Press.
Snijders, T., & Bosker, R. (2012). Multilevel analysis: An introduc-
tion to basic and advanced multilevel modelling (2nd ed.).
London: Sage.
Steer, R. A., Ball, R., Ranieri, W. F., & Beck, A. T. (1999).
Dimensions of the beck depression inventory-II in clinically
depressed outpatients. Journal of Clinical Psychology, 55(1),
117–128. doi:10.1002/(SICI)1097-4679(199901)55:1<117::
AID-JCLP12>3.0.CO;2-A
Strunk, D. R., Adler, A. D., & Hollars, S. N. (2013). Cognitive
therapy skills predict cognitive reactivity to sad mood following
cognitive therapy for depression. Cognitive Therapy and
Research, 37(6), 1214–1219. doi:10.1007/s10608-013-9570-z
Psychotherapy Research 19
Strunk, D. R., Brotman, M. A., DeRubeis, R. J., & Hollon, S. D.
(2010). Therapist competence in cognitive therapy for
depression: Predicting subsequent symptom change. Journal of
Consulting and Clinical Psychology, 78(3), 429–437. doi:10.
1016/j.brat.2010.03.011
Trepka, C., Rees, A., Shapiro, D., Hardy, G., & Barkham, M.
(2004). Therapist competence and outcome of cognitive
therapy for depression. Cognitive Therapy and Research, 28(2),
143–157. doi:10.1023/B:COTR.0000021536.39173.66
Vallis, T. M, Shaw, B. F, & Dobson, K. S. (1986). The cognitive
therapy scale: Psychometric properties. Journal of Consulting
and Clinical Psychology, 54(3), 381–385. doi:10.1037/0022-
006X.54.3.381
Waller, G., & Turner, H. (2016). Therapist drift redux: Why well-
meaning clinicians fail to deliver evidence-based therapy, and
how to get back on track. Behaviour Research and Therapy, 77,
129–137. doi:10.1016/j.brat.2015.12.005
Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist
adherence/competence and treatment outcome: A meta-ana-
lytic review. Journal of Consulting and Clinical Psychology, 78
(2), 200–211. doi:10.1037/a0018912
Wedge, P. (2005). The development of the homework adherence and
competence scale (HAACS): A measure for assessing therapist adher-
ence and competence in administering homework assignments within
cognitive behaviour therapy (Unpublished master’s thesis). Albany:
Massey University.
Weerasekera, P. (1996). Multiperspective case formulation: A step
towards treatment integration. Melabar, FL: Krieger Publishing.
World Health Organization. (1997). Composite international diag-
nostic interview (CIDI-auto), version 2.1. Geneva: Author.
Wynants, L., Boumeester, W., Moons, K. G., Moerbeek, M.,
Timmerman, D., Van Huffel, S., … Vergouwe, Y. (2015). A
simulation study of sample size demonstrated the importance
of the number of events per variable to develop prediction
models in clustered data. Journal of Epidemiology, 68(12),
1406–1414. doi:10.1016/j.jclinepi.2015.02.002
Young, J, & Beck, A. T. (1980). Cognitive therapy scale rating
manual. Retrieved from www.beckinstitute.org.
Zarafonitis-Müller, S., Kuhr, K., & Bechdolf, A. (2014). The
relationship between therapist’s competence and adherence to
outcome in cognitive-behavioural therapy: Results of a metaa-
nalysis. Fortschritte der Neurologie-Psychiatrie, 82(9), 502–510.
doi:10.1055/s-0034-1366798