You are on page 1of 9

J. Behav. Ther. & Exp. Psychiat.

48 (2015) 66e74

Contents lists available at ScienceDirect

Journal of Behavior Therapy and
Experimental Psychiatry
journal homepage:

The utility of case formulation in treatment decision making; the
effect of experience and expertise
Robert Dudley a, b, *, Barry Ingham b, Katy Sowerby a, Mark Freeston b, c

Doctorate of Clinical Psychology, Newcastle University, UK
Northumberland Tyne and Wear NHS Trust, UK
Institute of Neuroscience, Newcastle University, UK

a r t i c l e i n f o

a b s t r a c t

Article history:
Received 7 July 2014
Received in revised form
17 January 2015
Accepted 22 January 2015
Available online 18 February 2015

Objectives: We examined whether case formulation guides the endorsement of appropriate treatment
strategies. We also considered whether experience and training led to more effective treatment decisions. To examine these questions two related studies were conducted both of which used a novel
paradigm using clinically relevant decision-making tasks with multiple sources of information.
Methods: Study one examined how clinicians utilised a pre-constructed CBT case formulation to plan
treatment. Study two utilised a clinician-generated formulation to further examine the process of
formulation development and the impact on treatment planning. Both studies considered the effect of
therapist experience.
Results: Both studies indicated that clinicians used the case formulation to select treatment choices that
were highly matched to the case as described in the vignette. However, differences between experts and
novice clinicians were only demonstrated when clinicians developed their own formulations of case
material. When they developed their own formulations the experts' formulations were more parsimonious, internally consistent, and contained fewer errors and the experts were less swayed by irrelevant
treatment options.
Limitations: The nature of the experimental task, involving ratings of suitability of possible treatment
options suggested for the case, limits the interpretation that formulation directs the development or
generation of the clinician's treatment plan. In study two the task may still have limited the capacity to
demonstrate further differences between expert and novice therapists.
Conclusions: Formulation helps guide certain aspects of effective treatment decision making. When
asked to generate a formulation clinicians with greater experience and expertise do this more effectively.
Crown Copyright © 2015 Published by Elsevier Ltd. All rights reserved.

Cognitive therapy

Case formulation is the process of blending the theoretical
framework and scientific knowledge the clinician brings with the
unique experience of the client to help understand the presenting
issues, and to select the optimal treatment (Kuyken, Padesky, &
Dudley, 2009; Mumma & Mooney, 2007). Consequently, formulation is considered to be at the heart of effective Cognitive Behavioural Therapy (CBT, Butler, 1998). However, the status afforded
case formulation is somewhat mismatched with the relatively scant
evidence base (Bieling & Kuyken, 2003).

* Corresponding author. Doctorate of Clinical Psychology, Ridley Building, Newcastle University, Newcastle Upon Tyne, England, NE1 7RU, UK. Tel.: þ44 191 222
E-mail address: (R. Dudley).
0005-7916/Crown Copyright © 2015 Published by Elsevier Ltd. All rights reserved.

Most of the previous research on formulation has examined the
extent to which clinicians agree with each other or with an expert's
formulation (e.g. Dudley, Park, James, & Dodgson, 2010; Kuyken,
Fothergill, Musa, & Chadwick, 2005). Clinicians demonstrate
modest levels of agreement (see for example Dudley et al., 2010;
that demonstrated most elements of a formulation were not agreed
on by over 70% of clinicians); although this is better for more overt
presenting issues (where agreement was often over 70%). Also,
clinicians with advanced training, more clinical experience, and
accreditation as a CBT therapist, produce more reliable formulations (Kuyken et al., 2005; Persons & Bertagnolli, 1999).
Research considering validity has shown that therapist expertise (Kuyken et al., 2005) and training in case formulation
(Kendjelic & Eells, 2007) improve the quality of case formulations

e. 10. The model for the CBT case conceptualisation diagram (see Fig. 2005). Participants made judgements as to which treatment planning options were the best fit for the presented case formulations. a full formulation was provided. Conversely. & Mackenzie. There were two aims to this research.3 41. Lombart. Study one If formulation guides treatment.. The withinsubjects (formulation type with two levels. 0% ¼ ¼ ¼ ¼ 11. 1992).8 8 117. Ghaderi. validity and utility is mixed and at present it is not clear how formulation affects treatment outcome (Kuyken. CBT-related qualifications. 48 (2015) 66e74 produced and lead to higher quality treatment plans (Eells & Lombart. 0% 0.67 R. Research into the utility of formulation has considered the relationship between CBT case formulation and therapeutic outcome and has produced mixed results (Chadwick. or plausible but actually unhelpful or irrelevant treatment options. Then an initial psychological conceptualisation (described as developed after two assessment sessions) outlined an early working hypothesis for the development and maintenance of difficulties.3.2 175. Of course.1 14. Novice (n ¼ 23) Experienced (n ¼ 20) Gender (n. Shaw. Beshai. greater experience in CBT should increase recognition of such features (i. Schulte. Design A mixed (between-within) design was used. Behav.4 64. Gower et al. 2003. a referral letter from the client's GP relevant history from childhood to present day including early experiences and problem history. novice or experienced) tested the secondary hypothesis that differences in clinician experience would account for variance in performance. stressing the importance of affiliations and interpersonal relations). 2003. Beck. 2015). Participants Two groups (labelled as novice and experienced practitioners) were recruited. & Exp. & Lucas. 2.8 8. 55% 75% 50% 90% . 2007). Eells. Mumma & Mooney. This consisted of. The sample came from a variety of backgrounds but were clinicians who had supervised others in training and practice of CBT (e. & Schulte-Behrenburg. Participants were presented with two prepared case formulation vignettes followed by multiple-choice options of potential CBT treatments. Method 2. experienced clinical psychologists. further/higher education) within clinician experience groups.2 86. Previous research has indicated that greater experience in CBT leads to better performance on formulation tasks (Kuyken.3 16. multiple factors potentially impact on treatment outcome. Pepping. 1998).. 15. even though both people would meet the criteria for major depressive disorder. The second study utilised a clinician-generated formulation. Kunzel. 2011). The secondary hypothesis was that more experienced clinicians would endorse more formulation matched interventions than the novice clinicians and endorse less the less pertinent or mismatched treatment options than novice clinicians. Abel. stressing the importance of independence and freedom of choice) and sociotropy (i. So whilst formulation is afforded a central role in CBT the evidence for its reliability. Padesky. first year trainee clinical psychologists). Ther. Finally.1 4. autonomy and sociotropy) and guide clinicians towards treatment focussed on the key dimension rather than less pertinent features. To examine these questions two related studies were conducted both of which used a novel paradigm using clinically relevant decision-making tasks with multiple sources of information. two levels. psychiatrists. less experienced clinicians may be more distracted by less pertinent. The primary hypothesis is that the content of a CBT case formulation would help the therapist rate as more appropriate formulation matching interventions rather than less pertinent or mismatched interventions.e.3 1. Rush. the results of assessment measures relating to emotional distress. 0% 0. The focus of the two studies reported here is whether formulation guides certain aspects of treatment decision making and planning. and Harrison (1983) identified two dimensions important in the development of depression: autonomy (i. 1 for an example) was based on the cognitive model of depression (Beck. a completed thought record and a completed activity schedule.4 14.4 6.g. Psychiat. / J. Turner. Dudley et al. 2005). 18. Williams.2 41. 2008. The demographic information for the participants is shown in Table 1. Hence. 1979) and featured a diagrammatical representation of the longitudinal (which incorporates information about early experience and how this predisposes a person to emotional problems owing to their core beliefs. then formulations that differ in key features should lead to different interventions (Butler. Information about the case was presented sequentially to the participant.2.5 2. sociotropy or autonomy) manipulation was used to test the main hypothesis that CBT case formulations have an effect on ratings of treatment options. Kendjelic. Therefore. 74% 0. & Emery.4 ¼ ¼ ¼ ¼ 17. The second was to consider whether greater training and experience leads to more effective use of CBT case formulations when making these ratings of treatment relevance. 1. The experienced group included 20 clinicians with extensive experience and training in CBT.e. The between-subject manipulation (experience. Measures A novel task was developed to assess use of CBT case formulations to plan treatment. rules and assumptions) and cross-sectional (also described Table 1 Demographic information (including professional experience.g. to examine the process of formulation development and the impact on treatment planning and decision making. rather than a pre-constructed one. 2006.1. % female) Diploma in CBT trained BABCP accredited Supervised on Diploma level CBT training N N N N N N N N Mean Sd Mean Sd Age (yrs) Further/higher education (yrs) Months of clinical experience Months qualified in profession 31. 2. a formulation of a person with a “sociotropic” depression would differ from that of someone with an “autonomous” depression. & Dudley. These studies have a number of methodological limitations one of which is that the quality of the formulation has generally not been evaluated (Mumma. Epstein. 2. The first study examined how clinicians utilised a pre-constructed CBT formulation to make treatment decisions. The first was to establish whether formulation guides the ratings of treatment interventions that do or do not fit the formulation. The novice group consisted of 23 clinicians who had an introductory training in CBT and a limited amount of practice in the use of CBT (e. nurse specialists in CBT). Owing to the complexity of linking any one aspect of therapy to outcome it may be more helpful to investigate the impact of formulation on an intermediate feature such as treatment planning (Eells et al. Both studies considered the effect of therapist experience. Gorg. Dudley.

/ J. forming the basis for maintenance cycles) systems of the CBT case formulation. & Exp. Dudley et al.68 R. one of the cases was derived from a published case study and served as a template for the second case that was based on the clinical work undertaken by the authors. 1995) case formulations (see Dudley & Kuyken. Kuyken et al. such as the Padesky five factor maintenance model..g. as a maintenance formulation which emphasises what factors lead to the perpetuation of distress. 48 (2015) 66e74 Fig. Information consistent with sociotropy or autonomy featured at the longitudinal (e. Greenberger & Padesky. . To establish the extent to which the formulation influences judgments of relevance of treatment options the two vignettes differed in content. Psychiat. Ther. 2008). in the early experience and core beliefs) and cross-sectional (e.g. 1996). three experienced CBT clinicians acted as an expert reference group and completed a rating sheet 1 The first case material was derived from a published chapter. parsimony and overall clinical credibility. The two cases (and their formulations) had to be equivalent in their accuracy. 1. Behav. The second vignette (Gerald) was based on the same conceptualisation model. then it would not be possible to demonstrate that different performance was not owing to some confounding variable. coherence. Jess's vignette was high autonomy and low sociotropy. but differed on a key theoretical dimension.1 Then. and Gerald's was high sociotropy and low autonomy. 2013. the cases were anonymised with identifiable information removed. The second was an amalgamation of several cases the authors had worked with. Hence. and or disguised to ensure that the person/people they were based on where not identifiable. To help ensure these criteria were met. CBT case conceptualisation of Jess's difficulties. If they differed except on the key dimension of sociotropy and autonomy. complexity. In addition. The first formulation vignette (Jess) was based on a published case of depression (Blackburn & Twaddle.

2 It was important that the treatment options really were regarded as matched. Hence. The treatment planning questions were taken from CBT manuals (e. Ther. Behav. For each of these questions. Each option was either a good match (pertinent).R. Each was rated on a scale from 1 to 10 (with 10 being high quality. where 0 ¼ doesn't fit and 10 ¼ definitely fits) how good a fit that option was to the CBT case formulation. The reviewers also completed a quality of case formulation measure (Kuyken et al. use of thought records (2). Dudley et al. mismatched or as irrelevant. behavioural experiments (2). Within each question for each of the options participants were asked to rate (on a scale of 0e10. relapse prevention planning (1) and identification of potential therapeutic barriers/problems (1). 2 for an example of a treatment planning question). The match treatment options were taken from the case study as published for Jess. 1995) and included problem list development (1). Consistent with the second hypothesis the experienced group was expected to endorse the match 2 Copies of all the materials including treatment options are available from the corresponding author on request.g. 2. goal setting (2). the case materials were considered to be equivalent. Participants had to judge how good a fit those options were with the formulations (see Fig. iii) how realistic the cases were.g. the same number of behavioural experiment. An example treatment planning question following from Jess's formulation. Psychiat.e. Two sets of CBT based treatment planning tasks (with 14 questions in each set) were developed that followed from Jess and Gerald's formulation vignettes respectively. The experts endorsed all the items above 7. that provided feedback on the consistency and coherency between and within vignettes. 2005) and both of the experimenter provided formulations were rated ‘good enough’ on the quality of case formulation measure. and for Gerald were based on a round of feedback and discussion with a group of experienced therapists before being presented to an expert in CBT. Beck. ii) the quality of the case conceptualisation. / J. role plays (2). & Exp. Then the materials were piloted with participants representative of the potential sample (i. continuum method (2).. The formulations were also endorsed as differing on the key dimension of sociotropy or autonomy. coherence etc. or cognitive restructuring interventions were presented). novice [n ¼ 3] and experienced [n ¼ 3] clinicians). three options were presented. activity scheduling (1). In keeping with the first hypothesis it was anticipated that all the participants would rate the match option higher than the mismatch and irrelevant options. . A further round of feedback from an expert clinician was then completed prior to full experimental testing. The experts were asked to assess each set of materials for i) the coherence of the assessment information and conceptualisation. mismatched (not pertinent) or a plausible but irrelevant option (a“red herring”). 48 (2015) 66e74 69 Fig. For each of the two vignettes (Gerald and Jessica) the questions were equivalent (e. and iv) the coherency of the treatment planning options in relation to the case.).

89) and mismatch (F(1.6 0.00. Deviation Mean Std. Within the above ANOVA. p ¼ 0. note that the match option was not endorsed at ceiling. The participants were told to imagine that they were the therapist and that they need to plan treatment based on the assessment information and the case conceptualisation. An example unrelated to either of the two subsequently presented cases was provided to help familiarise them with the task.3 2. The order of the treatment planning questions and options was also counterbalanced throughout. 2012).7 1.70 R.5. CI ¼ .7 7.3 1. and mismatched and red herring were not at floor. and may help account for the lack of differences between the experienced and novice clinicians.001 h2 ¼ .3 2.18). A number of potential limitations need to be considered when interpreting the findings. Dudley et al.8 1.3 2. h2 ¼ . Table 2 Mean ratings for response type across the two clinician experience levels and vignette. there was no difference between the novice and experienced practitioners. The analysis was also run with Order as a between subjects variable. Review procedures and ethics The work was subject to independent peer review. Planned simple contrasts showed that match responses were significantly higher in rated fit to the vignette and provided case formulations than both irrelevant (F(1.41) ¼ 221.62. at least in this type of task to plan treatment.0 7.40) ¼ 388. Ratings on the response to treatment planning tasks are outlined in Table 2 that shows the ratings across the two groups and by vignette. p < 0.8 0. Essentially. in this study clinicians rate predetermined treatment options from a number of possible options. .6 2. It was predicted that novice clinicians would more often endorse the red-herring/irrelevant and mismatch options. This may be owing to limitations within this study outlined below and that are addressed in the subsequent study. p < 0. which clinicians may develop.3 1. Previous studies have shown that more experienced and expert clinicians construct higher quality and more reliable CBT case formulations (Persons & Bertagnolli.41) ¼ 365. They were instructed they needed to decide how well the treatment options fit in relation to the case.93).2 4.0 1. As expected. Jess Novice Experienced Total Mean Std. 41) ¼ 1. First. . .9 0. / J.19).3 2. 4. 2 41) ¼ 1.2 1. Extreme responses were identified by boxplot and winsorised. Procedure Participants were recruited from local psychology and CBT training courses and were provided with an information sheet.19. Hence.7 1. There was also no significant interaction effect between experience and response type (F(2. CI ¼ . and signed a consent form. However. h2 ¼ . and ‘response type’ with three levels [match/irrelevant/mismatch]) examined differences between the ratings for responses to treatment planning tasks.5 4.2 1.00. and irrelevant responses were significantly higher than mismatch response types (F(1.9 4. & Exp. Deviation Gerald Match Irrelevant Mismatch Match Irrelevant Mismatch 7. Mauchley's test of sphericity was significant so the lower-bound Epsilon correction was used when determining F values.85. 2.85. Behav.00.01.001. Discussion This study considered whether a predetermined formulation guided treatment planning decisions and whether CBT experience effected these decisions.27.1 1. CI ¼ . This is consistent with the first hypothesis. After reading through the case information and formulations.88. 1999).93). prepared formulation and this may have reduced the potential for difference between the two groups. This supports previous case formulation research proposing a link between formulation and treatment plans (Butler.5 2. Data analysis was completed using SPSS 20 for Windows (SPSS.04.09). Perhaps knowledge of CBT techniques provided through basic CBT training is sufficient. the difference in experience may not be in the use of the formulation for treatment planning but in the development of the formulation.4 1.26. was registered with the Research and Development Department of the local NHS trust and received a favourable opinion from a Local NHS Research Ethics Committee.8 7. regardless of level of training.5 1. Deviation Mean Std.76. p < 0. The order of presentation was counterbalanced. p ¼ 0.001 h2 ¼ . A mixed ANOVA with a between subject variable (‘experience’ with two levels [novice/experienced) and two within subject variables (‘vignette type’ with two levels [Jess/Gerald].5 1.4 8. CI ¼ .001. so overall the treatment plans.84.90. this was a recognition task.83. If clinicians had a greater role in constructing a formulation then this may lead to differences in the formulations produced and increase the likelihood of detecting differences in the resultant treatment plans based on these differing formulations. CI ¼ .9 5. This study indicates that clinicians who are provided with a case formulation and a list of treatment planning options are able to choose options that fit with the formulation. . This is easier than when the clinician generates their own treatment option. p ¼ 0. 40) ¼ . 2. h2 ¼ .10.41) ¼ 516. Second. Results Preliminary analysis revealed no missing data. . Experience did not have an effect on treatment decision making hence hypothesis two was not supported. participants rated match responses significantly higher on fit to the formulation than the other options. participants worked through the treatment planning task.3 4. p < 0. 48 (2015) 66e74 option more than the novice group. the task involved using a constrained. would include a proportion of matched elements they would also include less than optimal elements. CI ¼ . Psychiat.35. The novice group was expected to less strongly endorse appropriate treatment options and more strongly endorse the red herring and irrelevant options. Each participant was presented with both of the case vignettes. 1998).7 1. There was a second hypothesis that clinician experience would effect how CBT case formulation was used to inform treatment decisions.9 7. There was a main effect for response type (F(2.90. Ther. . there was no effect for experience (F(1.74.7 0.03. . it had no bearing on the results and is not reported further. However. h2 ¼ .93. 3.4 4. However.95). CI ¼ . The main effect for vignette type across treatment planning options was not significant (F(1.5 0. h ¼ .2 .4.00.84.

Table 3 indicates that the expert group were very highly experienced (with a mean of 271 months of experience). it was hypothesised that expert formulations would contain more essential (and less extraneous) content from the vignette and thus. Often samples have been composed of experienced rather than ‘expert’ clinicians. Klein & Muntz. it was predicted that the formulations of experts would be of a higher quality than those of novices. ii) ‘process-tracing methods’ to measure processes underpinning performance.8 251.5 0 0 0 4.2. Kruger & Dunning. 50% 1.5 64. supervision of other CBT therapists. which may make the groups too similar to be contrasted (Skovholt. and iii) a detailed understanding of the individual's background to aid classification of expertise. the inclusion criteria used to distinguish between the novice and experienced groups may not have been sufficient to distinguish the two. Furthermore. 48 (2015) 66e74 Third. internal consistency or coherence of the formulation (which was scored out of a maximum of 11).. This three-stage descriptive and inductive framework is proposed to underpin the empirical analysis of expert performance. Hence. Whilst these participants were more experienced than the group of experienced therapists in study one they were less likely to have completed a diploma or equivalent in CBT. 94% 0. in this study an attempt was made to explore or trace the processes underpinning performance during each stage of the formulation generation (Bennett. 6% Mean Sd Mean Sd 27 28. they would be more parsimonious and more internally consistent than those of novice clinicians. and irrelevant options) than novice clinicians as it was assumed that the expert produced formulations were of a higher quality than the novices and would better direct treatment planning. it was predicted that experts would make fewer errors than novices. 13% 8. This change increased the potential differences between the participants. books. Four components were chosen as important measures for the current study namely. (2005) and Eells et al. it was hypothesised that experts would choose better treatment planning options (endorsing more the match option. and utilised Ericsson and Smith's (1991) Expert Performance Approach as a model to increase the difference between novices and experts.4 63. 1988). Following this framework led to three key changes to the task outlined in study one. Dudley et al. 0% 0.9 11. Einhorn & Hogarth.7 154. / J.1. A manual4 used to score the quality of the participants’ formulations was utilised in this study. Ronnestad. have been shown to perform faster. 1997). flexibility (number of changes made as new information was provided) and errors (number of mistakes in either providing material that experts had not seen as appropriate in the formulation or entering the correct information in the wrong section of the formulation). 1997). Study two addresses some of these possible limitations. Hence. 2008. Second. whilst making fewer errors. instead they were asked to generate their own which was recorded on to a blank formulation template. . 56% 2.3 6. 5. & Jennings. However. or evidence of commitment through delivery of training and continuing professional development (CPD) events on the topic of formulation (Skovholt et al. Ther. demonstrating frequent self-monitoring and adjustments of their work (Glaser & Chi.71 R. 2006. or to supervise on diploma level training but were just as likely to be BABCP accredited. 1997). arguing for i) ecologically valid tasks.7 12. and experienced but not expert clinicians (this data was not used within the main analyses as it was used for manual development) who undertook the same task as the novice and experts in this study. and rating as less relevant to the formulation the mismatched. % female) Diploma in CBT trained BABCP accredited Supervised on Diploma level CBT training Age (yrs) Months of clinical experience Months qualified in profession Research publications Number of workshops delivered Number of CBT cases seen Novice (n ¼ 31) Expert (n ¼ 16) N N N N N N N N ¼ ¼ ¼ ¼ 31. parsimony (correct items divided by total items entered). experts. Initially five of 6. 1981.3. 1979). 1999). These criticisms may apply to study one and may be a reason for a failure to detect difference between experienced and novice therapists. Previous research has indicated that novices are often unaware of their lack of skill and hence are overconfident in their estimations of task performance (Davis et al. Hence it was predicted that novices would report a higher level of confidence in key aspects of the formulation than the experts. therapists were asked to generate their own formulation when treatment planning. Design A mixed between groups (novice versus expert) with within subject (three levels treatment option) design was utilised.2 231. In other domains. 0% 0. participants were not provided with the provisional or final completed formulation. in terms of publications.. 6. One group consisted of 31 novice therapists all in their first year of clinical psychology training.1 2.7 17.9 19. Experts have been shown to generate better quality (Kuyken et al. 4 A copy of this is available from the corresponding author. Behav.2 18. 3 Further information about the expert participants is available on request from the corresponding author. Checkel. & Exp. evidence of continuous reflection regarding formulation and/or CBT techniques through research.7 0 0 0 4. and other esteem indicators the experts differed substantially.9 49 271. As with study one. (2005). 2010. Study two This study investigated the role of formulation in treatment planning and drew on and adapted the method in study one.. First. 2005) formulations than novices. However. Measures The assessment and formulation materials (Jess) developed in study one were utilised within study two. Gender (n. Specifically. Kleinmuntz. 2008.. as compared to novices. Psychiat.3 20. in order to increase the ecological validity. It was based on the manuals developed by Kuyken et al. in this study particular effort was made to recruit expert rather than experienced therapists. The quality manual was developed and tested on the formulations produced by a separate group of 30 novice. Einhorn. Participants Two groups (different to those in study one) were recruited. Third. the tasks used in the research may have been too simple to differentiate between groups (Skovholt et al.0 6. publications/conference appearances on related topics. Method Table 3 Demographic information for each group. 0% ¼ ¼ ¼ ¼ 16. They then used this to answer the treatment planning questions.. The second group consisted of 15 expert CBT therapists who met at least two of three criteria: a minimum of ten years of experience.

Ther.45.3%) than experts (18. changes made and legitimacy of changes were also recorded. They were invited by email to participate. The manual was used to score the expert and novice formulations by the same two raters who were blind to group membership with a similarly high rate of agreement between raters (Cohen's kappa greater than 0. Discrepancies were discussed and resolved and the manual was updated.8 0. Each piece of assessment material was colour coded and for each a corresponding coloured pen was provided. Cohen's d ¼ 0. Therapists were asked to follow the same procedure for each piece of assessment material. or perhaps placing trigger/precipitant information in the early experience part of the formulation).3% consistency in scoring. Eighty-two percent agreement was achieved.2) 91. / J.2) 79. 48 (2015) 66e74 these formulations were used to generate content examples for the manual. or maintained.61). However. including changing confidence ratings. Data was replaced with the mean of the person's rating on the other items.1.3e94.8 7. 14. the client's difficulties. Psychiat. & Exp. they imply that the experts were more cautious than the novices in the early stages of formulation. Process measures It was not possible to record how quickly each individual performed the task as some of the testing sessions were undertaken in groups. The formulation The task was structured to allow a formulation to be built up step-by-step from the sequential presentation of the five pieces of assessment material. Many of the errors made by the novices involved either placing incorrect material in the formulation (i. and a set of coloured pens.6) than novices (M ¼ 3.3 8. In order to explore the way in which judgments and decisions change as more information comes to light therapists were asked to neatly score through information they later believed to be irrelevant or erroneous.4. a blank formulation template based on the format in Fig.6) #0. Quality component Parsimony (%) Mean (SD) Confidence intervals Coherence/Consistencya Mean (SD) Confidence intervals Flexibility (changes made) Mean (sd) Confidence intervals Errors made Total a Novice (n ¼ 31) Expert (n ¼ 15) 81.9 (.9) 7. Individuals were asked to start with the referral letter and using the corresponding coloured pen note down emerging ideas about the case on the blank formulation template. and or were regular contributors to national and international conferences in CBT. .3% in stage two and the rest between three and five.2. The two researchers then scored the 15 remaining formulations and agreement increased to 86%. sd ¼ 0. Parametric assumptions were met.1. This would be consistent with the hypothesis that novices are relatively overconfident. The experts were more parsimonious in that they reported more correct information relative to the irrelevant information (t(44) ¼ #7. treatment planning questions. p < 0.0 (3. Formulation quality The Quality of formulations was scored using the manual. Dudley et al.3) 7.6.8 for experts and novice ratings and overall).01e0. p < 0.75) and fewer errors. Similarly 35.6e8.7% of experts identified it in stage one. Review procedures and ethics The work was subject to the same review processes as study one. a complete set of assessment materials from the case vignette from study one (referral letter.1e4. 6. Errors.1% by stage two. and the remaining 10. p < 0. Cohen's d ¼ 1. Treatment planning Once all five pieces of assessment material had been seen and the formulation template was completed participants were asked to consider the treatment planning questions (taken from study 1) in light of their own clinician-generated formulation. most experts reached the confidence threshold by stage two or three. assessment scores.01. 7. Treatment planning Preliminary analysis on the dataset identified four missing values within the treatment planning questions. Experts made fewer changes (t(44) ¼ 3.7 (1. relevant history.3. 1). and produced formulations that were more internally consistent and coherent (t(44) ¼ 3. means and standard deviations for parsimony and consistency are summarised in Table 4.4 (0.9e9. thought record. 6. 6. 1. Results 7.8) 2. M ¼ . 6.02. 7.5. core beliefs that were not related to the client formulation) or placing information in the wrong part of the formulation (so labelling core beliefs as rules or assumptions or as thoughts. 7. Experts were identified if they published on the topic of formulation.73) than the novices.72 R.1. with novices making 35 errors between them and experts making only two.7.8e83.005.4. This supports the hypothesis that experts would make fewer errors. Subsequently. and the stage at which they were at least 70% certain that this contributed to. Individuals were provided with the task instructions. a mixed ANOVA with a between subjects variable of Group (2 levels) and within subjects of Table 4 Parsimony and coherence of the expert and novice generated formulations. and activity schedule).01. Fifty-seven percent of novices identified the key dimension of autonomy by stage one.4 3. indicating the usability of the manual. four extra maintenance cycle templates (as in the lower part of Fig.9% between stages three and five. A further five formulations were scored in light of these revisions. Cohen's d ¼ 2.2%) were confident about the importance of the key theme by stage one. At this point the two researchers achieved 67. both were blind to the experience level of the participants. sd ¼ 3.7. Two members of the research team then independently scored a further five formulations. Behav.7 35 2 Coherence is scored out of a maximum of 11. Overall it appeared that the experts waited until certain and then did not need to revise their formulations. More novices (33.4 (3.6 93.7 (5. Confidence intervals. so a proxy measure of ‘speed of processing’ was to record the stage at which participants detected the key dimension within their formulation. None of these trends were significant.e. 32. Procedure The novices were recruited via Clinical Psychology training courses. Every time new information was added therapists were asked to rate their certainty (0e100% certainty) regarding the contribution of it to the emergence or maintenance of Jess's difficulties.

p < 0.25) was significantly higher than irrelevant ( 1. Cohen's d ¼ 1. Both groups identified the important features. Of course. As in the first study. However. but experts are less likely to rate or endorse less-relevant or less-appropriate interventions. sd ¼ 0. It is plausible that as individuals move towards expertise they become better at seeing the key themes. One plausible explanation (given the small number of errors made by experts) was that the task was not difficult enough to challenge experts.g. 48 (2015) 66e74 Table 5 Mean ratings for response type across the two clinician experience levels. these were either match.91. when participants were asked to generate their own formulations. experts were shown to make far fewer errors than novices. or indeed on ratings of appropriate treatment plans.43.07). irrelevant.22. It was further hypothesised that clinician experience would affect treatment planning choices.79e36. Dudley et al. Given the predicted differences in performance by the groups on the task these were explored as these pertained to the hypotheses. p < 0. 2005).9 6. This fits with Glaser and Chi's assertion that experts require more information than novices before making a decision. There was no significant interaction between the response patterns of the two groups (F (2. given the finding in study one that given a comprehensive formulation all groups can plan treatment equally well. experts have been shown to notice meaningful patterns more readily than novices (Glaser & Chi.73. In general the experts demonstrated a more cautious approach.27e23.001) and hence the Greenhouse-Geisser correction was applied. Deviation Match Irrelevant Mismatch 8. In this way the study moved towards providing a more comprehensive operationalisation of expertise. it did draw out a number of crucial differences between experts and novices in line with previous research (e. In study one a comprehensive formulation was provided and there were no differences in ratings of fit treatment options. sd ¼ 1.9 1.23) with the expert group (M ¼ 5. 42) ¼ 137.001. however when individuals were asked to generate their own formulation significant differences in treatment choices were evident between novice and expert clinicians. especially at the formulation stage is crucial. There was a main effect of group (F (1. h2 ¼ 0. p ¼ 0. h2 ¼ 0.001. An ANOVA comparing experts versus novice clinicians on response type (match. The importance of expert supervision therefore.9 1. Future research may consider manipulating the content of the cases so that there are simple and more complex presentations .61. Also. filtering out the less salient information when formulating. it was surprising that experts made fewer changes than novices when presented with new information. Expert clinicians included less inappropriate or superficial information. it is important to remember that both studies were essentially a recognition task which substantially reduces the ecological validity of the tasks.R.5 7.14e33. A second hypothesis was that novices would be overconfident in their formulations relative to experts. 9.6).28). but differ solely on the focus and clarity with which this is developed and used. sd ¼ 2. p ¼ 0.65). Butler (1998) proposed that the formulation provides a plausible explanation for a person's symptoms and is of central importance as it is thought to instil hope in the client as well as enhancing the alliance.9 2.4 2. may lead to better treatment plans. and choosing the ‘best’ of all available treatment options.42) endorsing all options less than the novices (M ¼ 6. and provided more internally consistent and coherent formulations than the novices.3 4. Cohen's d ¼ 1. Novices reported the key theme earlier and felt more confident in it earlier than expert therapists but not to a significant degree. p ¼ 0. CI ¼ 14. There was no significant difference between experts and novice clinicians' ratings on match questions (t(44) ¼ 1. irrelevant) demonstrated a significant main effect of response type (F (2. As predicted. Whilst these were planned analyses owing to the lack of interaction effect in the omnibus test we applied a Bonferroni correction and the latter two analyses remained significant.85. & Exp. 8.52).1. the experts rated the mismatch option as less relevant to the formulation than the novices (t(44) ¼ 4. Eells et al.19. it is important to note that experts and novices did not differ on the correct information within the formulation. 2005). Mauchly's test was violated (c2 (2) ¼21. Deviation Mean Std.92. it resulted not only in novices generating less parsimonious formulations than experts. Ther.42) ¼ 3.77. However. Ratings on overall response to therapeutic prediction tasks are outlined in Table 5. researching in the area) and differed from novices on a range of key variables taken to be evidence of expertise (such as years of experience.76). General discussion Study one demonstrated that providing a comprehensive formulation enabled clinicians of all levels to make similar treatment decisions.2 response type (3 levels) was undertaken with planned comparisons to compare responses to the treatment planning tasks between the groups. sd ¼ 0.27. 2003. This indicates that all clinicians were able to identify important aspects of a client's presentation on the case formulation generation task as was used here. 43) ¼ 12. In study two. Psychiat.45) and mismatch (3.61. these provide the summed responses from the task. sd ¼ 1.5 1. but also negatively impacted upon the appropriateness of their treatment choices. however.9 1. Novice Expert Mean Std.. Experts rated the irrelevant or red herring treatment planning questions as a lower fit to the formulation than the novices (t(44) ¼ 4. or patterns. Whilst the task may have been too easy to challenge experts.52. and provide opportunities for intervention. evidence of reflection and/or evidence of commitment to Continuing Professional Development). p < 0. The experts were selected based on a broad range of criteria shown to be fundamental to the development of expertise (such as involvement in supervising others. A poorer quality formulation as generated by the novice clinicians is less likely to fulfil these aims and may even lead to therapeutic ruptures.05. It seems therefore that both experts and novices can identify 73 important treatment options. especially during the formulation stage. This has important clinical implications. CI ¼ 12. Previous studies have also suggested that experience and expertise leads to differences in treatment planning from a formulation (Eells & Lombart. or mismatch. in turn becoming more efficient and hopefully effective. receipt of expert supervision. Cohen's d ¼ 0. based on Glaser and Chi's (1988) assertion that experts are more able to self-monitor their performance and adapt accordingly. Experts rated inappropriate treatment plans as less relevant than novices. Eells et al.001. Certainly. mismatch. p < 0.46. / J. CI ¼ #3. 1988). and that irrelevant/red herring was significantly higher than mismatch (all p values <. although there was no significant difference in ratings of the matched treatment options.001). Behav. although it was approaching a significant value. Discussion As predicted Experts generated higher quality formulations than novices..6 4.43). h2 ¼ 0. the experts seemed to be able to do so more elegantly. Planned simple contrasts showed that match (M ¼ 8.

& Hogarth. & Padesky. (1983). Cambridge: Cambridge University Press. & Exp. & Kuyken. The science and practice of case conceptualisation.. Tailor-made versus standardized therapy of phobic patients.. McManus. Impact of case formulation in cognitive behaviour therapy for psychosis. A. W. Fordis. 52e69. Working effectively with clients in cognitive behavioural therapy. International Studies Review. R. Blackburn. (2009). published online 28/01/15. 1e16. (2008). & Dudley. Padesky. 187e204. & Padesky. Persons. Kuyken. / J. Johnstone. P. & M. Inter-rater reliability of cognitive-behavioral case formulations of depression: a replication. Beshai.. Is cognitive case formulation science or science fiction? Clinical Psychology-Science and Practice. 31(4). In A.. & Bertagnolli. The nature of expertise. Glaser. L.... & Chadwick. R. R. Brunner Routledge. 185e200. P. (1981).. 296(9). C.. & Smith. European Journal of Psychological Assessment. Psychotherapy. E. Ronnestad. Collaborative case conceptualisation. T. Kuyken. 465e485. 48 (2015) 66e74 (Dudley.. Dudley. 66e77. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. attitudes and personality dimensions in depression. S. C. (2010). 1e24). (2008). 44. J. Turner.. A. 213e224. Kuyken. A. Disorder specific and transdiagnostic case conceptualisation. J. Brady. K. 579e589. C. . Dudley. 53e88. A. P. Stephen Barton. Lanham: Rowman and Littlefield Publishers Inc. T. Padesky. Rate of agreement between clinicians on the content of a cognitive formulation of delusional beliefs: the effect of qualifications and experience. (1997)... Linear regression and process-tracing models of judgment. K.. (2008). 271e283. In M. 362e370. Behaviour Research and Therapy.. References Beck. (2006). & Perrier. 44. (1992).).). C. 361e369. (2013).). E.. W. 38. T. E. Prospects and limits in the empirical study of expertise: an introduction. H. T. & Dudley.. Mazmanian. Dudley. A. N. Assessing competence in collaborative case conceptualization. experienced and novice cognitive-behavioural and psychodynamic therapists. Ther. Musa. & D. (2003). R. N. & Kuyken. Einhorn. Checkel. J. Acknowledgements We would like to thank Peter Armstrong. (1995). C. & Twaddle. Hillsdale. & Chi. G. In J. D. (1996). Psychotherapy Research.. J. H. J.). Dudley et al. Williams. M. Brady. Behavioral decision Theory: processes of judgment and choice.). Mumma. The Oxford Handbook of Political Methodology (pp. Cognitive Therapy and Research. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence Lead to inflated self-assessments. Pepping. 1027/1015-5759/a000054. Statistical Packages for Social Sciences (SPSS). & Mackenzie. J. (2005). (pp. Generic psychotherapy case formulation training improves formulation quality. 27(1). New York: Pergammon Press. 32. Chicago: SPSS inc.. 671e680. 1121e1134. Dallos (Eds. & Padesky. Ivy Blackburn and Willem Kuyken who kindly acted as the expert reviewers of the materials used in these studies. (2012). R. In H. B. Psychiat.. & Padesky. C. H. Rush. G. J. Psychology Review. & Muntz... psychotherapy and professional psychology. I.. Eells. (1979). 1187e1201. E. D. (1995).74 R.0 for Windows. Schulte. M. G. & Dodgson. B.. Butler. (1999). Beck. P. Comprehensive clinical psychology (pp. & Dunning. & Jennings.. 43. (1998). Epstein. Tracing Causal Mechanisms. Dudley. Validity issues in cognitive-behavioral case formulation. R. Skovholt. T. Kuyken. M. Bennett. Clinical Psychology Review.. Collier (Eds. (2011). Behavioural and Cognitive Psychotherapy. Cognitive Therapy and Research.. (1991). Farr (Eds. Fothergill. E. James. T. Behavioural and Cognitive Psychotherapy.doi.. Glaser. 8(2). The quality of case formulations: a comparison of expert. (1979). Chi. Process-tracing: a Bayesian Perspective. 36(Special Issue 06).). 10.).. R.. Comparing the validity of alternative cognitive case formulations: a latent variable. Kruger. R. G. 29e49. & Mooney. Mumma. & Lombart. Journal of Personality and Social Psychology. P... Kuyken. Behavioral and Cognitive Psychotherapy. and expert cognitive-behavioural and psychodynamic therapists. using think out aloud methods may provide a means of understanding the process of formulation rather than just seeing the outcome of the formulation process. Abel. H. Educational Psychology Review. I. H.. (2011). (2007). Journal of the American Medical Association. J.. New York: Guilford Press. (2010). A. Collier (Eds. Gower. Davis.: development and preliminary psychometric properties of the collaborative case conceptualization rating scale (CCC-RS). D. In KA. L. & M. Case formulation in cognitive behavioural therapy: a principle driven approach. Mind over mood: Change how you feel by changing the way you think. W.. R. 451e481. G. & Shulte-Bahrenberg. Does individualization matter? A randomized trial of standardized (focused) versus individualized (broad) cognitive behavior therapy for bulimia nervosa.. multivariate time series approach. T. (2007). 757e768. Van Harrison. V. Thorpe. Cognitions. A. D. G. 1. Greenberger. M. Overview. L. Such material may challenge novice clinicians and better reveal the value that expertise brings to the process of formulation. Cognitive therapy: Basics and beyond. Kleinmuntz. E. Bellack. Cognitive therapy of depression. E. D. W.. 41. Kuyken. Kendjelic. J. Behaviour Research and Therapy. Clinical formulation. Bieling. K. 67e92. Beck. 207e220). Behav. P.. 23. 9. & Emery. Box-Steffensmeier. A.. (2005). K. A... (2003). H. C. NJ: Lawrence Erlbaum. Kendjelic. SPSS 20. 77(6). & Smith (Eds. F. & D. W. & Harrison. Finally. S. (2003). Bennett. B. D. Gorg.. Einhorn. 31. The reliability and quality of cognitive case formulation. Advances in Behaviour Research and Therapy.). R. & Lucas. Rethinking Social Inquiry: Diverse Tools Shared standards (2nd ed. Ericsson. (1988).. G. A. Klein. Toward a general Theory of Expertise: Prospects and limits (pp. R. & R. J. Chadwick. New York: Guilford. (1999). M.. Journal of Consulting and Clinical Psychology. 14. W. S. C. Ericsson. M.. 2011). N. T.. D. In L. I. R. Formulation in psychology and psychotherapy (2nd ed. 702e721). Shaw. W. A. Case formulation and treatment concepts among novice. R. Annual Review of Psychology. http://dx. K. F. British Journal of Cognitive Psychotherapy. Behaviour Research and Therapy. 13. 273e288. New York: Guilford Press. Hersen (Eds. M. M.. Oxford: Oxford University Press. (2006).. 1e38).. Cognitive therapy in action: A practitioners' case book... A. The search for expertise in counselling.. Eells. 73(4). 86. Lombart. J. Process tracing and causal inference. Park. Kunzel. Ghaderi. D.. G. C. 1094e1102. J. Guildford Press. D. J. experienced. & Eells. T.