PALLIATIVE

Original
Original Article
Article MEDICINE
Palliative Medicine
25(8)
0(00) 748–757
1–10

Using mixed methods to develop and ©

!TheTheAuthor(s)
Reprints
Author(s)2011
Reprints and
2011
and permission:
permissions:
sagepub.co.uk/journalsPermissions.nav
evaluate complex interventions in sagepub.co.uk/journalsPermissions.nav
DOI:
DOI: 10.1177/0269216311417919
10.1177/0269216311417919
pmj.sagepub.com
pmj.sagepub.com
palliative care research

Morag C Farquhar General Practice and Primary Care Research Unit, Department of Public Health & Primary Care, University of Cambridge,
Institute of Public Health, UK
Gail Ewing Centre for Family Research, University of Cambridge, UK
Sara Booth Palliative Care Team, Addenbrooke’s Hospital, Cambridge University Hospitals’ NHS Foundation Trust, UK

Abstract
Background: there is increasing interest in combining qualitative and quantitative research methods to provide com-
prehensiveness and greater knowledge yield. Mixed methods are valuable in the development and evaluation of complex
interventions. They are therefore particularly valuable in palliative care research where the majority of interventions are
complex, and the identification of outcomes particularly challenging.
Aims: this paper aims to introduce the role of mixed methods in the development and evaluation of complex inter-
ventions in palliative care, and how they may be used in palliative care research.
Content: the paper defines mixed methods and outlines why and how mixed methods are used to develop and evaluate
complex interventions, with a pragmatic focus on design and data collection issues and data analysis. Useful texts are
signposted and illustrative examples provided of mixed method studies in palliative care, including a detailed worked
example of the development and evaluation of a complex intervention in palliative care for breathlessness. Key challenges
to conducting mixed methods in palliative care research are identified in relation to data collection, data integration in
analysis, costs and dissemination and how these might be addressed.
Conclusions: the development and evaluation of complex interventions in palliative care benefit from the application of
mixed methods. Mixed methods enable better understanding of whether and how an intervention works (or does not
work) and inform the design of subsequent studies. However, they can be challenging: mixed method studies in palliative
care will benefit from working with agreed protocols, multidisciplinary teams and engaging staff with appropriate skill
sets.

Keywords
Breathlessness, complex interventions, methodology, mixed methods, randomized controlled trials

Introduction ask and answer questions that are more important to

There is increasing interest in combining qualitative
and quantitative approaches to health and social
research. Traditionally, different stakeholders have
valued different types of outcomes and therefore,
implicitly, different research methods. Commissioners
have favoured quantitative outcomes, in particular
cost effectiveness, whereas qualitative methods may
complex intervention providers and users (patients,
carers and referrers). Further, there is increasing inter-
est in the patient experience as an outcome at the policy
level (see, for example, the Secretary of State for
Health’s June 2010 speech ‘My ambition for patient-
centred care’1). In palliative care, patient experience is
often the primary outcome.

Corresponding author:
Morag C Farquhar, General Practice and Primary Care Research Unit, Department of Public Health & Primary Care, University of Cambridge, Institute
of Public Health, Robinson Way, Cambridge CB2 0SR, UK
Email: mcf22@medschl.cam.ac.uk

Downloaded from pmj.sagepub.com at Univ of Connecticut / Health Center / Library on June 20, 2015
Farquhar
2 et al. 749
Mixed method research brings together quantitative
and qualitative research methods from the different
research paradigms of positivism and interpretivism.
Combining quantitative and qualitative methods in a
mixed method approach provides comprehensiveness2
and greater knowledge yield: ‘a whole greater than the
sum of the parts’.3 This paper focuses on mixed method
approaches to developing and evaluating complex
interventions in palliative care, illustrating these
through a description of the purpose, design and
added value of mixed methods identified in examples
from the palliative care research literature and a more
detailed worked example of the development and eval-
uation of a palliative care complex intervention for
breathlessness.
What is ‘mixed methods’?
Amongst the many definitions of mixed methods, some
are particularly suited to the development and evalua-
tion of complex interventions, that is, those capturing
both individual studies and linked programmes of stud-
ies. For example, Tashakkori and Teddlie4 defined
mixed method studies as combining ‘qualitative and
quantitative approaches into the research methodology
of a single study or multi-phased study’ and Creswell5
defines the approach as one in which the researcher
collects, analyses and integrates both quantitative and
qualitative data in a single study or in multiple studies
in a sustained programme of inquiry. Thus mixed
methodology sits well within the multi-phase model
of complex intervention development and evaluation
as advocated by the Medical Research Council
(MRC) framework.6–8 Indeed, the most recent update
on the framework states that ‘wherever possible evi-
dence should be combined from a variety of sources
that do not share the same weaknesses’.8
Many resources exist in the growing number of
mixed method textbooks (recent examples: Creswell
and Plano Clark,9 Greene,10 Johnson and
Christensen,11 Ridenour and Newman,12 Andrew and
Halcombe,13 Teddlie and Tashakkori,14 Onwuegbuzie
et al15 and Tashakkori and Teddlie16), overview papers
(examples: Sandelowski,17 Brannen,18 Bryman,19
Mason,20 Moran-Ellis et al.,21 Bryman,22 O’Cathain
et al.23 and Östlund et al.24), and dedicated journals,
for example, the Journal of Mixed Methods Research
and the International Journal of Multiple Research
Approaches.
Why should we use mixed methods with
complex interventions?
Table 1 outlines Greene’s five purposes of mixing
methods.10 It illustrates different ways in which
Downloaded from pmj.sagepub.com at Univ of Connecticut / Health Center / Library on June 20, 2015
Palliative Medicine 0(00)
mixed methods can contribute to the development
and evaluation of complex interventions. Insights
gained from quantitative and qualitative approaches
complement each other to provide more in-depth
understanding. This deeper understanding can inform
the process of evaluation, the potential range of out-
comes and facilitate replication of the intervention
through greater knowledge of the active component(s)
and potential barriers to implementation. Thus,
throughout the process of the development and evalu-
ation of a complex intervention, each approach can
contribute to the description of the context and the
justification for the intervention, as well as its design,
testing, remodelling and formal evaluation. Mixed
methods help us answer whether an intervention
works (or not) and why (or not), and also its fidelity
(how it is being delivered).
Mixed methods are particularly valuable in palliative
care research, where the majority of interventions are
complex, and the process of evaluation and identifica-
tion of suitable outcomes is particularly challeng-
ing.25,26 Qualitative data tells us about subjective
experiences, understandings, effects and impacts; thus,
it can augment quantitative data in hypothesis genera-
tion, defining interventions, questionnaire or instru-
ment design (e.g. by informing content and cognitive
piloting), intervention fidelity testing and by illuminat-
ing explanations for associations found within quanti-
tative analysis (or not).
Table 1. Five purposes of mixing methods (Greene10)
1) Triangulation Where different methods used to measure
the same phenomenon, to increase
confidence in conclusions reached – if
consistent or convergent conclusions
are reached.
2) Complementarity Where methods are used to investigate
different aspects or dimensions of the
same phenomena to deepen or broaden
the interpretations and conclusions
from a study.
3) Development Where results from one method are used
to inform the development of other
method e.g. instrument development,
but also sampling and implementation.
4) Initiation Where different methods are used to
investigate different aspects or dimen-
sions of same phenomenon but, in
contrast to complementarity, the inten-
tion is divergence in order to generate
new understandings.
5) Expansion Where different methods are used to
assess different phenomena to expand
the scope and range of study.
750 et al.
Farquhar
How should we use mixed methods with
complex interventions?
A number of typologies, matrices and frameworks
exist for various aspects of the design and execution
of mixed method studies, for example, in relation to
rationale for mixing methods, sampling schemes,
study design and validation of inferences. Describing
these is beyond the scope of this paper: Collins and
O’Cathain27 provide a useful summary. We will focus
instead on pragmatic issues of design, data collection
and analysis.
Design and data collection
Most commonly, mixed method research takes a prag-
matic approach with designs driven by the nature and
context of research questions. Table 2 outlines three
potential designs for mixing methods within studies,
or within programmes of studies: sequential, concur-
rent or truly integrated.
Quantitative and qualitative approaches taken will
vary according to the research question. Chosen
approaches require justification, just as they do for
single-method studies. For example, sampling decisions
should relate to both the quantitative and qualitative
components. For the quantitative component of a ran-
domized controlled trial (RCT), power analyses and
sample size justification remain paramount, just as jus-
tification for the likely participant numbers to achieve
data saturation remains essential for qualitative com-
ponents of that trial.
Table 2. Potential designs for mixing methods
Designs Within study
Sequential Methods occur one after the other within a study
e.g. quantitative then qualitative, or qualitative
then quantitative.
With the same sample, sub-samples or separate
samples.
Concurrent (or parallel) Qualitative and quantitative methods run along-
side one another, in tandem, but may not be truly
integrated until analysis.
Integrated Quantitative and qualitative methods occur
together e.g. qualitative and quantitative data
collection may occur concurrently with the
same research subjects within the same
interview.
Within each of these qualitative and quantitative aspects may have equal or different weights or dominance, and this dominance or
equality may also vary at different stages of the research process e.g. quantitative data collection can sometimes be lengthy to
collect, but analysis can be quick, whereas the opposite can be true of qualitative data.
Downloaded from pmj.sagepub.com at Univ of Connecticut / Health Center / Library on June 20, 2015
Palliative Medicine 25(8)3
Qualitative methods most commonly ‘mixed’ with
quantitative include observation, case study methodol-
ogy, qualitative interviews and focus groups, for exam-
ple, a qualitative topic guide included with quantitative
instruments (interviewer or self-administered) in an
interview schedule, or conducted at separate times
depending on schedule length and content. Multiple-
perspective studies sometimes used to evaluate complex
interventions might use this integrated method with
some respondent groups (e.g. patients and carers), but
other methods (e.g. qualitative topic-guided interviews)
with other respondent groups (e.g. referrers or interven-
tion providers) due to the different research questions
being addressed with each group. Choice of method
should be driven not only by the research question
but also by the respondents targeted, for example,
Ewing et al.28 found short interviews focused on four
or five key issues to be highly successful in a study
involving consultants in secondary care.
Data analysis
Mixed method analysis requires use of qualitative and
quantitative analytic techniques either concurrently (at
the same time or relatively close in time) or sequentially
(one type of analysis conducted first, informing the
other).14,29 Onwuegbuzie and Teddlie29 suggest using
a combination of the seven phases of the mixed
method analysis process summarized in Table 3.
Where both quantitative and qualitative methods
have been used to collect data from the same sample,
a further approach is to use quantitative attributes to
Within a programme of studies
Sequential set of linked studies e.g. a quantitative
then qualitative study, or qualitative then
quantitative study.
Component studies may appear to standalone
but their findings inform one another.
Qualitative and quantitative studies run alongside
one another, in tandem, in a programme but
may not be truly integrated until analysis.
Less common.
Quantitative and qualitative studies occur
together.
Less common.
4Farquhar et al. Palliative Medicine 0(00)
751

Table 3. Phases of the mixed method analysis process programme and within some of the component studies
(Onwuegbuzie and Teddlie29) of the programme.
Table 5 outlines the programme of work and sum-
1) Data reduction e.g. quantitative data analysed using
descriptive statistics and qualitative
marizes for each phase, and for the programme as a
data are categorized as descriptive whole, the purpose of mixing methods (in relation to
themes Greene’s purposes,10 as Table 1), the mixed method
2) Data display e.g. data from both sources are orga- design used (according to Table 2) and the added
nized and presented visually in value of mixed methods.
graphs and matrices At the preclinical phase, qualitative interviews with
3) Data transformation Quantitative data are converted or lung cancer and chronic obstructive pulmonary disease
‘qualitized’ into narrative codes that (COPD) patients and their carers provided the evidence
can be analysed using qualitative base for the need for an intervention, its role and the
techniques and qualitative data are way it should work, for example, despite being a hos-
converted or ‘quantitized’ into pital outpatient service it should be community based,
numerical codes and analysed using work holistically to a palliative care model and
quantitative techniques address carer, as well as patient, need.34 Thus it
4) Data correlation Correlating quantitative data with informed the development of the pilot intervention.
qualitative data, or vice versa The resulting pilot BIS aimed to help people live with
5) Data consolidation Different data types merged into one breathlessness from any diagnostic cause (malignant or
dataset non-malignant), using a toolkit of pharmacological and
6) Data comparison Comparing data from two different non-pharmacological evidence-based interventions, by
sources means of a flexible, multidisciplinary hospital palliative
7) Data integration Integrating quantitative and qualitative care outpatient service that functioned in the commu-
data into one coherent whole that nity (home visits).35
will be analysed and interpreted At Phase I qualitative interviews were conducted
simultaneously as a single dataset or
with patients who had used the pilot service, their
two datasets (quantitative and quali-
tative) to be analysed separately
carers, referrers and service providers. These multiple-
perspective interviews focused on the experience of
breathlessness, what was liked about the service,
identify cases for detailed qualitative analysis through whether it worked, how it might work and what could
purposive sampling of the qualitative dataset. The be improved.36 The qualitative results were fed-back to
increasing sophistication of qualitative data analysis the service leading directly to refinement of the complex
software packages can facilitate integration with quan- intervention. In addition, qualitative data informed the
titative data. choice of primary outcome for a Phase II pilot RCT of
Table 430–33 outlines four examples of mixed method the service: patient distress due to breathlessness.
studies in palliative care and summarizes for each the Phase II combined quantitative and qualitative
purpose of mixing methods (in relation to Greene’s pur- methodologies in a mixed method pilot pragmatic
poses,10 see Table 1), the mixed method designs used (in single-blind fast-track RCT of the re-developed BIS
relation to Table 2) and the added value of mixed meth- versus standard care for patients with COPD. Given
ods. Below we use a particular example of the develop- the known difficulties of conducting RCTs in palliative
ment and evaluation of a complex intervention in care,37,38 this mixed method study sought to test the
palliative care to illustrate in more detail the added feasibility of the proposed mixed method trial.
value of mixing methods. The mixed method pilot trial integrated qualitative
topic-guided interviews with quantitative outcome
Added value of mixed methods with measures for patients and carers within an interview
complex interventions: the example of schedule and conducted concurrent qualitative inter-
views with referrers to, and providers of, the interven-
the Breathlessness Intervention Service tion. The mixed method feasibility study used
Mixed methods have been, and continue to be, used qualitative data (interviews with intervention users
throughout a programme of work developing and eval- and providers and fieldnotes) to evaluate acceptability,
uating a Breathlessness Intervention Service (BIS) for and quantitative data (response rates, protocol comple-
patients with advanced disease. The programme follows tion rates, un-blinding rates, missing data rates) to
the MRC framework for the development and evalua- evaluate feasibility. Thus mixed methods showed that
tion of complex interventions.6–8 Methods have been fast-track trial methodology was acceptable to COPD
mixed both across component studies of the patients, carers, referrers and providers, and that trial

Downloaded from pmj.sagepub.com at Univ of Connecticut / Health Center / Library on June 20, 2015
752 et al.
Farquhar
Table 4. Examples of mixed method studies in palliative care: purpose, design and added value
Example Outline (extracted
papers from the paper)
Sampson Mixed method approach to
et al., the development and eval-
200830 uation of a complex inter-
vention for patients with
advanced dementia and
their carers.
Seers Uses a mixed method ques-
et al., tionnaire in 3 centres with
200931 782 participants to
describe and measure
individualized experiences
of integrative cancer sup-
port care (complementary
therapies and biomedical
treatment).
Shipman Mixed method evaluation of
et al., the impact of a national
200832 palliative care education.
and support programme
on the knowledge and
confidence of members of
district nursing teams,
using before-and-after
postal questionnaires (tar-
geting n ¼ 1280), qualita-
tive interviews (n ¼ 39)
and one focus group.
Vale- Mixed method study to
Taylor, explore which post-funeral
200933 remembrance activities are
most significant and
important to bereaved
people living with loss, and
why they are chosen. Used
self-report questionnaire
(n ¼ 25) and semi-struc-
tured interviews (n ¼ 18)
with the bereaved, and
focus group with bereave-
ment counsellors.
a
Interpreted from the original paper in relation to Table 1 (Greene10).
b
Interpreted from the original paper in relation to Table 2.
c
Extracted from the original paper.
MRC: Medical Research Council, RCT: randomized controlled trial.
Downloaded from pmj.sagepub.com at Univ of Connecticut / Health Center / Library on June 20, 2015
Palliative Medicine 25(8)
5
Purpose of
mixing methodsa Mixed method designb Added value of mixed methodsc
Triangulation Sequential. The authors state that qualitative
Complementarity Programme-wide mixed phase was vital in investigating
Development methods following and teasing out the important
Initiation MRC framework: ingredients of an intervention.
Expansion Phase I standalone quali- Highlighted the challenges to
tative study and improving end of life for this
planned Phase II group and lack of understand-
exploratory trial. ing of the natural history of
Unclear if Phases II–III will dementia and its progression.
include qualitative. Pilot RCT will identify out-
come measures and generate
data for a power calculation
for Phase III cluster RCT.
Triangulation Integrated The authors state that it was
Complementarity unlikely that meaningful eval-
Expansion uations of this complex inter-
(Initiation?) vention could be carried out
by quantitative methods
alone.
Triangulation Sequential The authors state that the qual-
Complementarity itative elements of the evalu-
Expansion ation were essential to
(Initiation?) provide comparative informa-
tion as well as a context for
the findings from the quanti-
tative, as there was no feasible
alternative to a ‘before-and-
after’ design.
Triangulation Concurrent The authors state that the focus
Complementarity group added further contex-
Expansion tual data to support the study,
(Initiation?) and mixed methods (the
inclusion of a postal ques-
tionnaire) enabled the inclu-
sion of participants who might
not wish to meet face-to-face
to discuss their experiences.
Different methods provided
different information: the
postal questionnaire collected
data on ‘what’ participants did
whereas the interviews
explored ‘why’.
Farquhar
6 et al. 753
Palliative Medicine 0(00)

Table 5. Use of mixed methods in the development and evaluation of the Breathlessness Intervention Service (BIS): purpose, design
and added value

Phase of
the MRC Purpose of Mixed method
framework Outline mixing methodsa designb Added value of mixed methods

Preclinical Standalone qualitative study of n/a
(Booth et al., lung cancer and chronic
200334) obstructive pulmonary dis-
ease (COPD) patients’ expe-
rience of breathlessness
Phase I (Booth Standalone qualitative study of n/a
et al., 200636) users’ experiences of the pilot
service
Phase IIc Mixed method pilot pragmatic Triangulation, com-
(Farquhar single-blind fast-track ran-
et al., 200939; domized controlled trial of
Farquhar the re-developed BIS -v-
et al., 201040; standard care for patients
Farquhar with COPD, plus concurrent
et al., 201041) qualitative study
Phase IIIc Mixed method pragmatic single- Triangulation, com-
(Farquhar blind fast-track randomized
et al., 201142) controlled trial of BIS -v-
standard care for patients
with any diagnosis, plus con-
current qualitative study
Phase IV May use: case study methodol-
ogy within a mixed method
framework
Whole Mixed method programme fol-
programme lowing the MRC framework
for the development and
evaluation of complex
interventions
a
In relation to Table 1 (Greene10).
b
In relation to Table 2.
c
Trial registration: Phase II NCT00711438; Phase III NCT00678405 / ISRCTN04119516.
RCT: randomized controlled trial, MRC: Medical Research Council.
Sequential (standa- Provided evidence-base for:
lone qualitative - need for the intervention
study) - role of the intervention
- way it should work
Thus informed the development of
the pilot intervention.
Sequential (standa- Provided evidence-base for:
lone qualitative - refinement of the complex
study) intervention
- choice of primary outcome mea-
sure for Phase II pilot RCT
Mixed method Provided evidence-base that:
plementarity, RCT: integrated. - fast-track trial methodology was
development, Qualitative study: acceptable
initiation, concurrent. - trial procedures worked
expansion - some outcome measures
unsuitable
Thus informed the design of Phase
III.
Provided evidence-base for:
- further refinement of the complex
intervention
Mixed method Will identify:
plementarity, RCT: integrated. - effectiveness of intervention
development, Qualitative study: - reasons for success / failure of
initiation, concurrent. intervention
expansion - capture the changing context of
the intervention
May:
- identify reasons for variation in
effects across individuals
- facilitate intervention targeting
May be: triangula- May be mixed May identify:
tion, comple- method case - variation in effects across sites
mentarity, devel- study: integrated - reasons for success/failure of
opment, initia- intervention in different contexts
tion & expansion - barriers and facilitators to
sustainability
Triangulation, com- Sequential, concur- Will:
plementarity, rent & - facilitate the development and roll
development, integrated out of an evidence-based inter-
initiation & vention that is effective, feasible
expansion and acceptable in a variety of
contexts
- capture the changing context the
intervention was developed in

Downloaded from pmj.sagepub.com at Univ of Connecticut / Health Center / Library on June 20, 2015
754 et al.
Farquhar
procedures worked (e.g. patient identification, recruit-
ment, randomization, single blinding), but that some of
the outcome measures were unsuitable.39
In this mixed method approach all interview data
collection was audio-recorded, not just the qualitative
data, thus it was possible to conduct a qualitative anal-
ysis of the process of administration of quantitative
outcome measures as part of the feasibility analysis.
This led to the rejection of one of our outcome mea-
sures on the evidence base that aspects of the process of
administration questioned the validity of the quantita-
tive scores attained with this patient group.40 This sys-
tematic analysis would not have been possible without
a mixed method approach: without it, the process of
administering the instrument is unlikely to have been
audio-recorded. Feasibility requires a mixed method
approach: it is not just about whether instruments can
be completed and missing data avoided, but how those
instruments are completed – the validity of the data
obtained.
In addition, the qualitative data enabled further
refinement of the intervention through feedback of
mixed method findings.41 It also established that the
length of the intervention could be reduced – an impor-
tant finding for a palliative care intervention. As a result,
the length of the trial protocol could also be reduced – an
important finding for a palliative care RCT, particularly
one using fast-track trial methodology.
Phase III is currently underway: a mixed method
pragmatic single-blind fast-track RCT of BIS versus
standard care for patients with any diagnosis.42 Its
design was informed by the Phase II mixed method
findings in terms of the service design, RCT design
and outcome measures. It consists of three components:
(1) a mixed method RCT integrating qualitative topic-
guided interview with quantitative outcome measures
for patients and carers; (2) a concurrent qualitative
interview study with referrers to the service; and
(3) sequential qualitative interviews with service pro-
viders. As well as describing the effectiveness of the
intervention, the mixed method data should enable
analyses to identify reasons for the success or failure
of the intervention, and reasons for variation in effects
across individuals (which could facilitate intervention
targeting).
In the future, Phase IV may use case study method-
ology within a mixed method framework, for example,
observation, mixed method interviews (qualitative topic
guides with quantitative outcome measures) with
patients and carers and qualitative interviews or focus
groups with providers and referrers. Thus, a mixed
method approach has the potential to identify variation
in effects across sites, reasons for success or failure of
the intervention in different contexts and barriers and
facilitators to sustainability.
Downloaded from pmj.sagepub.com at Univ of Connecticut / Health Center / Library on June 20, 2015
7
Palliative Medicine 25(8)
Challenges in mixed method research
with complex interventions
Mixing methods bears an inherent philosophical chal-
lenge: the bringing together of methods from different
paradigms has generated much debate. Purists take the
view that combining or mixing paradigms is wrong due
to philosophical incompatibilities and differing assump-
tions; others take a more pragmatic approach, focusing
on the research question and how this drives the choice
of method (see Greene10 for discussion of the debate in
some detail). Here we shall focus on four practical chal-
lenges unique to the implementation of mixed method
studies: data collection, data integration in analysis,
costs and dissemination.
Data collection
The challenge of data collection in mixed methods
relates to the unique skills required for each method,
and the need for flexibility within a mixed method
framework. The tendency in health services research
for data collection to be delegated to junior researchers
needs careful consideration in mixed methods, particu-
larly where data collection is truly integrated. For
example, skilled interviewers are required where an
interview schedule includes a qualitative topic guide
and interviewer- and/or self-administered instruments.
These skills relate to an understanding of the data qual-
ity requirements of each paradigm, managing the con-
tent and flow of complex research interviews, working
flexibly and monitoring respondent expectations and
burden. These are in addition to the particular sensitiv-
ities and empathy required of palliative care researchers
working with vulnerable patients, often in their own
homes.
Mixed method interviews may be longer than those
focusing on quantitative or qualitative methods alone.
Careful choice of quantitative instruments, greater
focusing in the qualitative interview and piloting can
help. Interviewers need to be aware of respondent
comfort, especially in palliative care, and offer to split
interviews over two or more sessions; however, as expe-
rienced palliative care researchers we have found such
offers rarely taken up. Instead the interviewer must be
prepared to work flexibly, judging the ability of respon-
dents to continue and prioritizing interview components
(e.g. primary outcome measure and a more focused
qualitative interview). This requires skilled interviewers,
adequate training, but also careful design of the inter-
view schedule (prioritizing certain aspects): piloting is
essential.
Some of the complexity of managing mixed method
interviews can be overcome through office-based prep-
aration for the interview, such as careful ordering of
Farquhar
8 et al. 755
measures and topic guides. In addition, audio-record-
ing the entire interview rather than just the qualitative
sections is helpful: not having to switch the recorder on/
off midway removes attention from the equipment,
maintaining interview flow. Furthermore, it provides
an audio record for missing data queries, valuable pro-
cess data (as described in relation to the BIS Phase II
pilot trial) and additional qualitative data that quanti-
tative tools may generate through acting as prompts.
Data integration in analysis
One of the greatest challenges in mixed methods
remains the integration, in the analysis, of data col-
lected. There is a tendency to analyse and present the
findings of the respective methods separately, as stan-
dalone studies.2,43 This limits the value of mixed meth-
odology. To address this, O’Cathain et al.44 have
usefully summarized three techniques for integrating
data in mixed method studies: the triangulation proto-
col, following a thread, and the mixed method matrix.
As well as generating further understanding from the
data, such techniques help increase the credibility of the
integration, encourage transparency and further
develop methodology.
Discrepancies may arise in the findings of mixed
method studies.45 In itself this can justify the use of
mixed methods, by enhancing the clinical and ethical
utility of studies,38 but it also presents researchers with
dilemmas of interpretation. Table 6 outlines Moffatt
et al.’s46 six strategies for managing and investigating
conflicting findings from mixed methods. Their paper
illustrates these through a pilot study evaluating the
impact of welfare rights advice on health and social
outcomes among a population aged 60 and over
where quantitative findings showed little impact of the
intervention, yet the qualitative findings suggested
wide-ranging positive impacts.
To support the interpretation of findings, as well as
the design and execution of mixed method studies,
establishing and utilizing a comprehensive multidisci-
plinary research team is important (the disciplines
here being qualitative and quantitative, as well as
Table 6. Strategies for managing and investigating conflicting
findings from mixed method research (Moffatt et al.46)
(1) Treat the methods as fundamentally different
(2) Explore the methodological rigour of components
(3) Explore dataset comparability
(4) Collect further data and make further comparisons
(5) Explore the process of the intervention
(6) Explore whether the outcomes of the two components
match
Downloaded from pmj.sagepub.com at Univ of Connecticut / Health Center / Library on June 20, 2015
Palliative Medicine 0(00)
clinical). Team working in mixed methods brings its
own challenges.47 One strategy is to develop, at the
design stage, a mixed method protocol, outlining
which team members will be responsible for, and par-
ticipating in, different aspects of the study from data
collection through data processing, analysis and dis-
semination. The inclusion of timelines can help
manage expectations and facilitate inter-disciplinary
understanding of the time required to process and ana-
lyse different methods.
Costs
Using skilled researchers for data collection and engag-
ing a multidisciplinary research team has cost implica-
tions. Costs of mixed method studies can also be higher
due to the cumulative costs of combing two research
paradigms each with their own methodological require-
ments, for example, method-specific software and costs
such as, for example, the time requirements of observa-
tional work, transcription and analysis in qualitative
studies and large sample sizes or clinical tests in quanti-
tative studies. On the other hand, there may be cost sav-
ings for travel through concurrent data collection, and
an ultimate cost saving from increased confidence in the
data. In order to be cost effective, mixed method studies
need to make the best use of data, avoiding waste.
There is a need, therefore, for funding bodies to be
aware of the potentially greater utility, but potentially
higher cost, of mixed methods. This awareness is also
required in the peer reviewers that report to funding
panels, and funding panels themselves. Membership
of relevant funding panels should include those with
an understanding of mixed methods, or representation
from both paradigms.
Dissemination
Publication of mixed method findings can be difficult
due to word count restrictions, but some journals now
offer guidance in relation to publishing mixed method
studies (e.g. Journal of Advanced Nursing48), supple-
mentary material can often be published online (e.g.
in Palliative Medicine49) and, as noted earlier, specialist
journals exist. One strategy is to publish a mixed
method paper that is then supported by, and referenced
in, a more in-depth qualitative paper. Dahlberg et al.50
and Leech et al.51 are useful resources on developing
mixed method manuscripts for publication.
Conclusion
The development and evaluation of complex interven-
tions in palliative care benefit from the application
of mixed methodology within the MRC framework.
756 et al.
Farquhar
As demonstrated in the examples given, mixed methods
provide evidence from a variety of sources, enabling
better understanding of whether and how an interven-
tion works (or does not work) and inform the design of
subsequent studies. However, mixing methods should
not be seen as a panacea;38 mixed methods are not
always appropriate. They can be costly and bring with
them their own unique challenges in relation to data col-
lection, integration in analysis and dissemination. Mixed
method studies in palliative care will benefit from work-
ing with agreed protocols, multidisciplinary teams and
engaging experienced staff with appropriate skill sets.
Acknowledgement
The authors would like to acknowledge the support of admin-
istrative staff at the University of Cambridge and Cambridge
University Hospitals’ NHS Foundation Trust.
Funding
MF is funded by a Macmillan Cancer Support Post-Doctoral
Fellowship and the National Institute for Health Research
(NIHR) Collaborations for Leadership in Applied Health
Research and Care (CLAHRC) for Cambridge and
Peterborough as a Senior Research Associate in the End of
Life theme. GE is a Senior Research Associate at the Centre
for Family Research, funded by a sub contract from Dimbleby
Cancer Care and the University of Manchester. SB is
Honorary Senior Visiting Research Fellow at the University
of Cambridge and Honorary Senior Lecturer at the
Department of Palliative Care and Policy at King’s College
London, and was supported by a SuPac award. The clinical
work of the BIS team has received support from The Gatsby
Foundation and Macmillan Cancer Support. The BIS studies
were funded by The Gatsby Foundation, Cicely Saunders
International, NIHR Research for Patient Benefit (RfPB) pro-
gramme and the Addenbrooke’s Charitable Trust. This paper
includes independent research (BIS Phase III RCT) commis-
sioned by the NIHR under its RfPB programme (Grant
Reference Number PB-PG-0107-11134). The views expressed
are those of the authors and not necessarily those of the
National Health Service (NHS), the NIHR or the
Department of Health.
Conflict of interest statement
SB has led the overall development and evaluation of the BIS.
MF has co-led components of the evaluation: Phases I, II and
III. GE provided locum interviewing on Phase II.
References
1. Lansley A (Secretary of State for Health). ‘My Ambition
for Patient-centred Care’, http://www.dh.gov.uk/en/
MediaCentre/Speeches/DH_116643 (2010, accessed 18
March 2011).
2. O’Cathain A, Murphy E and Nicholl J. Why, and how,
mixed methods research is undertaken in health services
research in England: a mixed methods study. BMC Health
Serv Res 2007; 7: 85.
Downloaded from pmj.sagepub.com at Univ of Connecticut / Health Center / Library on June 20, 2015
9
Palliative Medicine 25(8)
3. Barbour RS. The case for combining qualitative and
quantitative approaches in health and services research.
J Health Serv Res Policy 1999; 4: 39–43.
4. Tashakkori A and Teddlie C. Mixed methods: combining
qualitative and quantitative approaches. Thousand Oaks,
CA: Sage, 1998.
5. Creswell JW. Research design: qualitative, quantitative
and mixed method approaches. Thousand Oaks, CA:
Sage, 2003.
6. Campbell M, Fitzpatrick R, Haines A, et al. Framework
for the design and evaluation of complex interventions to
improve health. Br Med J 2000; 321: 694–696.
7. Campbell NC, Murray E, Darbyshire J, et al. Designing
and evaluating complex interventions to improve health.
Br Med J 2007; 334: 455–459.
8. Craig P, Dieppe P, Macintyre S, et al. Developing
and evaluating complex intervention: the new
Medical Research Council guidance. Br Med J 2008;
337: a1655.
9. Creswell JW and Plano Clark VL. Designing and conduct-
ing mixed method research. Thousand Oaks, CA: Sage,
2007.
10. Greene JC. Mixed methods in social inquiry. San
Francisco, CA: John Wiley & Sons, 2007.
11. Johnson B and Christensen L. Educational research:
quantitative, qualitative, and mixed approaches, 3rd edn.
Thousand Oaks, CA: Sage, 2008.
12. Ridenour CS and Newman I. Mixed methods research:
exploring the interactive continuum. Carbondale, IL:
Southern Illinois University Press, 2008.
13. Andrew S and Halcombe EJ. Mixed methods research for
nursing and the health sciences. Oxford: Wiley-Blackwell,
2009.
14. Teddlie C and Tashakkori A. Foundations of mixed meth-
ods research: integrating quantitative and qualitative
approaches in the social and behavioural sciences.
Thousand Oaks, CA: Sage, 2009.
15. Onwuegbuzie AJ, Slate JR, Leech N, et al. Mixed
research. London: Routledge Academic, 2010.
16. Tashakkori A and Teddlie C. Sage handbook of mixed
methods research: integrating quantitative and qualitative
approaches in the social and behavioural sciences.
Thousand Oaks, CA: Sage, 2010.
17. Sandelowski M. Combining qualitative and quantitative
sampling, data collection, and analysis techniques in
mixed-method studies. Res Nurs Health 2000; 23:
246–255.
18. Brannen J. Mixed methods research: a discussion paper.
NCRM Methods Review Papers, ESRC National Centre
for Research Methods, December 2005.
19. Bryman A. Integrating quantitative and qualitative
research: how is it done? Qual Res 2006; 6: 97–113.
20. Mason J. Mixing methods in a qualitatively driven way.
Qual Res 2006; 6: 9–25.
21. Moran-Ellis J, Alexander VD, Cronin A, et al.
Triangulation and integration: processes, claims and
implications. Qual Res 2006; 6: 45–59.
22. Bryman A. Barriers to integrating quantitative and
qualitative research. J Mixed Methods Res 2007; 1:
8–22.
Farquhar
10 et al. 757
23. O’Cathain A, Murphy E and Nicholl J. The quality of
mixed methods studies in health services research. J
Health Serv Res Policy 2008; 13: 92–98.
24. Östlund U, Kidd L, Wengström Y, et al. Combining
qualitative and quantitative research within mixed
method research designs: a methodological review. Int J
Nurs Stud 2011; 48: 369–383.
25. Paterson C, Baarts C, Launsø L, et al. Evaluating com-
plex health interventions: a critical analysis of the ‘out-
comes’. BMC Compl Alternative Med 2009; 9: 18.
26. Cawley D, Waterman D, Roberts D, et al. A qualitative
study exploring perceptions and experiences of patients
and clinicians of Palliative Medicine Outpatient Clinics in
different settings. Palliat Med 2011; 25: 52–61.
27. Collins KMT and O’Cathain A. Ten points about mixed
methods research to be considered by the novice
researcher. Int J Multiple Res Approaches 2009; 3: 2–7.
28. Ewing G, Farquhar M and Booth S. Delivering palliative
care in an acute hospital setting; views of referrers and
specialist providers. J Pain Symptom Manage 2009; 38:
327–340.
29. Onwuegbuzie AJ and Teddlie C. A framework for ana-
lyzing data in mixed methods research. In: Tashakkori A
and Teddlie C (eds) Handbook of mixed methods in social
and behavioural research. Thousand Oaks, CA: Sage,
2003, pp.351–383.
30. Sampson EL, Thuné-Boyle I, Kukkastenvehmas R, et al.
Palliative care in advanced dementia: a mixed methods
approach for the development of a complex intervention.
BMC Palliat Care 2008; 7: 8.
31. Seers K, Gale N, Paterson C, et al. Individualised and
complex experiences of integrative cancer support care:
combining qualitative and quantitative data. Support
Care Canc 2009; 17: 1159–1167.
32. Shipman C, Burt J, Ream E, et al. Improving district
nurses’ confidence and knowledge in the principles and
practice of palliative care. J Adv Nurs 2008; 63(5):
494–505.
33. Vale-Taylor P. ‘‘We will remember them’’: a mixed-
method study to explore which post-funeral remem-
brance activities are most significant and important to
bereaved people living with loss, and why those activities
are chosen. Palliat Med 2009; 537–544.
34. Booth S, Silvester S and Todd C. Breathlessness in cancer
and chronic obstructive pulmonary disease: using a qual-
itative approach to describe the experience of patients
and carers. Palliat Support Care 2003; 1: 337–344.
35. Booth S, Moffat C, Farquhar M, et al. Developing a
breathlessness service for patients with palliative and sup-
portive care needs, irrespective of diagnosis. J Palliat
Care 2011; 27: 28–36.
36. Booth S, Farquhar M, Gysels M, et al. The impact of a
breathlessness intervention service (BIS) on the lives of
patients with intractable dyspnoea: a qualitative Phase I
study. J Palliat Support Care 2006; 4: 287–293.
37. Grande GE and Todd CJ. Why are trials in palliative care
so difficult? Palliat Med 2000; 14: 69–74.
38. Flemming K, Adamson J and Atkin K. Improving
the effectiveness of interventions in palliative care: the
Downloaded from pmj.sagepub.com at Univ of Connecticut / Health Center / Library on June 20, 2015
Palliative Medicine 0(00)
potential of qualitative research in enhancing evidence
from randomized controlled trials. Palliat Med 2008;
22: 123–131.
39. Farquhar M, Higginson IJ, Fagan P, et al. The feasibility
of a single-blinded fast-track pragmatic randomised con-
trolled trial of a complex intervention for breathlessness
in advanced disease. BMC Palliat Care 2009; 8: 9.
40. Farquhar M, Ewing G, Higginson IJ, et al. The experi-
ence of using the SEIQoL-DW with patients with
advanced chronic obstructive pulmonary disease: issues
of process and outcome. Qual Life Res 2010; 19: 619–629.
41. Farquhar M, Higginson IJ, Fagan P, et al. Results of a
pilot investigation into a complex intervention for breath-
lessness in advanced chronic obstructive pulmonary dis-
ease. Palliat Support Care 2010; 8: 143–149.
42. Farquhar M, Prevost AT, McCrone P, et al. Study pro-
tocol: Phase III single-blinded fast-track pragmatic ran-
domised controlled trial of a complex intervention for
breathlessness in advanced disease. Trials 2011; 12: 130.
43. Lewin S, Glenton C and Oxman AD. Use of qualitative
methods alongside randomised controlled trials of com-
plex healthcare interventions: methodological study. Br
Med J 2009; 339: b3496.
44. O’Cathain A, Murphy E and Nicholl J. Three techniques
for integrating data in mixed methods studies. Br Med J
2010; 341: 1147–1150.
45. Protheroe J, Bower P and Chew-Graham C. The use of
mixed methodology in evaluating complex interventions:
identifying patient factors that moderate the effect of a
decision aid. Family Pract 2007; 24: 594–600.
46. Moffatt S, White M, Mackintosh J, et al. Using quanti-
tative and qualitative data in health services research –
what happens when mixed method findings conflict?
BMC Health Serv Res 2006; 628.
47. O’Cathain A, Murphy E and Nicholl J. Multi-disciplin-
ary, interdisciplinary, or dysfunctional? Team working in
mixed-methods research. Qual Health Res 2008; 18:
1574–1585.
48. Journal of Advanced Nursing. ‘Empirical Research -
Mixed Methods’, http://www.journalofadvancednur-
sing.com/default.asp?file¼guidemixed (accessed 29
March 2011).
49. Palliative Medicine. ‘Guidelines for Submitting
Supplemental Files’, http://www.uk.sagepub.com/reposi-
tory/binaries/pdf/PMJ_Manuscript_Guidelines.pdf#9.4.4
(accessed 29 March 2011).
50. Dahlberg B, Wittink MN and Gallo JJ. Funding and
publishing integrated studies: writing effective mixed
methods manuscripts and grant proposals.
In: Tashakkori A and Teddlie C (eds) Sage handbook of
mixed methods research: integrating quantitative and qual-
itative approaches in the social and behavioural sciences.
Thousand Oaks, CA: Sage, 2010.
51. Leech NL, Onwuegbuzie AJ and Combs JP. Writing pub-
lishable mixed methods research articles: guidelines for
emerging scholars in the health sciences and beyond. Int
J Multiple Res Approaches 2011; 5: 7–24.