Social Science & Medicine 56 (2003) 803–814

Eliciting preferences of the community for out of hours care

provided by general practitioners: a stated preference discrete
choice experiment
Anthony Scotta,*, M. Stuart Watsonb, Sue Rossc
a
Health Economics Research Unit, University of Aberdeen, Foresterhill, Aberdeen AB25 9ZD, UK
b
Department of Public Health, University of Aberdeen, UK
c
Health Services Research Unit, University of Aberdeen, UK

Abstract

Access to primary care services is a major issue as new models of delivering primary care continue develop in many
countries. Major changes to out of hours care provided by general practitioners (GPs) were made in the UK in 1995.
These were designed in response to low morale and job dissatisfaction of GPs, rather than in response to patients’
preferences. The aim of this study is to elicit the preferences of patients and the community for different models of GP
out of hours care. A questionnaire was sent to parents of children in Aberdeen and Glasgow in Scotland who had
received a home visit or attended a primary care emergency centre, or were registered with a GP. The questionnaire used
a discrete choice experiment that asked parents to imagine their child had respiratory symptoms. Parents were then
asked to choose between a series of pairs of scenarios, with each scenario describing a different model of out of hours
care. Each model varied by waiting time, who was seen, location, and whether the doctor listened. The response rate
was 68% (3893/5718). The most important attribute was whether the doctor seemed to listen, suggesting that policies
aimed at improving doctor–patient communication will lead to the largest improvements in utility. The most preferred
location of care was a hospital accident and emergency department. This suggests that new models of primary care
emergency centres may not reduce the demand for accident and emergency visits from this group of patients in urban
areas. Preferences also differed across sub-groups of patients. Those who had never used out of hours care before had
stronger preferences for waiting time and the doctor listening, suggesting higher expectations of non-users. Further
research is required into the demand for out of hours care as new models of care become established. r 2002 Elsevier
Science Ltd. All rights reserved.

Keywords: Out of hours; General practice; Discrete choice experiments; Patients’ preferences; Scotland; UK

Introduction primary care, increasing public expectations, and the

There have been many changes to the delivery of
primary care in many countries, as health care reforms
introduce and experiment with new models of care. The
provision of out of hours (or after hours) care by
primary care physicians is a crucial element of primary
health care in many countries. Concerns about access to
*Corresponding author. Tel.: +44-1224-553866; fax: +44-
1224-662994.
E-mail address: a.scott@abdn.ac.uk (A. Scott).
effects on stress and family life for primary care
physicians, have led to reform of out of hours primary
care in several countries. A common theme in these
reforms has been a degree of centralisation and
integration of services through system-wide telephone
triage (Vedsted & Olsen, 1999; Hansen & Munck, 1998)
and growth in co-operative models of out of hours care,
including the establishment of primary care out of hours
emergency centres (Department of Health, 2000; Scot-
tish Out of Hours Study Group, 2001). These changes
have reduced the on-call commitment of primary care

0277-9536/03/$ - see front matter r 2002 Elsevier Science Ltd. All rights reserved.
PII: S 0 2 7 7 - 9 5 3 6 ( 0 2 ) 0 0 0 7 9 - 5
804 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814
physicians, increased the proportion of telephone con-
sultations, and changed the location and provider of
health care. The effects of these major changes in service
provision have therefore been supply-led, rather than
demand-led. There is little evidence of the effects on
patients.
In the UK, the provision of out of hours care by
general practitioners (GPs) has changed radically since
1995. As a result of deepening concern felt by GPs about
increasing workload out of hours and the ability to
maintain their 24 hour contractual commitment, the
arrangements for providing care out of hours were
altered in 1995. Funding was made available to set up
GP co-operatives, where many practices formally
collaborate to provide out of hours care. This funding
paved the way for the set up of new primary care
emergency centres open during evenings, nights and
weekends, and staffed by GPs from member practices.
In many areas, this has led to reductions in the numbers
of home visits as more patients are asked to travel to the
emergency centre and more advice is given over the
telephone. The structure and organisation of co-
operatives vary, with some providing nurse triage,
patient and doctor transport, and being supported by
a formal management board and administrative struc-
ture. Others have a less formal structure, and are more
similar to rotas. These changes have influenced mainly
urban areas of the UK, with care in rural areas provided
along more traditional lines with GPs providing their
own cover for their own patients (Hallam and
Henthorne, 1999; Department of Health, 1998).
These arrangements were devised and implemented to
reduce stress and improve the morale of GPs, with little
recourse to the preferences of patients and the commu-
nity. The changes have had important effects on the
process of care, including location of care, waiting times,
and who is seen as well as potential effects on health
status.
Several patient satisfaction studies have been con-
ducted which have used a variety of instruments, and
asked about a variety of different models of care
(McKinley et al., 1997a, b; Salisbury, 1997; Bain,
Gerrard, Russell, Locke, & Baird, 1997). However,
satisfaction studies suffer from several known short-
comings. Although they ask about patients’ experience
of the care they have had, which is relevant, they do not
directly ask about preferences for alternative models of
care. It is difficult to determine the relative importance
of attributes from satisfaction studies, since dissatisfac-
tion (or satisfaction) with an attribute does not
necessarily indicate that it is the most important to
patients (Scott & Smith, 1994; Carr-Hill, 1992).
Satisfaction studies also ignore the notions of sacrifice
and opportunity cost: resources for out of hours care are
finite and choices need to be made about where they are
likely to have the best effect. To have more of one
characteristic means less of another. It is impossible for
patients to receive a home visit within 5 min of their call,
from their own GP. It is therefore important to find out,
from the patients’ perspective, which attribute they
would most like to be improved, given that they cannot
have the best level of every attribute. In short, priorities
need to be set.
The aim of this study was to elicit the preferences of
users and non-users (i.e. the community) for different
models of out of hours care, and to examine the relative
importance of attributes of out of hours care. The study
uses a discrete choice experiment which has its origins in
mathematical psychology, market research, and eco-
nomics and has been developed as a method of
examining preferences for attributes or characteristics
of goods and services (Brunel University, 1993). Its
application in health care is growing (Ryan, 1999).
Method
A postal questionnaire was developed which pre-
sented respondents with a number of pairs of scenarios,
where each scenario described a particular model of out
of hours care. For each pair, respondents were asked to
choose which scenario they preferred (Fig. 1). The
attributes for the scenarios were chosen from the
existing literature and from face-to-face and postal
piloting of the questionnaire (Table 1). Several studies
have suggested that who the patient consults is
important, as is the time between initial contact and
actual consultation with a doctor (McKinley et al.,
1997a; Sawyer & Arber, 1982; Prudhoe, 1984; Bollam,
McCarthy, & Modell, 1988; Dixon & Williams, 1988;
Cragg, Campbell, & Roland, 1994).
Furthermore, the current policy context suggests that
the location of the out of hours consultation is
important. The doctor–patient relationship is also a
relevant factor and has been shown by many previous
patient satisfaction studies to be more important than
other aspects of care in general practice (Smith &
Armstrong, 1989; Calnan et al., 1994). The attribute
representing the quality of the doctor–patient relation-
ship was ‘whether the doctor seems to listen to what you
have to say’. This was chosen on the basis of the results
of a previous choice experiment examining the impor-
tance of different attributes of the GP consultation (Vick
& Scott, 1998). ‘Whether the doctor listens’ was found to
be the most important to patients. The scenarios
therefore described out of hours care in terms of the
location of the consultation, time between initial contact
and consultation, who was seen, and whether the doctor
seemed to listen (Table 1).
A vignette was used to place the hypothetical
scenarios in a recognisable and realistic setting. The
scenarios were therefore presented in the context of
 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814 805

• Imagine that during the night, your child is short of breath, wheezing and coughing
and that you decide to call a doctor. You have several options about the care you
receive. These differ according to who your child sees, where they are seen, the
time it takes between making the telephone call and receiving treatment, and
whether the doctor seems to listen to what you have to say.

• For each question below, you are asked to choose which type of consultation you
would prefer for your child during the night (Consultation A or Consultation B).

1. Which consultation would you prefer? (please tick box below)

Consultation A Consultation B
Where your child is seen: Emergency centre run Your home
by GPs
Who your child sees: A GP who doesn’t work A GP who doesn’t
at your practice/health work at your
centre practice/health centre
Time taken between the 60 minutes 20 minutes
telephone call and treatment
being received:

Whether the doctor seems The doctor seems to The doctor seems to
to listen to what you have to listen listen
say:

Prefer consultation A Prefer consultation B

(please tick one box)

Fig. 1. Example of a choice presented to patients.

Table 1 Gilmour, & Sullivan, 1991). Parents of young children

Attributes and levels used in the questionnaire
Attribute Levels of each attribute
Where your child is seen Emergency centre run by
GPs
Your home
A hospital accident and
emergency department
Who your child sees A GP from your
practice/health centre
A GP who doesn’t work
at your practice/health
centre
Time taken between the 20 min
telephone call and 40 min
treatment being received 60 min
80 min
Whether the doctor The doctor seems to
seems to listen to what listen
you have to say The doctor does not
seem to listen
children with respiratory illness (Fig. 1). Children
represent a large proportion of consultations out of
hours, with respiratory disease contributing significantly
to these consultations (Cragg et al., 1994; Morrison,
have also been shown to be the most dissatisfied users of
the service in one of our study sites (Glasgow) (Wilson
et al., 2001).
The attributes and levels (Table 1) were organised into
scenarios using a factorial experimental design. From a
possible 48 scenarios, 16 were generated using a
fractional factorial design. This provides the minimum
number needed to maintain an orthogonal design. One
scenario was chosen to be constant across all pairs
(Scenario A), and the others compared with it (Fig. 1).
The constant scenario was chosen to represent the new
model of an emergency centre. The remaining 15
scenarios were then divided randomly across two
questionnaires, one with eight pairs and one with seven.
Each type of questionnaire was allocated randomly to
respondents.
Once scenarios are paired (whether using a constant
scenario or a random pairing), the orthogonality of the
original main effects design is not guaranteed. Ortho-
gonality was therefore checked by examining the
associations between attribute differences (i.e. the
difference between the levels of each pairwise choice)
in the 15 scenarios. Appendix A shows that associations
between attribute differences were low and not statisti-
cally significant.
806 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814
Face validity was assessed during piloting in a
respiratory clinic at Aberdeen Children’s Hospital, and
at the Glasgow and Aberdeen primary care emergency
centres. Internal consistency was examined by analysing
those pairs of scenarios where one scenario was
obviously dominant. The experimental design produced
three scenarios that were dominant, assuming a priori
that individuals prefer shorter waiting times, prefer a
home visit to a GP emergency centre visit, and prefer to
see a GP they know. One questionnaire had two
dominant scenarios and the other version had one. If a
respondent chose inconsistently (i.e. choose the non-
dominant scenario), then all of the respondent’s choices
were removed from the data set. Theoretical validity
was assessed using the signs of the coefficients to
test prior hypotheses about the nature of preferences
(e.g. shorter waiting time is preferred to longer waiting
time).
To examine the effect of respondent characteristics
and past experiences on the relative importance of
attributes, additional questions were asked for demo-
graphic details, past experiences of out of hours care and
wheezing illness, and about a range of access factors
such as the use of a car at night, and availability of
someone else to look after children. For example, it
might be expected that parents with younger or fewer
children are more likely to prefer shorter waiting times
or the doctor listening. Parents’ own characteristics may
also influence their preferences, such as gender and level
of education. It may also be expected that past
experiences of out of hours care might influence
preferences, indirectly testing hypotheses related to
stability of preferences and path-dependence.
The questionnaire was sent to parents of children in
Aberdeen and Glasgow, allowing comparison of two
communities with contrasting social structures (affluent
and one of the most deprived in the UK, respectively).
As a guide to relative deprivation using the Carstairs
deprivation index, around half of Greater Glasgow
Health Board’s population were classified in categories
six and seven (where seven is the highest category of
deprivation). This is compared with none of the
population in Grampian (including Aberdeen) being
classified in these categories (McLaren & Bain, 2000).
Each area had a new co-operative, Grampian doctors on
call service (GDOCS) based in Aberdeen, and Glasgow
emergency medical service (GEMS) in Glasgow.
From each area (and with ethical approval), three
random samples of children aged under 13 years old
were identified: those who had visited a primary care
emergency centre; those who had received a home visit;
and those from the lists of GPs (i.e. who may or may not
have used out of hours care before). Duplicates from the
samples of users and temporary residents were excluded
from this latter sample. Information about those who
had a centre visit or home visit was gathered from the
records of the emergency centres. Records of GDOCS
users were available for a 12-month period and those for
GEMS were available for a 6-month period. On the
basis of previous experience of using choice experiments
in the community, expected response rates and expected
sub-group analysis, we estimated sample sizes of 1800
non-users and 715 in each user group from each area
were required to obtain sufficient responses for analysis
(Pearmain, Swanson, Kroes, & Bradley, 1991). Remin-
ders were sent at 3 and 6 weeks if necessary.
Econometric model
Each model of out of hours care is represented by an
indirect utility function. With two models of out of
hours care (i and j), y
n is a latent variable representing
the difference in utility between them, with n individuals
making a choice. Since it is the choice that is observed
rather than the difference in utility, y
n is binary.
Therefore:
yn ¼ 1 if y
n > 0 and 0 else
and
y
n ¼ ða þ bxi þ qsn þ ein Þ  ða þ bxj þ qsn þ ejn Þ; ð1Þ
where a; b and @ are coefficients, x are the attributes of
each model of care, s are socio-economic characteristics
reflecting influences on tastes and e is the random
component of utility accounting for the analyst’s
inability to accurately observe individual’s behaviour
(Manski, 1977). Further, assume that there are
taste variations such that the marginal utility of x
depends on s:
b ¼ p þ lsn : ð2Þ
This gives
y
n ¼ ða þ pxi þ lsn xi þ qsn þ ein Þ
 ða þ pxj þ lsn xj þ qsn þ ejn Þ: ð3Þ
Because the discrete choice experiment presents each
respondent with several pairs of scenarios, multiple
observations from each respondent mean that errors are
not independent and so an error term mn capturing
random variation across respondents is included:
y
n ¼ ða þ pxi þ lsn xi þ qsn þ ein þ mn Þ
 ða þ pxj þ lsn xj þ qsn þ ejn þ mn Þ: ð4Þ
Taking differences for each pairwise choice (k), the
equation to be estimated becomes
y
kn ¼ pxk þ lsn xk þ ekn : ð5Þ
Terms common to both indirect utility functions drop
out of the model (i.e. a; qsn ; and mn ).
However, the inclusion of a constant term (a) and
error term across respondents (mn ) can be used to test for
mis-specification due to unobservable attributes and
 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814 807

unobservable interaction terms between GPs’ socio-
economic characteristics and attributes. The constant
term can be interpreted as the difference in the average
utility of scenario i and j; caused by the use of a constant
scenario, left/right bias or an omitted dummy variable
that is a function of other included attributes (Scott,
2000). The model to be estimated then becomes
y
kn ¼ a þ pxk þ lsn xk þ ekn þ mn :
This model was estimated using random effects probit
regression. A model with main effects was estimated
first, and used to calculate marginal rates of substitution
between each attribute and the time attribute. A full
model was estimated including main effects and inter-
action terms for which hypotheses existed. This was then
reduced to a more parsimonious model by excluding
variables, one at a time, with p-values >0.10.
Results
Excluding those questionnaires that were not deliv-
ered (586), the final response rate was 3893/5718
(68.1%). Two questionnaires were completed by chil-
dren and were excluded from analysis. Time to complete
the questionnaire ranged from 2 to 60 minutes (mean
12 minutes), and 1760/3757 (46.8%) of respondents
considered that the questionnaire was easy to answer.
Characteristics of the respondents and their last out of
hours consultation are presented in Tables 2 and 3.
For the regression analysis, the final sample size was
3326 individuals (24,789 observations), after the deletion
of observations where data on the dependent and
independent variables were missing (2391; 8%), and
inconsistent responses (2023; 7.5%). The main effects
model is shown in Table 4. r; measuring the correlation
between observations of the same respondent, was
statistically significant, suggesting that a random effects
specification was appropriate and that there may be
unobservable interaction terms between attributes and
ð6Þ
characteristics of respondents.
The constant term is statistically significant and
negative, suggesting a general preference for scenario
A. Scenario A represented the new co-operative model
with a primary care emergency centre, and Scenario B
represented other models of care. Although this suggests
there was a general preference for new models of care,
other explanations for the significant constant term
should not be ruled out (i.e. ‘right’ bias and omitted
attributes).
Face and content validity were established through
piloting, there was a low level of inconsistent responses
(8%), and the results were consistent with prior
hypotheses (e.g. shorter waiting times are preferred to
longer), confirming the techniques’ theoretical validity.
Using data from this study that has been reported
elsewhere, the instrument used was also found to be
reliable (San Miguel et al., forthcoming).
For the main attributes, the signs of coefficients
suggest that respondents preferred to see a GP from
their own practice or health centre, preferred shorter to
longer waiting times, and preferred the doctor to listen.
These are as expected and confirm the theoretical
validity of the technique. In terms of the location of

Table 2
Descriptive characteristics of respondents

Characteristic Response

From Aberdeen 44% (1701/3890)

Age of parent/guardian (mean years, range) 34 (16–75)
Gender of parent/guardian (% female) 87.4% (3394/3883)
Age of children (mean years, range) 6.9 (0–25)
Number of children (mean, range) 2.2 (0–12)
Highest level of education of parent/guardian University 17% (659/3858)
College 32% (1215/3858)
Secondary 50% (1937/3858)
None 1.1% (43/3898)
Health status of child Excellent 47% (1812/3878)
Good 41% (1586/3878)
Fair 10% (374/3878)
Poor 2% (106/3878)
Child has ever suffered from asthma or other ‘wheezy’ illness Yes 38% (1463/3881)
Child has other medical problems for which they see a doctor regularly Yes 16% (630/3869)
Access to a car during the night Yes 73% (2805/3830)
Availability of others to help look after children during the night Yes 74% (2756/3715)
Used out of hours care before Yes 42% (1634/3890)
808 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814

Table 3 marginal rates of substitution (Table 4). Waiting time

Characteristics of last out of hours visit for those who had used becomes more important than ‘whether the doctor
an out of hours service listens’ if the difference in waiting time was longer than
Characteristics Response 66 minutes. This indicates that respondents would be
of last out of willing to wait up to 66 minutes to see a doctor who
hours visit: seems to listen. Waiting time became more important
than ‘who was seen’, if the difference in waiting time was
When did your Less than 1 3% (106/3879)
longer than 14 minutes, suggesting that respondents
child last see a month ago
doctor during 1–6 months ago 15% (591/3879)
would be willing to wait 14 minutes to see a doctor from
the night? 6–12 months ago 17% (660/3879) their own practice. Similarly, waiting time was more
More than 1 24% (938/3879) important than being seen in an A&E department if the
year ago difference waiting time was longer than 33 minutes. This
Never 33% (1266/3879) also represents the extra time respondents are willing to
Can’t remember 8% (318/3879) wait to be seen in an A&E department rather than a
Doctor seemed 84% (2197/2623) primary care emergency centre. Waiting time became
to listen more important than being seen at home rather than in a
Doctor did not 10% (258/2623) primary care emergency centre, if the difference in
seem to listen
waiting time was longer than 11 minutes.
Can’t remember 6% (168/2623)
Place of visit A&E 13% (353/2631) The reduced model is shown in Table 5. The main
Emergency 33% (865/2631) effects are of a similar sign, except for the constant term
centre and whether there is a home visit or a primary care
Home visit 48% (1263/2631) emergency centre visit. The constant term is now
Can’t remember 6% (150/2631) positive, suggesting a general preference for Scenario
Time between Around 20 min 29% (742/2592) B. This will be a function of the addition of interaction
phoning and Around 40 min 33% (845/2592) terms that involve dummy variables, and show that once
seeing a doctor Around 60 min 15% (394/2592) respondents’ characteristics are included in the model,
Around 80 min 10% (256/2592) the general preference is for more traditional models of
>80 min 0.03% (14/2592)
care, rather than the new co-operative model with a
Can’t remember 13% (341/2592)
Who was seen GP from 25% (641/2616) primary care emergency centre. However, other expla-
practice nations for the significant constant term should not be
GP not from 68% (1781/2616) ruled out (i.e. ‘right’ bias and omitted attributes). The
practice sign of the variable examining preferences for either a
Can’t remember 7% (194/2616) home visits or a visit to a primary care emergency centre
has also changed, suggesting that once respondents’
characteristics are included, a primary care emergency
care, respondents preferred a home visit compared to a centre visit is preferred to a home visit.
primary care emergency centre run by GPs, and a The coefficients of the interaction terms show how
hospital A&E department to a primary care emergency preferences differ depending on respondents’ character-
centre. istics and past experiences. For location of care, home
The relative importance of each attribute can be visits and A&E visits were more likely to be preferred to
examined by using the main regression coefficients, primary care emergency centre visits for those who had a
which are measured on the same scale, as well as college and university education, compared to those who
marginal rates of substitution. The most important had a secondary school education. Those with children
attribute was ‘whether the doctor seemed to listen’, in fair/poor health compared to excellent health also
followed by being seen at an A&E department compared preferred to receive a home visit.
to a primary care emergency centre, seeing a GP from There was also evidence that the location of their last
your own practice, and being seen at a home visit out of hours visit influenced preferences. A home visit
compared to a primary care emergency centre. Waiting was preferred to a primary care emergency centre visit
time was also a significant predictor, with the impor- by those who had previously had an A&E visit and a
tance of waiting time compared to other attributes home visit compared to those who never visited before.
depending on how long they had to wait. The coefficient Emergency centre visits were more likely to be preferred
gives the relative importance for a difference of 1 minute to home visits for those who previously had an
in waiting time between each model in the scenarios. emergency centre visit, and those who received a visit
It is possible to examine the approximate ‘thresholds’ more than a year ago. An A&E department was more
at which waiting time becomes more important than the likely to be preferred to a primary care emergency centre
other attributes, by examining the implied trade-offs or visit for those who previously received a home visit.
 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814
Table 4
Regression results (main effects only)
Variable b (SE)
Constant 0.2015 (0.0399)*
Who is seen 0.4686 (0.0254)*
Waiting time 0.0332 (0.0006)*
Whether the doctor listens 2.1925 (0.0275)*
Home visitb 0.3638 (0.0422)*
Hospital A&E department visitb 1.0892 (0.0363)*
r 0.3227 (0.0121)*
2 log likelihood 9832
Model chi-squared (d.f.) 6959 (5)*
Psuedo R2 29%
Notes: *=po0:0001:
a
MRS ¼ bx =bwaiting time : Standard errors calculated from a Taylor series approximation to the variance of a function of random
variables, where varðMRSÞ ¼ 1=b2waiting time ½varðbx ÞF2MRS covðbx ; bwaiting time Þ þ MRS2 varðbwaiting time Þ (Propper, 1988).
b
Relative to a primary care emergency centre.
Preferences for seeing a GP from their own practice
(or not) also differed across sub-groups. Those sampled
from Glasgow were more likely to prefer to see a GP
from their own practice, compared to those from
Aberdeen, as were those sampled from the general
population compared to those who were sampled from
home visits. Seeing a GP from their own practice had a
higher marginal utility for respondents with older
children, with a secondary rather than university
education, and for those who saw a GP from their
own practice at their last visit.
A shorter waiting time was preferred by respondents
whose child had not previously suffered from asthma, by
those with fewer and younger children, and by those
with children in excellent compared to good, fair or poor
health. Preferences for waiting time were also influenced
by parent’s characteristics, independent of their chil-
dren’s characteristics. A shorter waiting time was
preferred by those sampled from Glasgow compared
to Aberdeen, by those with a college and university
education compared to secondary education. Past
experiences also influenced the marginal utility of
waiting times, as shorter waiting times were preferred
by those who had never had an out of hours visit
compared to those who visited up to 1 year ago, and by
those who waited 20 minutes at their previous visit.
Those who waited more than 1 hour for their last visit
were more likely to prefer longer waiting times,
compared to those who had never had a visit.
Preferences for the doctor–patient relationship were
also influenced by respondents’ characteristics. Those
who were more likely to prefer the doctor who seem to
listen included respondents who were younger, who were
sampled from the general population compared to those
809
MRSa (SE) 95% CI
— —
14 (0.83) 15.73 to 12.48
— —
66 (1.11) 68.17 to 63.82
11 (1.34) 13.58 to 8.32
33 (1.25) 35.23 to 30.35
— —
% 0’s predicted correctly 96%
% 1’s predicted correctly 66%
Number of observations 24789
Number of individuals 3326
sampled from centre visits, and who had a college or
university education compared to those who had a
secondary education. Respondents’ childrens’ character-
istics also had an effect, with the doctor listening being
preferred by those whose children had no other medical
problems, and who were in excellent rather than poor or
fair health. Past experiences also had an impact. The
doctor listening was preferred by those who had never
received an out of hours visit compared to those who
visited up to 1 year ago, and by respondents whose
doctor had seemed to listen at their last visit.
The version of the questionnaire respondents were
sent also influenced preferences. Although the scenarios
were allocated to each version of the questionnaire
randomly, this does not guarantee that there will be no
systematic differences in preferences. The inclusion of
these variables in the model controls for these biases.
Discussion
The most important attribute was whether the doctor
seemed to listen. This was independent of whether the
patient knew the doctor, perhaps reflecting the real
reason why other studies have reported low satisfaction
with deputising services. This finding is consistent with
other studies examining patient satisfaction with general
practice and the doctor–patient relationship (Williams &
Calnan, 1991; Savage & Armstrong, 1990; Wissow,
Roter, & Wilson, 1994), with out of hours services
(Scottish Out of Hours Study Group, 2001), as well as
previous discrete choice experiments examining the
doctor–patient relationship (Vick & Scott, 1998) and
out of hours care (Morgan, Shackley, Pickin, & Brazier,
810 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814

Table 5
Regression results (model with interaction terms)

Variable B SE

Constant 0.3586 0.0593***

Who is seen 0.3975 0.0624***
Waiting time 0.0618 0.0024***
Whether the doctor listens 3.3710 0.1189***
Home visita 0.7080 0.1011***
Hospital A&E department visita 1.7586 0.0809***
Centre visit sample * home visitc 0.2574 0.0531***
Home visit sample * home visitc 0.2061 0.0606**
College education * home visitd 0.1926 0.0676**
University education * home visitd 0.1566 0.0786**
Fair/poor health * home visite 0.2012 0.0729**
Last visit more than 1 year ago * home visitf 0.1538 0.0627**
Last visit at A&E * home visitf 0.2747 0.0897**
Last visit at home * home visitf 0.1555 0.0609**
College education * A&E visitd 0.1238 0.0643*
Last visit at home * A&E visitf 0.1732 0.0476**
Grampian Health Board * who is seenb 0.0987 0.0411**
Age of children * who is seen 0.0139 0.0054**
Centre visit sample * who is seenc 0.0988 0.0594*
University education * who is seend 0.1640 0.0710**
Saw a GP from own practice at last visit * who is seenf 0.1377 0.0534**
Grampian Health Board * waiting timeb 0.0027 0.0008**
Whether child has asthma * waiting time 0.0028 0.0009**
Number of children * waiting time 0.0013 0.0004**
Age of children * waiting time 0.0003 0.0001**
College education * waiting timed 0.0029 0.0013**
University education * waiting timed 0.0049 0.0016**
Good health * waiting timee 0.0024 0.0008**
Fair/poor health * waiting timee 0.0056 0.0017**
Last visit up to 6 months ago * waiting timef 0.0029 0.0013**
Last visit between 6 months and 1 year * waiting timef 0.0048 0.0012**
Waited 20 min at last visit * waiting timef 0.0023 0.0011**
Waited 60 min at last visit * waiting timef 0.0036 0.0013**
Waited over 80 min at last visit * waiting timef 0.0052 0.0016**
Other health problems of child * doctor listens 0.1170 0.0374**
Centre visit sample * doctor listensc 0.0861 0.0414**
Age * whether the doctor listens 0.0167 0.0024***
College education * doctor listensd 0.3137 0.0497***
University education * doctor listensd 0.1551 0.0539**
Fair/poor health * doctor listense 0.1727 0.0462**
Last visit up to 6 months ago * doctor listensf 0.2258 0.0461***
Last visit between 6 months and 1 year ago * doctor listensf 0.1429 0.0433**
Doctor listened at last visit * doctor listens 0.1058 0.0331**
Questionnaire * home visit 0.7898 0.1245***
Questionnaire * A&E visit 1.4554 0.0647***
Questionnaire * waiting time 0.0128 0.0014***
r 0.3097 0.0126***

2 log likelihood 9199 % 0’s predicted correctly 90%

Model chi-squared (d.f) 5916 (45)*** % 1’s predicted correctly 72%
Psuedo-R2 39% Number of observations 24789
Number of individuals 3326

Notes: ***=po0:0001; **=0.0001Xpo0:05; *=0.10Xpp0.05.

a
Relative to a primary care emergency centre.
b
Relative to Greater Glasgow Health Board.
c
Relative to general population sample.
d
Relative to secondary education.
e
Relative to excellent health.
 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814
2000). The finding suggests that if GP out of hours
organisations want to increase patients’ utility the most,
then they should focus on improving doctors’ commu-
nication skills.
Preferences differed by respondents’ characteristics.
Socio-economic differences are reflected by respondents’
education. Highly educated respondents had a stronger
preference for A&E department visits to primary care
emergency centre visits and home visits to primary care
emergency centre visits. They were also less concerned
about seeing a GP from their own practice, more
concerned about a lower waiting time, and had a
stronger preference for the doctor listening. This may
have implications for the provision of services and the
pattern of demand in affluent compared to deprived
areas. A previous satisfaction study of out of services in
Glasgow showed that those from affluent areas were
more likely to be dissatisfied (Wilson et al., 2001).
The characteristics of parents’ children also influenced
preferences. Respondents with children in fair or poor
health were more likely to prefer a home visit than a
primary care emergency centre visit. Those with children
in excellent health, and who had not previously had
asthma had a stronger preference for a shorter waiting
time. This may be related to limited experience of health
care services or illness where expectations with respect to
waiting times may be high. Parents of children with
other health problems were less likely to prefer the
doctor to listen. Parents with younger and fewer
children were more likely to prefer a shorter waiting
time, whilst those with older children had a stronger
preference to see a GP from their own practice.
Previous utilisation of GP out of hours services
influenced preferences. Those who had never visited an
out of hours service before were more likely to prefer a
visit at a primary care emergency centre compared to a
home visit, and more likely to prefer a shorter waiting
time and for the doctor to listen. Those sampled from
the general population had a stronger preference for the
doctor listening and seeing a GP from their own
practice. This has implications for the education of
potential users about appropriate use of out of hours
services, and also shows that the potential users may
have higher expectations of out of hours care compared
to those who have used the service before.
What happened at the respondent’s last out of hours
visit also influenced preferences, such that respondent’s
preferred what they had previously experienced. This
confirms previous studies, and has implications for when
preferences should be elicited during a health care
episode (Salkeld et al., 2000). Along with the influence of
past utilisation, it suggests that to a certain extent,
preferences are endogenous. It is not necessarily the case
that individuals hold a set of stable and complete
preferences as implied by standard neo-classical micro-
economic theory. Once a dynamic element is introduced,
811
preferences may be modified by past experiences and
may also exhibit a degree of path-dependence. This
highlights the importance of gathering information
about these past experiences in preference elicitation
studies.
The most preferred model of GP out of hours care for
parents of children who have respiratory symptoms in
urban areas, is a hospital accident and emergency
department where the doctor seems to listen, where
waiting time between the initial phone call and treatment
being received is low, and where they see a doctor from
their own practice. This describes a model of care that
does not exist in most places of the UK, although there
are examples of GPs working in A&E departments, and
of primary care emergency centres located adjacent to
A&E departments. For example, GEMS in Glasgow has
six primary care emergency centres, several of which are
located alongside A&E departments.
These preferences were elicited without any informa-
tion on the costs of providing this model of care and so
it may not be possible, for example, for patients to see a
GP from their own practice in an A&E department. The
results of this study would need to be combined with
costs of different combinations of attributes to establish
the most cost-effective model of care.
Nevertheless, the results have important implications
for the demand for new models of GP out of hours care
which are based in primary care emergency centres and
where the patient may not see a GP from their own
practice. For parents of children with respiratory
symptoms, these new models may not be the most
preferred. The consequence of this may be that new
models of GP out of hours care may not reduce the
demand from parents of young children with respiratory
problems, who may continue to use A&E services. New
primary care emergency centres may not therefore
reduce the demand for A&E services. It also suggests
that primary care emergency centres that are located
within or adjacent to A&E departments may be more
effective at reducing demand at A&E departments, than
centres located away from acute hospitals.
These results should, however, be interpreted with
several issues in mind. First, the results presented here
apply to the parents of children under 13, when their
child has respiratory symptoms. Although this is a
relatively common condition seen by GPs out of hours,
preferences may differ for other groups of individuals,
such as the elderly, and for other types of symptoms of
differing severities. In addition, the study applied to
urban areas only and extrapolation to rural areas would
be difficult.
Second, some individuals responding may not be
using compensatory decision rules. This can be proble-
matic when a lexicographic ordering exists since utility
functions cannot be defined (Scott, 1998). These
individuals have very strong preferences for a given
812 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814
attribute since they are not prepared to trade off with
other attributes. Generally, individuals may have used a
variety of non-compensatory decision making rules
and heuristics that lie between the extremes of fully
compensatory decision making and lexicographic
orderings (Payne, Bettman, & Johnson, 1992). The
implications of non-compensatory decision making
are that policy recommendations based on the marginal
rate of substitution should be treated with caution.
Our results found that individuals were prepared to wait
an additional 66 minutes to see a doctor who seems
to listen. This implies that a policy that reduces waiting
times by 66 minutes would generate the same improve-
ment in utility as a policy that improved doctors’
communication skills. However, individuals with a
lexicogrpahic ordering would not be willing to
wait at all. In this case, any policies to reduce
waiting times would not influence their utility, or
demand. For those sub-groups of the sample with a
lexicographic ordering, the implication is that reducing
waiting times would be of no value to them. A further
implication is that inclusion of those with lexicographic
orderings will overestimate the size of regression
coefficients for those attributes where individuals have
dominant preferences.
It is important to find ways of identifying these sub-
groups of individuals, although this may only be
possible through face-to-face interviews that explore
individuals’ preferences in more depth. However, there
may be different reasons for the existence of these
heuristics, that include the complexity of the question-
naire (bounded rationality) or because they genuinely
have a dominant preference. Alternative ways of
examining preferences should also be explored, such as
those reviewed by Ryan et al. (2001).
There are also other issues when interpreting the
attributes. The study has not investigated why respon-
dents preferred the A&E department. This may because
of a higher perceived ‘quality’ of service and faster
access to other hospital facilities. This implies other
attributes of potential importance that require further
research. Furthermore, different wording of the attri-
butes may have influenced responses, especially for the
doctor listening. Further research should be conducted
on framing effects in choice experiments.
A further issue is the treatment of inconsistent
responses. Eight percent of responses were inconsistent,
suggesting those individuals did not understand the task
being set. This compares favourably with other methods
of eliciting preferences in health care (with levels of
inconsistency of between 7% and 77%) (Vick & Scott,
1998). Although the inclusion of dominant scenarios to
check for inconsistency may reduce the number of
choices that involve trading-off attributes, this is not a
major concern and has not manifested itself in a poor
quality econometric model.
Finally, there are several issues about the experi-
mental design. When placing scenarios into pairs, the
property of orthogonality that was in the original linear
design is no longer guaranteed. What is required is a
design that minimises colinearity between attribute
differences. However, it is not necessarily the case that
correlations have to equal zero as there are other aspects
to the design that need to be considered, such as balance
(that the levels of attributes appear with equal
frequency) and minimal overlap of attribute levels. It
has been recognised that, ‘‘y for most combinations of
attributes, levels, alternatives and parameter vectors, it is
impossible to create a design that satisfies (all of) these
principles’’ (Zwerina, Huber, & Kuhfeld, 1996).
A common assumption in choice experiments is that
interactions between the main effects are negligible.
Because there were no specific hypotheses that the main
effects in this study were interdependent, we used a
fractional factorial design and so assumed interaction
effects were negligible. If there are interaction effects,
then reliable estimation of these is only possible if they
are built into the experimental design (or if a full factorial
design is used). However, this does not rule out the
possibility that interaction effects exist. This may reduce
the explanatory power of the model and could lead to
incorrect estimates of regression coefficients for the main
effects, and must be borne in mind when interpreting
results. Although there have been few studies in health
that have examined interactions, previous empirical
studies in transport suggest that interactions between
main effects are negligible (Louviere, 1988).
Further research is required into the demand for out
of hours services, especially as alternative models begin
to emerge. It is not sufficient to reconfigure services to
suit the preferences of providers, as this takes no
account of patterns of demand. Preferences of both
users and non-users need to be accounted for.
Acknowledgements
Thanks go to Nicola Torrance for collecting data, and
to the staff of G-DOCs and GEMS. Thanks also go to
anonymous referees and Cristina Ubach for helpful
comments. This project was funded by the NHS R&D
Primary–Secondary Care Interface Programme. The
Health Economics and Health Services Research Units
are funded by the Chief Scientist Office of the Scottish
Executive Health Department (SEHD). The views in this
paper are those of the authors and not SEHD.
Appendix A. Testing whether the design is orthogonal
The differences between the levels of the paired
attributes were tested for orthogonality. The sample
 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814 813

Table 6
Measures of association between attribute differences

D WHERE D WHO D LISTEN D TIME

a b c b
D WHERE — 0.07 /0.38 0.006 /0.094 0.22d
D WHO 0.07a/0.38b — 0a/0.07b 0.03d
D LISTEN 0.006c/0.094b 0a/0.07b — 0.03d
D TIME 0.22d 0.03d 0.03d —
a
l: A measure of association which reflects the proportional reduction in error when values of the independent variable are used to
predict values of the dependent variable. A value of 1 means that the independent variable perfectly predicts the dependent variable. A
value of 0 means that the independent variable is no help in predicting the dependent variable.
b
Measures of association based on chi-square including: contingency coefficient (measure of association based on chi-square—the
value ranges between zero and 1, with zero indicating no association between the row and column variables and values close to 1
indicating a high degree of association between the variables. The maximum value possible depends on the number of rows and
columns in a table.); f (a chi-square based measure of association that involves dividing the chi-square statistic by the sample size and
taking the square root of the result); cramer’s V (a measure of association based on chi-square).
c
Goodman and Kruskal t: Similar to l; except the marginal proportions are used for predictions, rather than modal category.
d
Z is a measure of association that ranges from 0 to 1, with 0 indicating no association between the row and column variables and
values close to 1 indicating a high degree of association.

size for these tests was 15 (the number of pairwise
choices in the questionnaires), and so the power to
detect statistically significant correlations was low, and
so none of the values in the table below are statistically
significant. However, all are close to zero suggesting that
the attribute differences exhibit low associations. Since
there are a mixture of nominal and interval data, the
measures of association used varied and so are not
directly comparable with each other (see Table 6).
References
Bain, J., Gerrard, L., Russell, A., Locke, R., & Baird, V. (1997).
The Dundee out of hours co-operative: Preliminary out-
comes for the first year of operation. British Journal of
General Practice, 47, 573–574.
Bollam, M. J., McCarthy, M., & Modell, M. (1988). Patients’
assessment of out of hours care in general practice. British
Medical Journal, 296, 829–832.
Brunel University. (1993). The valuation of changes in quality in
the Public Services. Report prepared for HM Treasury,
HMSO, London.
Calnan, M., Katsoyiannopolous, V., Ouchavev, V. K., Pro-
khorskas, R., Ramic, H., & Williams, S. (1994a). Major
determinants of consumer satisfaction with primary care in
different countries. Family Practice, 11, 468–478.
Carr-Hill, R. (1992). The measurement of patient satisfaction.
Journal of Public Health Medicine, 14, 236.
Cragg, D., Campbell, S. M., & Roland, M. O. (1994). Out of
hours primary care centres: Characteristics of those attend-
ing and declining to attend. British Medical Journal, 309,
1627–1629.
Department of Health. (1998). GP out of hours services.
Working Group Report, Department of Health, Scottish
Office, Edinburgh.
Department of Health. (2000). Raising standards for patients.
New partnerships in out of hours care. An independent
review of GO out of hours services in England. Department
of Health, London.
Dixon, R. A., & Williams, B. T. (1988). Patient satisfaction with
general practitioner deputising services. British Medical
Journal, 297, 1519–1522.
Hansen, B. L., & Munck, A. (1998). Out of hours services in
Denmark. The effect of a structural change. British Journal
of General Practice, 48, 1497–1499.
Hallam, L., & Henthorne, K. (1999). Cooperatives and their
primary care emergency centres: organisation and impact.
Combined report on seven case studies. Health Technology
Assessment 3.
Louviere, J. (1988). Analysing decision making: Metric conjoint
analysis. California: Sage Publications Inc.
Manski, C. F. (1977). The structure of random utility models.
Theory and Decision, 8, 229–254.
McKinley, R. K., Cragg, D. K., Hastings, A. M., French, D. P.,
Mankau-Scott, T. K., Campbell, S. M., Van, F., Roland,
M. O., & Roberts, C. (1997a). Comparison of out of hours
care provided by patients’ own general practitioners and
commercial deputising services: A randomised controlled
trial. II: The outcome of care. British medical Journal, 314,
190–193.
McKinley, R. K., Mankau-Scott, T. K., Cragg, D. K.,
Hastings, A. M., French, D. P., & Baker, R. (1997b).
Reliability and validity of a new measure of patient
satisfaction with out of hours primary medical care in the
United Kingdom: Development of a patient questionnaire.
British Medical Journal, 314, 193–198.
McLaren, G., & Bain, M. (2000). Deprivation and health in
Scotland. Information and Statistics Division, Common
Services Agency, Edinburgh.
Morgan, A., Shackley, P., Pickin, M., & Brazier, J. (2000).
Quantifying patient preferences for out of hours primary
care. Journal of Health Services Research and Policy, 5,
214–218.
Morrison, J. M., Gilmour, H., & Sullivan, F. (1991). Children
seen frequently out of hours in one general practice. British
Medical Journal, 303, 1111–1114.
814 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814
Payne, J. W., Bettman, J. R., & Johnson, E. J. (1992).
Behavioural decision research: A constructive processing
perspective. Annual Review of Psychology, 43, 87–131.
Pearmain, D., Swanson, J., Kroes, E., & Bradley, M. (1991).
Stated preference techniques: A guide to practice. Steer
Davies Gleave, Hague Consulting Group.
Propper, C. (1988). Estimation of the value of time spent on NHS
waiting lists using stated preference methodology. Discussion
Paper 49, Centre for Health Economics, University of York.
Prudhoe, R. H. (1984). Deputising services. British Medical
Journal, 288, 718.
Ryan, M. (1999). Using conjoint analysis in health economics.
Paper presented to the International Health Economics
Association, Rotterdam.
Ryan, M., Scott, D. A., Bate, A., van Teijlingen, E., Russell, E.
M., Napper, M., Reeves, C., & Robb, C. (2001). Eliciting
public preferences for health care: a systematic review and
evaluation of methods. Health Technology Assessment, 5(5).
Salisbury, C. (1997). Postal survey of patients’ satisfaction with
a general practice out of hours co-operative. British Medical
Journal, 314, 1594–1598.
Salkeld, G., Ryan, M., & Short, L. (2000). The veil of
experience: do consumers prefer what they know best?
Health Economics, 9(3), 267–270.
San Miguel, F., Ryan, M., & Scott, A. (forthcoming). Testing
the assumptions of completeness and stability of preferences
using discrete choice experiments. Journal of Economic
Behaviour and Organisation, forthcoming.
Savage, R., & Armstrong, D. (1990). Effect of a general
practitioner’s consulting style on patient’s satisfaction: A
controlled study. British Medical Journal, 301, 968–970.
Sawyer, L., & Arber, S. (1982). Changes in home visiting and
night and weekend cover: The patient’s view. British
Medical Journal, 284, 1531–1534.
Scott, A. (1998). Giving things up to have more of others. The
implications of limited substitutability in eliciting prefer-
ences for health and health care. Discussion paper 01/98,
Health Economics Research Unit, University of Aberdeen,
1998.
Scott, A. (2000). Agency and incentives in general practice. Ph.D.
Thesis, University of Aberdeen.
Scott, A., & Smith, R. (1994). Keeping the customer satisfied:
Issues in the interpretation and use of patient satisfaction
surveys. International Journal for Quality in Health Care, 6,
353–359.
Scottish Out of Hours Study Group. (2001). A comparison of
models of delivery of out of hours general medical services in
Scotland. Final Report to Scottish Executive, Department
of General Practice, University of Edinburgh.
Smith, C. H., & Armstrong, D. (1989). Comparison of criteria
derived by government and patients for evaluating
general practitioner services. British Medical Journal, 299,
494–496.
Vedsted, P., & Olesen, F. (1999). Effect of re-organised after-
hours family practice service on frequent attenders. Family
Medicine, 31, 270–275.
Vick, S., & Scott, A. (1998). Agency in health care. Examining
patients’ preferences for attributes of the doctor–patient
relationship. Journal of Health Economics, 17, 511–644.
Williams, S. J., & Calnan, M. (1991). Key determinants of
consumer satisfaction with general practice. Family Prac-
tice, 8, 237–242.
Wissow, L. S., Roter, D. L., & Wilson, M. E. H. (1994).
Pediatrician interview style and mothers’ disclosure of
psychosocial issues. Pediatrics, 93, 289–295.
Wilson, P., McConnacchie, A., O’Donnell, C. A., Ross, S.,
Moffat, K. J., & Drummond, N. (2001). Assessing
dissatisfaction with an out of hours service: Reasons and
remedies. Health Bulletin, 59, 37–44.
Zwerina, K., Huber, J., & Kuhfeld, W. (1996). A general method
for constructing efficient choice designs. Working Paper,
Fuqua School of Business, Duke University.