Vol. 51 No.

3 March 2016 Journal of Pain and Symptom Management 609

Brief Methodological Report

Spanish Validation of the Care Evaluation Scale for Measuring the

Quality of Structure and Process of Palliative Care From the Family
Perspective
Miguel Angel Benitez-Rosario, MD, PhD, Raquel Caceres-Miranda, RN, and Armando Aguirre-Jaime, BS, MD,
on behalf of Grupo Espa~
nol de Investigaci
on en Cuidados Paliativos (GEICP)
Palliative Care Unit (M.A.B.-R., R.C.-M.), NS La Candelaria University Hospital, and Support Research Service (A.A.-J.) Primary Health
Care, Tenerife, Spain

Abstract
Context. A reliable and valid measure of the structure and process of end-of-life care is important for improving the
outcomes of care.
Objectives. This study evaluated the validity and reliability of the Spanish adaptation of a satisfaction tool of the Care
Evaluation Scale (CES), which was developed in Japan to evaluate palliative care structure and process from the perspective of
family members.
Methods. Standard forward-backward translation and a pilot test were conducted. A multicenter survey was conducted with
the relatives of patients admitted to palliative care units for symptom control. The dimensional structure was assessed using
confirmatory factor analyses. Concurrent and discriminant validity were tested by correlation with the SERQVHOS, a Spanish
hospital care satisfaction scale and with an 11-point rating scale on satisfaction with care. The reliability of the CES was tested
by Cronbach a and by test-retest correlation.
Results. A total of 284 primary caregivers completed the CES, with low missing response rates. The results of the factor
analysis suggested a six-factor solution explaining 69% of the total variance. The CES moderately correlated with the
SERQVHOS and with the overall satisfaction scale (intraclass correlation coefficients of 0.66 and 0.44, respectively; P ¼ 0.001).
Cronbach a was 0.90 overall and ranged from 0.85 to 0.89 for subdomains. Intraclass correlation coefficient was 0.88
(P ¼ 0.001) for test-retest analysis.
Conclusion. The Spanish CES was found to be a reliable and valid measure of the satisfaction with end-of-life care structure
and process from family members’ perspectives. J Pain Symptom Manage 2016;51:609e614 Ó 2016 American Academy of
Hospice and Palliative Medicine. Published by Elsevier Inc. All rights reserved.

Key Words
Family satisfaction, quality of care, quality indicators, palliative care audit, palliative care satisfaction, satisfaction scale

Introduction making. Hence, it is necessary to assess and monitor

In medical care and especially in palliative care, the
quality of life of patients and their families is greatly
influenced by the quality of the care dispensed.
However, patients with advanced cancer often do not
receive high-quality symptom control or support for
coordination of care, communication, and decision
the quality of care over time to ensure attention to
the goals of palliative care and the needs of
patients and their families. Quality care measurement
can be classified as pertaining to structure (environ-
ment in which health care is provided), process
(method by which health care is provided), and
outcome (consequences of health care).1,2 Patient

Address correspondence to: Miguel Angel Benitez-Rosario, MD, Accepted for publication: November 9, 2015.
PhD, Residencial Las Mercedes 34, 38200 La Laguna, Santa
Cruz de Tenerife, Spain. E-mail: mabenros@gmail.com

Ó 2016 American Academy of Hospice and Palliative Medicine. 0885-3924/$ - see front matter
Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jpainsymman.2015.11.002
610 Benitez-Rosario et al.
outcomes include symptom control, quality of life,
quality of death, and user’s perspective on the quality
of care.
A systematic review by Lorenz et al.3 showed that the
specific measurement of satisfaction with end-of-life
care supports a process-outcome relationship. More-
over, qualitative research has shown that patients and
families attribute great importance to satisfaction.3
Therefore, the assessment of patients’ and families’
satisfaction with care is a crucial aspect in the
evaluation of the effectiveness and quality of palliative
care.4 Although there is overlap between palliative
care satisfaction and general satisfaction domains,
there are also elements that are relatively unique to
this field.5 Several tools have been developed and
used in Spain and other countries to evaluate family
satisfaction with end-of-life care.4e9 However, to our
knowledge, there are no worldwide measurements of
satisfaction with palliative care validated in Spanish
patients and relatives.
The Care Evaluation Scale (CES) was developed in
Japan and used in Japan and Korea to assess the
need for improvements in the structure and process
of palliative care.10e14 It is a 28-item Likert-type scale
grouped into 10 subscales, namely physical care
(physicians and nurses), psycho-existential care, help
with decision making (for patients and family),
environment, family burden, cost, availability, and
coordination and consistency. Each item is scored on
a six-point Likert scale from ‘‘improvement is
highly necessary’’ to ‘‘improvement is not necessary
at all.’’ The scale showed excellent internal
consistency (Cronbach coefficient a ¼ 0.98) and
acceptable test-retest reliability (intraclass correlation
coefficient [ICC] ¼ 0.57) in the initial development
and validation study.10 Although the CES is not a
specific tool to measure family satisfaction, we
selected it because it evaluates several aspects of the
structure/process needed for quality end-of-life care.
Indeed, a huge advantage of assessments of user
satisfaction with the structure and process is that
they are less influenced by patient- and family-
related factors that health care providers cannot
change.
Consequently, the aim of this study was to adapt and
psychometrically validate the CES among relatives of
patients with advanced cancer in Spain.
Methods
Permission to use the tool was obtained from the
developers of the original scale. The English version
of the CES was translated into Spanish according to
the standard guidelines, involving forward and back-
ward translation.15,16 An English teacher, four medical
Vol. 51 No. 3 March 2016
professors, and six palliative care physicians collabo-
rated in the translation process. The Spanish
version of the tool was reviewed by the translator
team twice to obtain a definitive version. Finally, the
Spanish instrument was piloted in a convenience
subsample of 15 relatives of patients admitted to the
Palliative Care Inpatient Unit of Tenerife. All subjects
were asked to complete the questionnaire by
themselves, and then a structured interview was
performed to probe what they thought that each
questionnaire item meant and what their response
would be.
The psychometric properties of the CES were tested
in a multicenter study carried out in 11 Spanish palli-
ative care units: Tenerife, Las Palmas de Gran Canaria

(two sites), Fuerteventura, Pontevedra, Lleida, Alava,
Madrid, Barcelona, Badajoz, and Pamplona. These
sites were carefully selected, being limited to programs
offering specialist palliative care services with tertiary
palliative care units. At each site, the intention was
to enroll a consecutive sample of 30 relatives.
Eligibility criteria for participation were as follows:
age 18 years or older, primary family caregivers of
cancer patients admitted in the inpatient unit for
48 hours or more, capable of replying to a question-
naire, and gave verbal consent to participate into the
study. Relatives of the patients with a life expectancy
of three days or less were not enrolled. Questionnaires
were hand-delivered to relatives, and all subjects were
asked to complete the questionnaire by themselves.
The retest of the instrument was carried out with the
same relative 48e72 hours afterward if the patient’s
clinical condition remained stable. Ethical approval
was obtained from the Ethics Committee of Nuestra
Se~nora La Candelaria Teaching Hospital in Tenerife
and from the ethics committees of the other
participating sites when this was required by the local
guidelines.
In the absence of a Spanish tool for measuring
family satisfaction with palliative care that could be
used as a gold standard, we used the SERQVHOS17
and a scale of overall satisfaction for concurrent
validation of the CES. The SERQVHOS was developed
in Spain and evaluates the overall patient and family
satisfaction with hospital care. This instrument
consists of 19 items in two domains: subjective
satisfaction (related to professional care) and objec-
tive satisfaction (related to hospital care structure
and procedure). Each item is scored on a five-point
Likert scale from one (worst) to five (best). To
determine the overall satisfaction with the care
provided by the palliative care units, we used an
11-point numerical rating scale, ranging from 0 (worst)
to 10 (best), with the question: ‘‘Overall, were you
satisfied with the care received from the palliative
care unit?’’
Vol. 51 No. 3 March 2016 End-of-Life Care Structure and Process Satisfaction 611

Statistical Analysis Table 1

Data from the 11 sites were pooled and analyzed Characteristics of Caregiver Participants in the Study
aggregately because of the small sample size at each Characteristics Test n (%) Retest n (%)
site. For the analysis, each CES item was scored from Sex
one (least satisfaction) to five (most satisfaction), Men 80 (28.1) 66 (26.0)
Women 204 (71.9) 188 (74.0)
and the item scores were aggregated into a composite Relationship
score. Quantitative data were analyzed using Spouse 79 (33.0) 74 (29.1)
descriptive statistics. The construct validity of the Child 137 (43.4) 121 (47.6)
Sibling 21 (8.0) 18 (7.1)
CES was assessed by means of principal components Parent 22 (6.9) 20 (7.9)
factor analysis, followed by varimax rotation with Other 25 (8.2) 21 (8.4)
Kaiser normalization. Data suitability for this analysis Educational level
Bachelor’s degree 93 (32.7) 87 (34.2)
was checked by the Kaiser-Meyer-Olkin measure of Secondary school 109 (38.3) 95 (37.4)
sampling adequacy and by Bartlett’s sphericity test. Primary school 82 (28.8) 72 (28.4)
Criteria validity of the CES was estimated by conver- Age, yrs
mean (minemax) 50 (20e88) 50 (20e84)
gent strategy with the SERQVHOS and overall
satisfaction scales. ICCs of scores were estimated as a
measure of convergent validity and intrarater variance. They were as follows: 1) Factor 1 (questions
reliability. Internal consistency was measured using 1e12), related to biopsychological care, 2) Factor 2
Cronbach a coefficient. Test-retest reliability between (questions 13e15), related to the information given
items was evaluated by Cramer’s V test. All analyses and help with decision making for family; Factor 3
were done using SPSS, version 21 (IBM Corp., (questions 16e18), related to the quality of the
Armonk, NY). A P-value <0.05 was considered admission unit infrastructure; Factor 4 (questions 19
statistically significant. and 20), associated with help for the family burden;
Factor 5 (questions 21e23), related to availability for
admission; and Factor 6 (questions 24e26), related
Results to coordination and consistency. Factor loadings
The face validity of the CES was examined by the ranged from 0.56 to 0.72 (Table 2).
research team. The team considered the CES to be a
useful tool for assessing the level of the Spanish Feasibility and Valid Response Rate
family’s satisfaction with care on the palliative care The missing data rates for each item and valid
units. Two modifications (Appendix) were made to response rates for the floor (very much dissatisfied)
the questionnaire for its Spanish adaptation: 1) two and ceiling (very much satisfied) responses are
cost-of-care items were deleted because palliative described in Table 2. Missing data rates ranged from
care is free of charge in Spain and 2) the CES was 0% to 11%. The question related to ‘‘patient
adjusted to a five-point Likert scale (very much participation in the selection of the treatment’’ had
dissatisfied to very much satisfied). the highest rate of missing data, roughly 11%. The
question related to ‘‘room’s characteristics’’ had the
Validation Study Characteristics lowest rate of missing data.
A total of 284 primary caregivers participated in the
validation study, but the test-retest analysis was
undertaken with 254 relatives because of an error at
Concurrent Validity
The Spanish CES was correlated with the SERQV-
one participating center. The number of relatives
HOS (ICC ¼ 0.66; P ¼ 0.001) and with the overall
enrolled at each participating site ranged from six
satisfaction scale (ICC ¼ 0.44; P ¼ 0.001). Moreover,
(20% of the expected level) to 45 (150% of the
the overall satisfaction scale was correlated with the
expected level). The characteristics of the study
SERQVHOS (ICC ¼ 0.53; P ¼ 0.001).
participants are summarized in Table 1. The mean
age of family members interviewed was 50 years,
71.8% were women and 43.4% of participants were Reliability of the Spanish CES
the patients’ children. There were no significant Table 2 summarizes the Cronbach a coefficients for
differences in caregiver characteristics among the the items and ranged from 0.89 to 0.90. The Cronbach
participating sites. a coefficient was equal to 0.90 for the total scale and
0.85 to 0.89 for subdomains (Table 3).
Construct Validity Reliability test-retest analysis showed an ICC of 0.88
The results of the factor analysis suggested a (P ¼ 0.001) for total scores and ranged from 0.84 to
six-factor solution explaining 69% of the total 0.87 for subdomains (Table 3). However, reliability
612 Benitez-Rosario et al. Vol. 51 No. 3 March 2016

Table 2
Psychometric Characteristics of the CES Spanish Version
Construct Validity

Item Missing, n (%) Floor Response, n (%) Ceiling Response, n (%) Factor Factor Loading Internal Consistencya Reliabilityb,c

Q1 3 (1.1) 1 (0.4) 221 (77.8) 1 0.58 0.892 0.73

Q2 1 (0.4) 0 (0.0) 217 (76.4) 0.68 0.891 0.76
Q3 6 (2.1) 1 (0.4) 208 (73.2) 0.66 0.891 0.72
Q4 4 (1.4) 0 (0.0) 212 (74.6) 0.61 0.893 0.67
Q5 6 (2.1) 0 (0.0) 208 (73.2) 0.58 0.895 0.55
Q6 29 (10.3) 17 (6.0) 105 (37.0) 0.66 0.891 0.65
Q7 9 (3.2) 0 (0.0) 190 (66.9) 0.71 0.891 0.59
Q8 27 (9.6) 4 (1.4) 164 (57.7) 0.69 0.890 0.63
Q9 15 (5.3) 0 (0.0) 190 (66.9) 0.68 0.890 0.63
Q10 18 (6.3) 3 (1.1) 186 (65.5) 0.56 0.895 0.53
Q11 3 (1.1) 28 (9.9) 158 (55.5) 0.60 0.891 0.51
Q12 32 (11.3) 14 (4.9) 103 (36.3) 0.53 0.891 0.56
Q13 1 (0.4) 2 (0.7) 214 (75.4) 2 0.60 0.893 0.54
Q14 1 (0.4) 4 (1.4) 205 (72.2) 0.55 0.895 0.51
Q15 5 (1.8) 39 (13.7) 178 (62.7) 0.51 0.896 0.58
Q16 0 (0.0) 8 (2.8) 135 (47.5) 3 0.61 0.905 0.68
Q17 4 (1.4) 8 (2.8) 101 (35.6) 0.60 0.906 0.76
Q18 1 (0.4) 7 (2.5) 109 (38.4) 0.51 0.906 0.61
Q19 28 (9.9) 6 (2.1) 142 (50.0) 4 0.69 0.891 0.63
Q20 29 (10.3) 13 (4.6) 100 (35.2) 0.55 0.895 0.63
Q21 4 (1.4) 3 (1.1) 183 (64.4) 5 0.53 0.892 0.59
Q22 6 (2.1) 18 (0.4) 191 (67.3) 0.52 0.892 0.62
Q23 8 (2.8) 3 (1.1) 196 (69.0) 0.51 0.893 0.61
Q24 4 (1.4) 0 (0.0) 196 (69.0) 6 0.68 0.891 0.66
Q25 18 (3.7) 2 (0.7) 171 (60.2) 0.71 0.891 0.58
Q26 12 (4.2) 0 (0.0) 189 (66.5) 0.59 0.892 0.55
CES ¼ Care Evaluation Scale.
a
Cronbach a coefficients.
b
Cramer’s V coefficients.
c
P < 0.001 for all coefficients.

test-retest for every item was lower; the respective
Cramer’s V was 0.51 to 0.76 (Table 2).
Discussion
In this prospective multicenter study, we evaluated
the psychometric properties of the Spanish version
of the CES to measure family satisfaction with end-
of-life care structure and process. Factor analysis
confirmed the construct validity, and the tool showed
adequate concurrent and discriminant validity with
good reliability.
The CES was easily administered and completed
by family members, as shown by the relatively few
missing items. Missing rates were the highest for
Q6, Q8, Q12, and Q20, roughly 10%. These items
asked about themes related to nonusual activities
of palliative care units in Spain. Thus, missing rates
for items Q6, Q8, and Q20 can result from
deficiencies in psychotherapy, music therapy, art
therapy, or other complementary therapies in
Spanish palliative care units. Nevertheless, we did
not remove these items because of the potential
utility of the scale to evaluate the impact of
insufficiency of several types of interventions, such
as the aforementioned therapies, on the quality of
end-of-life care.
Reliability was established by excellent internal
consistency, slightly lower than that reported in the
original Japanese version10 (0.97 for total; 0.87e0.95
for subscales) and Korean adaptation14 (0.97 for total;
0.88e0.94 for subscales). The good reliability also was
confirmed by the temporary stability of the tool score
and subscales; however, the temporary stability for
each item was lower, with significant Cramer’s V values
between low and moderate range. To our knowledge,
the temporary stability of the CES has not been
previously evaluated.
Confirmatory factor analysis confirmed our hypoth-
esis regarding the underlying constructs of the CES. A
six-factor solution proved to be the most adequate for
our sample instead of the ten factors obtained with
both the original and Korean versions. The subscales
of the original tool for physical and psycho-
existential care were combined into one in the
Spanish CES, designated the biopsychological care
subscale. We decided to consider the first factor as
biopsychological care because the items involved are
not usually related to existential care in the Spanish
culture.18 The cost subscale is not present in the
Spanish CES because all items were removed from
the tool.
Limitations of this study were the concurrent and
discriminant validity evaluation of the CES because
Vol. 51 No. 3 March 2016 End-of-Life Care Structure and Process Satisfaction
Table 3
Reliability of the Spanish Version of CES
Dimension
Biopsychosocial care by medical and nurses
Information given and help with decision making for family
Quality of admission unit infrastructure
Help to family burden
Availability for admission
Coordination and consistency
CES ¼ Care Evaluation Scale; ICC ¼ intraclass correlation coefficient.
there are no Spanish gold standard scales to measure
family satisfaction with end-of-life care structure and
process. We used the SERQVHOS, which evaluates
hospital care satisfaction, and an 11-point ‘‘overall
satisfaction’’ scale as a methodological approximation.
The CES was only moderately associated with
satisfaction levels obtained with the SERQVHOS and
slightly associated with satisfactions levels reported in
the 11-point scale. These findings indicate both the
concurrent validity of the CES with tools that assess
hospital care structure and process satisfaction and
with discriminant validity with respect to scales of
overall satisfaction with palliative care. Moreover, the
results emphasize that satisfaction assessment in this
field should be carried out with specific tools.
Our results are consistent with the findings of
Morita et al.10 and show that the CES is a good
indicator of quality improvement because it denotes
the family satisfaction with palliative care structure
and process independently of family satisfaction with
the care. However, the Spanish version of CES was
used with an important modification from the
original. It was adapted to a satisfaction tool, and the
items were evaluated by the caregiver according to
satisfaction criteria instead of the ‘‘need for improve-
ment’’ criteria used in the original scale.
Another important change to the original was that
the Spanish CES was validated with relatives of
patients with advanced cancer who were admitted to
inpatient palliative care units for symptom control.
Our findings show that the CES can be used as a
quality measuring tool to detect current needs for
improvement in palliative care structures/processes
and to undertake the necessary improvements over a
short period. The use of the CES before patient death
avoids some methodological problems in measuring
the satisfaction of bereaved relatives, as there is a
subjectivity bias in retrospectively collected informa-
tion. However, the CESS does not preclude poor
discrimination in satisfaction levels secondary to
reluctance to criticize care provided.5,19
Understanding the key aspects of palliative care
through family satisfaction is important to evaluate
and improve care. Although there is no gold standard
instrument,5 it appears that development of new
613
Internal Consistency Cronbach a Test-Retest Reliability, ICC (95% CI)
0.85 0.88 (0.81e0.90)
0.87 0.84 (0.80e0.89)
0.86 0.87 (0.81e0.90)
0.89 0.87 (0.84e0.90)
0.89 0.85 (0.80e0.88)
0.87 0.86 (0.81e0.89)
instruments that focus on this area of enquiry is not
warranted. One good methodological option would
be to adapt available tools to build a pack of
instruments for valid cross-cultural and demographic
subgroup comparisons. For this, the conceptual and
psychometric measurement equivalence of scales
should be researched. To our knowledge, this is the
first tool for satisfaction evaluation that has been
adapted and validated in Spanish relatives of advanced
cancer patients. Further research should adapt and
validate this and other tools in different countries, to
select the best scale to learn the family’s perspective
on the quality of palliative care. Additionally, more
worldwide studies are needed to establish which
factors are essential in the levels of patients and
families satisfaction with palliative care.
In conclusion, our findings showed that the CES is a
reliable and valid instrument for measuring satisfac-
tion with the structure and process of end-of-life
care from the Spanish family’s perspective.
Disclosures and Acknowledgments
Researchers of GEICP: Cesar Labrador (Hospital de
Pontevedra); Lydia Beltra-Espulgas and Yliana
Topham-Alvarez (Hospital Insular, Las Palmas
de Gran Canaria); Josep Porta-Sales and Silvia
Llorens-Torrome (Institut Catala d’Oncologia-
Bellvitge, Barcelona); Juan Pedro Arbizu, Jaione
Azparren and Marcos Lama-Gay (Hospital San Juan
de Dios, Pamplona); Marı́a Jose Almaraz-Ruiz de
Eguı́laz, Arantza Pinedo-Asarta and Elena
Abascal-Ortega (Hospital Universitario Araba); Car-
men Batista-Rodrı́guez, Nuria Toledo-Cabrilla and Fa-
tima Marrero-Ojeda (Hospital Dr Negrı́n, Las Palmas
de Gran Canaria); Alberto Alonso-Babarro (Hospital
Universitario La Paz, Madrid); Marı́a Jose Redondo-
Moralo and Rafael Mota-Vargas (Equipo de Cuidados
Paliativos, Badajoz); Marı́a Nabal-Vicu~
na and Maria-
Teresa Jubero-Sancho (Hospital Universitario Arnau
de Vilanova, Lleida); Lucas Caballero (Hospital de
Fuerteventura); and Cristobalina Rodrı́guez-Alvarez,
Carmen Rijo and Angeles Arias (Area Medicina
Preventiva y Salud P ublica, Universidad La Laguna,
614 Benitez-Rosario et al.
Tenerife). This research received no specific funding/
grant from any funding agency in the public, commer-
cial, or not-for-profit sectors. The authors declare no
conflicts of interest.
References
1. Seow H, Snyder CF, Mularski RA, et al. A framework for
assessing quality indicators for cancer care at the end of life.
J Pain Symptom Manage 2009;38:903e912.
2. De Roo ML, Leemans K, Claessen SJ, et al. EURO
IMPACT. Quality indicators for palliative care: update of a
systematic review. J Pain Symptom Manage 2013;46:
556e572.
3. Dy SM, Shugarman LR, Lorenz KA, Mularski RA, Lynn J.
RAND-Southern California Evidence-Based Practice Center.
A systematic review of satisfaction with care at the end of
life. J Am Geriatr Soc 2008;56:124e129.
4. Lorenz K, Lynn J, Morton SC, et al. End-of-life care
and outcomes. Summary, evidence report/technology
assessment: number 110. AHRQ Publication No. 05-E004-1.
Rockville, MD: Agency for Healthcare Research and Quality,
2004.
5. Hudson PL, Trauer T, Graham S, et al. A systematic
review of instruments related to family caregivers of
palliative care patients. Palliat Med 2010;24:656e668.
6. Santacruz Saura V, Carbonell Riera JM. Evaluaci on de la
satisfacci
on de los usuarios en Cuidados Paliativos [Evalua-
tion of the user satisfaction on palliative care]. Medicina pal-
iativa 1999;6:48e55.
7. Molina Linde JM, Romero Cotelo J, Romero
Rodrı́guez Y, Rodrı́guez Sa~ nudo RM, Galvez Mateos R.
Encuesta de satisfacci
on a cuidadores en una Unidad de
Cuidados Paliativos [Relatives satisfaction at a palliative
care]. Medicina paliativa 2006;13:11e17.
8. Almaraz MJ, Aldasoro E, Sobradillo N. Estudio de
satisfacci
on y de identificaci
on de factores de la atenci on
hospitalaria que los familiares de personas fallecidas consid-
eran importantes en la fase final de la vida [Satisfaction and
factors related study of end-life patient relatives]. Inves-
tigaci
on Comisionada. Vitoria-Gasteiz: Departamento de Sa-
nidad. Gobierno Vasco, 2006. Informe no Osteba D-06-04.
9. Palomar C, Jimenez A, Barallat E, et al. Satisfacci
on de
los familiares atendidos por un equipo de soporte en un
hospital universitario [Family satisfaction with palliative
Vol. 51 No. 3 March 2016
care team at teaching hospital]. Medicina paliativa 2009;
16:328e333.
10. Morita T, Hirai K, Sakaguchi Y, et al. Measuring the
quality of structure and process in end-of-life care from
the bereaved family perspective. J Pain Symptom Manage
2004;27:492e501.
11. Miyashita M, Morita T, Hirai K. Evaluation of end-of-life
cancer care from the perspective of bereaved family
members: the Japanese experience. J Clin Oncol 2008;26:
3845e3852.
12. Yamagishi A, Morita T, Miyashita M, et al. Pain intensity,
quality of life, quality of palliative care, and satisfaction in
outpatients with metastatic or recurrent cancer: a Japanese,
nationwide, region-based, multicenter survey. J Pain
Symptom Manage 2012;43:503e514.
13. Miyashita M, Morita T, Sato K, Tsuneto S, Shima Y.
A nationwide survey of quality of end-of-life cancer care in
designated cancer centers, inpatient palliative care units
and home hospices in Japan: the J-HOPE study. J Pain
Symptom Manage 2015;50:38e47. e3.
14. Shin DW, Choi JE, Miyashita M, et al. Measuring the
structure and process of end-of-life care in Korea: validation
of the Korean version of the Care Evaluation Scale (CES).
J Pain Symptom Manage 2012;44:615e625.
15. Beaton DE, Bombardier C, Guillemin F, Marcos B.
Guidelines for the process of crosscultural adaptation of
self-report measures. Spine 2000;25:3186e3191.
16. Dewolf L, Koller M, Velikova G, et al. for EORTC Quality
of life group. EORTC quality of life group translation
procedure, 3rd ed. Brussels: EORTC, 2009. Available at:
http://groups.eortc.be/qol/downloads/translation_manual_
2009.pdf. Accessed May 9, 2015.
17. Mira JJ, Aranaz J, Rodrı́guez-Marı́n J, et al. SERVQHOS:
un cuestionario para evaluar la calidad percibida de la
asistencia hospitalaria [SERVQHOS: a satistaction tool to
assess hospital care]. Medicina Preventiva 1998;IV:12e18.
18. Benito E, Barbero J, Dones M. Espiritualidad en clı́nica.
Una propuesta de evaluaci on y acompa~ namiento espiritual
en cuidados paliativos [Spirituality in palliatve care. Spanish
Association of Palliative Care. Madrid, SECPAL, 2014]. SEC-
PAL, 2014.
19. Aoun S, Bird S, Kristjanson LJ, Currow D. Reliability
testing of the FAMCARE-2 scale: measuring family carer
satisfaction with palliative care. Palliat Med 2010;24:
674e681.
Vol. 51 No. 3 March 2016 End-of-Life Care Structure and Process Satisfaction 614.e1

Appendix

Spanish and Original English Versions of CES

Q1dLos m edicos intentaron aliviar las molestias fı́sicas del paciente (Doctors tried to relieve physical discom-
fort of the patient).
Q2dLos m edicos se ocuparon rapidamente de las molestias (sı́ntomas, problemas fı́sicos) del paciente (Doc-
tors dealt promptly with discomforting symptoms of the patient).
Q3dLos m edicos tenı́an los conocimientos y habilidades (pericia) adecuados (Doctors had adequate knowl-
edge and skills).
Q4dLas enfermeras/enfermeros respondieron rapidamente a las quejas/peticiones (llamadas) del paciente
(Nurses responded promptly to the patient’s needs (e.g., nurse calls)).
Q5dLas enfermeras/enfermeros tenı́an los conocimientos y habilidades adecuados (Nurses had adequate
knowledge and skills).
Q6dLas enfermeras/enfermeros ayudaron al paciente a disfrutar de su vida diaria durante el ingreso (entrete-
nimientos, m usica, pasatiempos) (Nurses helped the patient to enjoy daily life (recreation, music, and hobbies).
Q7dSe prest o atencion a resolver las dudas y preocupaciones del paciente (Consideration was paid to relieving
the patient’s concerns and worries).
Q8dEl equipo asistencial (m edicos/enfermerasdenfermeros) tomaron/aplicaron las medidas adecuadas cuan-
do el paciente se sintio deprimido (The staff took appropriate measures when the patient became depressed).
Q9dEl equipo de trabajo (m edicos/enfermeras - enfermeros) intent o satisfacer las expectativas del paciente
(The staff tried so that the patient’s hope could be accomplished).
Q10dLos m edicos dieron al paciente una explicaci on suficiente acerca de su situaci on actual y de los detalles
de su tratamiento (The doctors gave sufficient explanation to the patient about their present condition and the
details of medical treatment).
Q11dLos m edicos dieron una explicaci on suficiente al paciente acerca del resultado previsible (The doctors
gave sufficient explanation to the patient about the expected outcome).
Q12dSe invit o y favoreci
o que el paciente participara en la elecci on del tratamiento (Consideration was given
so that the patient could participate in the selection of treatment).
Q13dLos m edicos dieron una explicaci on suficiente a la familia acerca de la situaci on del paciente y de los
detalles de su tratamiento m edico (The doctors gave sufficient explanation to the family about the patient’s con-
dition and the details of medical treatment).
Q14dLos m edicos dieron suficiente explicaci on a la familia acerca de los resultados esperados (The doctors
gave sufficient explanation to the family about the expected outcome).
Q15dSe respetaron los deseos de la familia en la elecci on del tratamiento (The family’s wishes were respected
in the selection of treatment).
Q16dLa habitaci on del paciente era apropiada y confortable (The patient’s room was convenient and
comfortable).
Q17dLas medidas de aislamiento del ruido fueron adecuadas (Sound proofing measures were adequate).
Q18dLos servicios/ba~ nos eran adecuados (Toilet and washstand facilities were adequate).
Q19dSe prest o atenci
on a la salud de la familia (Consideration was given to the health of the family).
Q20dSe tuvo en cuenta que la familia pudiera disponer de su tiempo y continuara con su vida cotidiana
(Consideration was given so that the family could have their own time and continue to work).
Q21dCuando fue necesario el ingreso se realiz o sin demora (Admission (use) was possible when necessary
without waiting).
Q22dLos tramites del ingreso fueron sencillos (The procedures of admission (use) were simple).
Q23dEl ingreso se realiz o de acuerdo con los deseos del paciente y su familia (Admission (use) was in accor-
dance with the wishes of the patient and family).
Q24dla coordinaci on/cooperaci on entre los medicos y enfermeras del servicio fue buena (There was good
cooperation among staff members such as doctors and nurses).
Q25dLa atenci on fue realizada por los mismos medicos y enfermeros-as (The same doctors and nurses pro-
vided care).
Q26dEl tratamiento fue establecido en relaci on al curso/caracterı́sticas previas de la enfermedad (Treatment
was planned with appropriate consideration of the previous course of the disease).