PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 54th ANNUAL MEETING - 2010 850

MEASURING HOSPITAL SAFETY CULTURE: TESTING THE

HSOPSC SCALE
Cakil Sarac1 *, Rhona Flin1, Kathryn Mearns1, Jeanette Jackson1
1
Industrial Research Centre, School of Psychology, University of Aberdeen, Aberdeen, Scotland, UK

*Corresponding author: Industrial Research Centre, School of Psychology, University of Aberdeen,

Aberdeen AB24 2UB, Scotland, UK. E-mail: cakilsarac@abdn.ac.uk

ABSTRACT

As part of an organization’s safety management strategy, it is important to assess the level of safety climate
with a valid instrument. The aim of this study is to investigate the psychometric properties of the Hospital
Survey on Patient Safety Culture (HSOPSC) based on a sample of clinical staff from Scottish acute
hospitals and to determine the suitability for Scottish healthcare setting. The data were collected from 1966
clinical staff (estimated 22% response rate) from one acute hospital from each of seven Scottish Health
Boards. In order to test the psychometric properties of the questionnaire, a split-half cross-validation
technique was used. The data were randomly split into two, and Exploratory (EFA) and Confirmatory factor
analyses (CFA) were conducted on the calibration and validation data sets to investigate and check the
original US model fit from a Scottish sample. EFA results showed a 10 factor optimal measurement model.
The CFA were then performed to compare the model fit of two alternative models (10 factor alternative
model vs. 12 factor original model). It was demonstrated that both factor structures performed equally well
in a Scottish sample. Furthermore, reliability analyses of each component yielded satisfactory results.
Therefore no modifications are required to the original 12 factor model which is suggested for use since it
would allow researchers the possibility of cross-national comparisons.

BACKGROUND Netherlands (Smits, et al. 2008), Switzerland (Pfeiffer, et al.

Copyright 2010 by Human Factors and Ergonomics Society, Inc. All rights reserved. 10.1518/107118110X12829369606369

2008), and Belgium (Hellings, et al. 2007) and has been

Organizational safety culture has been shown to have
a significant influence not only on worker injuries - as well
illustrated in industrial research (Hofmann & Mark, 2006;
Neal & Griffin, 2006) - but also on patient safety outcomes
such as treatment and medication errors, and patient falls
(Naveh, Katz-Navon & Stern, 2005; Vogus & Sutcliffe, 2007;
Zohar, Livne, Tenne-Gazit, Admi, & Donchin, 2007). Safety
climate questionnaires have been widely used in this research
to get a snapshot of the prevailing safety culture (Mearns, Flin,
Fleming & Gordon, 1997) in an organization and therefore the
term safety culture and safety climate are often used
interchangeably in the literature (Cox & Flin, 1998). A number
of different instruments have been developed within healthcare
(Sexton et al. 2006; Neal, Griffin & Hart, 2000; Singer et al.,
2007; Vogus & Sutcliffe, 2007) and these have also been
adapted to assess safety culture within different countries.
An earlier review of some of these instruments found
a lack of theoretical framework and the psychometric
properties of most of the scales had not been assessed (Flin,
Mearns, Yule & Robertson, 2006). At that time, one notable
exception was the Hospital Survey on Patient Safety Culture
(HSOPSC) developed by the Agency of Healthcare Research
and Quality (AHRQ: Nieva & Sorra, 2003;
http://www.ahrq.gov). Although the instrument is of US origin,
it has now been widely used within a number of European
countries; including Norway (Olsen, 2008), England
(Waterson, Griffiths, Stride, Murphy & Hignett, 2010),
translated into 16 languages (Battles et al., 2009). Results
revealed inconsistent factor structures. The original 12 factor
structure was replicated in both the Norwegian and Belgian
samples. However, Dutch and English studies reported 10 and
9 factors respectively. Considering the disparate
methodologies used in implementing the instrument, namely;
the samples in the studies (clinical versus non-clinical staff) or
the analysis techniques, it is not entirely surprising to find that
these studies yield inconsistent results.
The main aim of this paper was to investigate the
psychometric properties of the HSOPSC in a Scottish
healthcare setting. If the validity and the reliability of the
instrument are found to be satisfactory, it can be used
nationwide to assess the level of safety climate regularly and
also to benchmark the data with other European countries.
Theoretically, a valid and a reliable instrument would allow
examination of the underlying causal mechanisms between
safety climate and safety performance (Christian, Bradley,
Wallace & Burke, 2009), and this would strengthen efforts to
improve patient safety.

Downloaded from pro.sagepub.com at University of Birmingham on June 13, 2015

PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 54th ANNUAL MEETING - 2010 851

METHOD
Table1. Demographics – Occupational Group
Procedure Occupational Group Frequency Percentage
Allied Health Professionals 412 21.0
The 14 NHS Health Boards in Scotland were Medical and Dental 230 11.7
contacted and invited to provide an acute hospital for the Registered nurse and Midwives 1013 51.5
study. If a Board had more than one acute hospital, then the Nursing or Healthcare Assistants 252 12.8
Not reported 59 3.0
research team suggested which site would enable them to have Total 1966 100
the correct mix of teaching/ non-teaching, large/ small
hospitals but the final decision was given to the Board in order Measure
to accommodate recent/ ongoing staff surveys, other research
studies, operational activities and logistical constraints. Selection of the Instrument: The reasons for
Overall, nine Boards agreed to be included in the sample. employing HSOPSC are as follows. At the time of selection,
Later, one Board decided to withdraw from the study due to a this questionnaire was found to be the most rigorously
concurrent survey at their acute hospital and in another Board, designed in the literature (Flin, et al. 2006). Since it has been
the response rate from the hospital was too small to include in used extensively in the USA with hundreds of hospitals, it
the analysis provides benchmark data and, in addition there are now new
The procedure for administration of the reports from studies of hospitals in several European countries
questionnaires was determined in conjunction with each which provide results for cross-cultural comparisons. Also
participating Board or hospital. Both paper based and web- recently, the report Promoting Patient Safety Culture
based surveys (SNAP 9) were made available. Paper published by the European Society for Quality in Healthcare
questionnaires were provided with a covering letter and with (2010) stated the extensive use of the HSOPSC within the
sealable envelopes for return to the University team or to the Member States. Practically, since the instrument covers a wide
collection point within the hospital unit. No names were range of factors, it allows the researchers to evaluate the
requested to enhance anonymity. current state of safety culture in a number of different areas.
Ethical Approval: Prior to data collection, ethical Development of HSOPSC was based on a
approval for the study was first obtained from the University comprehensive literature review, including both published and
Ethics Committee (which conforms to British Psychological unpublished safety culture surveys, in addition to interviews
Society ethical guidelines). Advice obtained from the NHS with hospital staff. The instrument was then pilot tested on
North of Scotland Research Ethics Service was that this study 1437 hospital employees from 21 hospitals in US (Sorra &
was a Service Evaluation and therefore would not require an Nieva, 2004). The final set of 42 HSOPSC items (17 reverse
NRES ethics application. The project plan was also approved coded) showed a 12 factor structure, 10 safety climate scales at
in 2008 by the Scottish Patient Safety Alliance Advisory unit and hospital levels plus two outcome variables. The scales
Group and Steering Group and the Scottish Workforce and with an example item are presented below:
Staff Governance group.
Unit level
Sample • Supervisor/manager expectations and actions promoting
patient safety (4 items) - My supervisor/manager seriously
The sample was drawn from seven large and small considers staff suggestions for improving patient safety
acute NHS hospitals in Scotland, between February and • Organizational learning-Continuous improvement (3 items)
September 2009. Regarding the results of previous studies - Mistakes have led to positive changes here
conducted with HSOPSC (that some items might not be fully • Teamwork within units (4 items) - When one area in this
suitable for non-clinical staff), it was decided to include only unit gets really busy, others help out
clinical staff in the study. A total of 1969 clinical staff (in • Communication openness (3 items) - Staff are afraid to ask
direct contact with patients) filled in the questionnaire with an questions when something does not seem right
estimated 22% response rate. Although the number of • Feedback and communication about error (3 items) - We
questionnaires sent to each participating hospital were known, are informed about errors that happen in this unit
it was not clear how many questionnaires actually were
• Non-punitive response to error (3 items) - Staff feel like
distributed within each unit. Also, three participants were
their mistakes are held against them
excluded from the overall sample since they were found to
• Staffing (4 items) - We have enough staff to handle the
complete less than half of all items. Consequently, the total
workload
sample size was reduced to seven hospitals and 1966
Hospital level
participants. The majority of the sample was nurses (50%)
• Hospital management support for patient safety (3 items) -
followed by Allied Health Professionals (21%) and doctors
The actions of hospital management show that patient
(14%) (see Table 1).
safety is a top priority
• Teamwork across hospital units (4 items) - Hospital units
do not coordinate well with each other

Downloaded from pro.sagepub.com at University of Birmingham on June 13, 2015

PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 54th ANNUAL MEETING - 2010
• Hospital handoffs and transitions (4 items) - Important
patient care information is often lost during shift changes
Outcome measures
• Overall perceptions of patient safety (4 items) - Patient
safety is never sacrificed to get more work done
• Frequency of event reporting (3 items) - When a mistake is
made that could harm the patient, but does not, how often
is this reported?
There are also two single item outcome measures (Patient
safety grade and Number of events reported) which were not
included in the factor analyses.
Adjustments: The HSOPSC was originally designed
for USA healthcare staff. Therefore it required customization
for a Scottish sample, as has been necessary in the other
European studies. Firstly, in order to check for the usability of
the instrument within a Scottish sample, given American
terminology, 10 interviews were conducted with healthcare
staff (including consultants, nurses, a pharmacist, a risk
manager and occupational psychologists). The participants
indicated that the main question set covered necessary areas
and also confirmed the suitability and relevance of the
instrument. However, considering demographic questions, job
codes and event reporting items were found to be inadequate
for Scotland. Therefore, the job titles section was replaced
with the relevant section of the NHS staff survey used in
England and Wales (Aston Business School, 2007) due to the
difficulty of rewording American job titles for NHS staff. In
addition to job titles, the term “event” was changed to
“incident” in order to adjust the terminology to Scottish
healthcare. Although the inadequacy of the “event reporting”
items were mentioned within the interviews, the dimension as
a whole was retained in the questionnaire in order to keep the
instrument comparable to its original version which might have
had further statistical implications for the psychometric
properties.
Statistical Analysis
Validity and reliability tests were carried out in order
to check the psychometric properties of HSOPSC on a Scottish
sample. Validity refers to “whether an instrument measures
what it was designed to measure” (Field, 2009, p.11). In order
to test the validity of the instrument, a split-half cross-
validation technique was used (Tabachnick & Fidell, 2007).
Cross-validation is performed by testing the factorial structure
on two independent samples. Therefore, SPSS 17.0 was used
to split the sample randomly into two independent groups.
Given the contradictory findings of the HSOPSC factor
structure discussed above, Exploratory factor analysis (EFA)
was first carried out on the calibration half of the data set
(Sample I, n = 965) to investigate the suitability of the 12
original factor model, as well as to identify an alternative
model which might fit better the data from the Scottish sample.
Subsequently, Confirmatory factor analysis (CFA) with
Maximum Likelihood Method using EQS (Byrne, 1994) was
performed on the validation half of the data set (Sample II, n =
1001) to test how well the two models (original 12 factor and
Downloaded from pro.sagepub.com at University of Birmingham on June 13, 2015
852
an alternative model) fit the Scottish data. Reliability was also
tested using Cronbach’s alphas, and inter-correlations were
examined for the whole sample to determine the discriminant
validity (i.e., correlations of unit and hospital level dimensions
with outcome measure).
RESULTS
Following splitting the data randomly into two halves,
EFA was conducted (n = 965) on the randomly selected
calibration half of the data set with 42 items to investigate the
factor structure on the Scottish sample. The Kaiser-Meyer
measure verified the sampling adequacy for the analysis
(KMO = .92). Also Bartlett’s test of sphericity (χ2 = 12808.9,
df = 861, p<.001) indicated that correlations between items
were sufficiently large to conduct EFA.
By using the Kaiser’s criterion and Cattell’s screen
plot as an extraction method, different numbers of factors were
tested to find the best factor structure. The final 10 factor
version appeared to be the best solution after the exclusion of
three items (Things “fall between the cracks” when
transferring patients from one unit to another(F3), Our
procedures and systems are good at preventing errors from
happening(A18), and It is often unpleasant to work with staff
from other hospital units(F6)). These items were excluded
based on following criteria: (1) no item loading on any of the
factors, (2) high cross loading, or (3) low item loading on a
different factor. The total 10 factor model accounted for
61.9% of the total variance in line with the literature (Hellings,
et al. 2007; Smits, et al. 2008; Waterson, et al. 2010).
Accordingly, the 10 factor structure was found to
differ from the original 12 factor structure as follows. Firstly,
Feedback and communication about error and
Communication Openness became one factor, and Staffing and
Overall perceptions about safety scales merged as one factor
(Similar to the English sample). Additional two EFAs were
conducted separately with only the items of the merging scales
to explore the factor loadings in detail and confirmed the
results. Therefore, 10 factor structure was decided to be the
optimal measurement solution.
Subsequently, the model fit of the original 12 factor
vs. alternative 10 factor structures were tested with CFA (using
EQS 6.1 for Windows) on the validation half of the data set.
Two measures of fit were employed; Comparative Fit Index
(CFI); values >.90 indicating an acceptable fit, >.95 indicating
a good fit (Bentler, 1990), and the Root Mean Square Error
Approximation (RMSEA) values < .05 considered as a
indication of a good model fit to the data (Kline, 2005).
The original 12 factor with 42 items (chi-square =
1708.9, df = 753) showed an adequate fit to the data (CFI=
0.91, RMSEA = 0.04) with standardized factor loadings
ranging from .39 to .89. Later, the alternative 10 factor
solution with 39 items was tested. This 10 factor model (chi-
square = 1554.5, df = 657) was found to be an acceptable fit to
the data (CFI= 0.91, RMSEA = 0.04) similar to the original 12
factor structure. The standardized factor loadings were found
to range from .35 to .89.
PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 54th ANNUAL MEETING - 2010 853

Reliability (internal consistency) was also tested for are required. Although the 10 factor model is more
each of the 12 factors on the whole data set. The Cronbach’s parsimonious in nature, no cross-national comparisons would
alpha scores ranged from .70 to .84 indicating satisfactory be possible with a modified version.
internal consistencies with two exceptions: Organizational Cronbach’s alpha scores also showed satisfactory
learning and continuous improvement (α = .64) and Staffing (α internal consistency of the scales with two exceptions:
= .60) dimensions. The same scales had lower alpha scores in Organizational Learning and Continuous Improvement and
the comparison data sets from other countries (See Table 2). Staffing. Similar findings for these scales were reported for the
US and European data.
Table 2. Cronbach’s alpha scores for Scotland (SCT), However, it is crucial to underline that two scales of
USA, Netherlands (NL), Norway (NOW), and England the instrument included in the CFA are labeled as outcome
(ENG) variables (Overall perceptions of safety and Incident
Scales/ No of Alpha Alpha Alpha Alpha Alpha reporting). Theoretically, the legitimacy of the two separate
items SCT US NL NOW ENG constructs (unit and hospital level safety culture dimensions vs.
Unit level
Supervisor
outcomes) should be examined using additional CFA.
0.79 0.75 0.70 0.77 0.68 Furthermore, although there is published literature reporting
Expectations
Organizational
0.64 0.76 0.57 0.51 0.66
the psychometric properties of the HSOPSC across nations, to
Learning date, no research has been related the instrument to objective
Teamwork
WITHIN units
0.80 0.80 0.66 0.77 0.73 outcome measures such as actual incident reporting data. Such
Comm. an investigation of safety culture dimensions against an
0.73 0.72 0.72 0.68 0.67
Openness objective measure of patient safety is required to test the
Feedback predictive validity of the instrument. Since healthcare involves
Comm. about 0.78 0.78 0.75 0.70 0.80
risks of harm for both patients and workers (Flin, 2007),
Error
Non-punitive testing the effects of safety culture on healthcare staff and
Response to 0.77 0.79 0.69 0.64 0.65 patient outcomes separately should be undertaken. The current
error study is a first step of achieving this goal, and future research
Staffing 0.60 0.63 0.49 0.65 0.58
will investigate the factor structure of safety culture and
Hospital level
Hospital
outcome dimensions independently and also the effects of
0.79 0.83 0.68 0.79 0.69 safety culture dimensions on worker and patient safety
Management
Teamwork
0.70 0.80 0.59 0.65 0.70
outcomes separately.
ACROSS units
Handovers 0.74 0.80 0.69 0.65 0.77
Outcome
ACKNOWLEDGEMENT
measures
Overall The study was funded by a Scottish Funding Council Grant to
0.84 0.84 0.79 0.82 0.83
Perceptions the Scottish Patient Safety Research Network.
Frequency of
0.71 0.74 0.62 0.76 0.67
Incidents
Number of REFRENCES
incidents - - - - - Aston Business School (2007). NHS Staff Survey. Retrieved
reported from www.nhsstaffsurveys.com [accessed 07 May 2008].
Patient safety Battles, J. B. & Hilford, R. J. (2003). Organizing patient safety
- - - - -
grade of unit
research to identify risks and hazards. Quality and Safety
in Healthcare, 12, ii2-ii7.
Finally, inter-correlations were calculated between the 10 unit
Battles, J. B., Sorra, J., Famolaro, T., Dyer, N., Nelson, D. &
and hospital level dimensions and the outcome measures to
Khanna, K. (2009). Results from the AHRQ Hospital
test the discriminant validity. Overall all 10 safety culture
Survey on Patient Safety Culture. 2009 17th International
dimensions significantly correlated with 3 of the 4 outcome
Conference of Ergonomics Association, Beijing, China.
variables. In contrast, the 10 safety culture scales were found
Bentler, P. M. (1990). Comparative fit indexes in structural
to be less strongly related to the outcome “Number of
models. Psychological Bulletin, 107, 238-46.
Incidents Reported” measured by a single item. Results
Byrne, B. M. (1994). Structural equation modeling with EQS
revealed the highest correlation was between Overall
and EQS/Windows. London: Sage 98-120.
Perceptions of Safety and Staffing (r = 0.54, p<.001),
Christian, M. S., Bradley, J. C., Wallace, J. C. & Burke, M. J.
(2009). Workplace safety: A meta-analysis of the roles of
DISCUSSION
person and situation factors. Journal of Applied
Psychology, 94, 1103-1127.
The current findings supported the original 12 factor
Cox, S., & Flin, R. (1998). Safety culture: Philosopher's stone
HSOPSC model on a Scottish sample. The alternative 10
or man of straw? Work & Stress, 12, 189-201.
factor solution was tested against the original 12 factor
European Society for Quality in Healthcare (2010). Use of
structure to find the best fit. Since both factor models
patient safety culture instruments and recommendations.
performed equally well, it was decided that no modifications

Downloaded from pro.sagepub.com at University of Birmingham on June 13, 2015

PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 54th ANNUAL MEETING - 2010 854

EUNetPas Project Report, Aarhus, Denmark: Ref Type: safety climate in healthcare organizations survey. Health
Report, http://90plan.ovh.net/~extranetn/ Services Research, 42, 1999-2021.
Field, A. (2009). Discovering statistics using SPSS (Rev. ed.).
London: Sage Publications Ltd. Smits, M., Dingelhoff, I. C., Wagner, C., Wal, G. &
Flin, R., Burns, C., Mearns, K., Yule, S., & Robertson, E. M. Groenewegen (2008). The psychometric properties of the
(2006). Measuring safety climate in health care. Quality 'Hospital Survey on Patient Safety Culture' in Dutch
and Safety in Health Care, 15, 109-115. hospitals. BMC Health Services Research, 8.
Flin, R. (2007). Measuring safety culture in healthcare: A case Sorra, J. & Nieva, V. (2004). Reliability and validity of the
for accurate diagnosis. Safety Science,45:653-667. Hospital Survey on Patient Safety. Westat: Rockville.
Hellings, J., Schrooten, W., Klazinga, N. & Vleugels, A. Tabachnick, B. G., & Fidell, L. S. (2007). Using Multivariate
(2007). Challenging patient safety culture: survey results. Statistics , 5th ed. Boston: Allyn and Bacon.
International Journal of Health Care Quality Assurance, Waterson, P. E., Griffiths, P., Stride, C., Murphy, J. & Hignett,
20, 620-632. S. (2010). Pychometric properties of the hospital survey
Hofmann, D. A, & Mark, A. (2006). An investigation of the on patient safety : Findings from the UK. Quality and
relationship between safety climate and medication Safety in Health Care. Published Online First: 8 March
errors as well as other nurse and patient outcomes. 2010. doi:10.1136/qshc.2008.031625.
Personnel Psychology, 59, 847-869. Vogus, T. J. & Sutcliffe, K. M. (2007). The safety organizing
Kline, R. B. (2005). Principles and practice of structural scale. Medical Care, 45, 1-10.
equation modelling (2nd ed.). New York: Guildford. Zohar, D., Livne, Y., Tenne-Gazit, O., Admi, H. & Donchin,
Mearns, K., Flin, R., Fleming, R., & Gordon, R. (1997). Y. (2007). Healthcare climate: A framework for
Human and organizational factors in offshore safety measuring and improving patient safety. Critical Care
(HSE Report OTH 543). Norwich: HMSO. Medicine, 35, 1312-1317.
Naveh, E., Katz-Navon, T., & Stern, Z. (2005). Treatment
errors in healthcare: A safety climate approach.
Management Science, 51, 948-960.
Neal, A., Griffin, M. A. & Hart, P. M. (2000). The impact of
organizational climate on safety climate and individual
behaviour. Safety Science, 34, 99-109.
Neal, A., & Griffin, M. A. (2006). A study of the lagged
relationships among safety climate, safety motivation,
safety behavior, and accidents at the individual and
group levels. Journal of Applied Psychology, 91, 946-
953.
Nieva, V. & Sorra, J. (2003). Safety culture assessment: a tool
for improving patient safety in health care organizations.
Quality and Safety in Health Care, 12 (Suppl II):ii,17–
23.
Olsen, E. (2008). Reliability and Validity of the Hospital
Survey on Patient Safety Culture at a Norwegian
Hospital. In J. Øvretveit and P. J. Sousa (Eds.), Quality
and Safety Improvement Research: Methods and
Research Practice from the International Quality
Improvement Research Network (QIRN) (pp. 173-186).
Lisbon: National School of Public Health.
Pfeffier, Y., Manser, T., Van Vegten, A. & Wehner, T. (2008).
Dimensionality and validation of the’ Hospital Survey on
Patient Safety Culture’ questionnaire for a Swiss sample.
Poster presented at the 2008 International Conference of
Healthcare Systems and Ergonomics and Patient Safety,
Strasbourg, France.
Sexton, J. B., Helmreich, R. L, Neilands, T. B., Rowan, K.,
Vella, K., Boyden, J., Roberts, P. R., & Thomas, E. J.
(2006). The safety attitudes questionnaire: Psychometric
properties, benchmarking data, and emerging research.
BMC Health Services Research, 6.
Singer, S., Meterko, M., Baker, L., Gaba, D., Falwell, A., &
Rosen, A. (2007). Workforce perceptions of hospital
safety culture: Development and validation of the patient

Downloaded from pro.sagepub.com at University of Birmingham on June 13, 2015