Clinical Audit - UH Bristol Guide

1_what_is_clinical_audit_v4 2
2_introduction_to_the_clinical_audit_cycle_v4 8
3_how_to_choose___prioritise_topics_v4 14
4_how_to_aim_objectives___standards_v4 20
5_how_to_sample_data_collection___form_v4 26
6_how_to_data_analysis___presenting_data_v4 37
7_how_to_write_a_report___presentation_v4 45
8_how_to_implement_change_v4 52
9_how_to_involve_patients_v4 59
10_how_to_ethics_v4_1 66
What is Clinical Audit?
INTRODUCTION
The aim of this guide is to provide a brief summary of what clinical audit is and what it isn’t. Aspects of this
guide are covered in more detail in the following ‘How To’ guides:
 How To: Choose and Prioritise Topics.
 How To: Set an Audit Sample & Plan Your Data Collection.
 How To: Engage Patients, Service Users & Carers in Clinical Audit.
 How To: Apply Ethics to Clinical Audit.
WHAT IS CLINICAL AUDIT

“Clinical audit is a quality improvement process that seeks to improve patient care and outcomes through
systematic review of care against explicit criteria…Where indicated, changes are implemented…and further
monitoring is used to confirm improvement in healthcare delivery.”
Principles for Best Practice in Clinical Audit (2002, NICE/CHI)
The key messages being that:

1. Clinical audit is not just a data collection exercise:
 It involves measuring current patient care and outcomes against explicit audit criteria (also termed
standards).
 There is an expectation from the outset that practice will be improved.
2. Further clinical audit may be required to confirm that practice has improved.
The main stages of the clinical audit process are:
1) Selecting a topic.
Choose
2) Agreeing standards of best practice (audit criteria).
topic
3) Collecting data. Agree /
4) Analysing data against standards. review
5) Feeding back results. standards
6) Discussing possible changes.
Implement Collect data
7) Implementing agreed changes. change if on current
8) Allowing time for changes to embed before re-auditing. needed practice
9) Collecting a second set of data.
10) Analysing the re-audit data. Compare
11) Feeding back the re-audit results. data with
12) Discussing whether practice has improved. standards
This process is called the Audit Cycle and is summarised in the diagram above.
THE HISTORY OF CLINICAL AUDIT

‘Medical audit’ undertaken by doctors was first formalised in 1989. Prior to this, audit activity was isolated
and infrequent. Four years later, in 1993, Medical, Nursing and Therapy audit were brought together to form
the multi-disciplinary activity that we now recognise as Clinical Audit.
Since 2008 there has been a shift in the national clinical audit strategy, which has seen the ‘reinvigoration’ of
clinical audit at a local level. In line with this the Health Quality Improvement Partnership (HQIP) and the
National Advisory Group on Clinical Audit and Enquiries (NAGCAE) have been tasked, by the Department of
Health, to oversee national audits and to lead the ‘reinvigoration’ of local clinical audit by promoting quality
in healthcare, and in particular increasing the impact that clinical audit has on healthcare quality in England
and Wales.
At a local level clinical audit links into both clinical effectiveness and clinical governance.
Firstly, clinical effectiveness aims to identify and appraise existing evidence of best practice. Once identified,
if necessary, local practice may be amended to ensure that it is conforming to best practice. Once
implemented a clinical audit project might be undertaken to ensure that:
a. Best practice is being followed.
b. That patient outcomes are the desired ones.
Secondly, concerns regarding clinical care are often identified through other clinical governance structures.
These concerns can often be used to inform a clinical audit project. This includes:
1. User views or complaints.
2. Adverse incident/near miss reporting, aka clinical/critical incident reporting.
3. Identified local priorities or concerns e.g. areas of high volume, risk or cost.
WHAT CLINICAL AUDIT IS NOT

Not all ‘audit’ that takes place within the health service is clinical audit. Clinical audit is a specific activity that
measures clinical care against explicit audit criteria (standards) as part of a quality improvement cycle. The
term ‘audit’ has a range of meanings and whilst people might want to ‘audit’ something it does not
necessarily mean that they are doing or want to do a clinical audit project.
Other forms of audit can include:

 Financial audit - Looking at accounts to establish whether they provide a true and fair view of the
organisation's financial position at a given time.
 Internal audit - An internal mechanism that traces non-clinical activities and systems along 'audit paths'
to see if things happened the way they should have. For example, tracing a patient complaint from the
initial letter of complaint through to resolution to establish whether Trust guidelines were followed
appropriately.
 Organisational audit - An external, independent and voluntary audit of the whole organisation, based on
a framework of explicit standards. Organisational audit looks at how well the organisation is set up and
runs on a daily basis. The King’s Fund is an example of an independent service that undertakes
organisational audits.
 Counting things/ Investigations - The collection of data which is not related to explicit audit criteria
(standards) is not considered to be clinical audit.
 Routine monitoring of clinical outcomes - The identification and measurement of clinical outcomes that
are explicitly linked to the change process may form part of a clinical audit project. However routine
monitoring of outcome data for purposes such as performance monitoring is not considered to be
clinical audit.
 Peer review including Mortality & Morbidity (M&M) - Peer review is a process whereby a group of
clinicians collectively assess a small sample of patients recently under their care to establish whether the
best possible care was provided or whether things might have been done differently. M&M reporting is
a specific peer review process that looks at specific, non-random, cases with adverse outcomes, such as
death or injury, to see what lessons can be drawn.
 Staff, patient, service user, carer surveys - Surveys are usually carried out as part of a research project or
as an engagement activity. They are primarily used to gain the opinions of staff, patients, service users
or carers regarding treatment and/or the quality of care in order to see if improvements can be made.
Surveys should only be used for clinical audit if the data sought cannot be collected from another source
and it is related to processes or outcome of care i.e. were standards of best practice being met.
2005 - 2017 UHBristol Clinical Audit & Effectiveness Team – Version 4 Page 2 of 6
CLINICAL AUDIT & RESEARCH: WHAT IS THE DIFFERENCE?

“Research is concerned with discovering the right thing to do; audit with ensuring that it is done right”
Smith R. Audit & Research. BMJ 1992; 305: 905-6
Research addresses clearly defined questions and hypotheses using systematic processes to generate new
evidence to refute, support or develop a hypothesis, by asking the question ‘what is best practice?’ As a
result of which a new service or new practice may be developed. The methodology is designed so that it can
be replicated and so that the results can be generalised to other similar groups.
Research may involve a completely new treatment or practice, the use of control groups or placebo
treatment for purposes of comparison, or allocating service users randomly to different treatment groups.
Patients should be involved in the design, implementation and analysis of the work.
Research must comply with Research Governance, and be registered with the Research & Innovation
Department. It also has to be submitted to the Research Ethics Committee (REC) for approval. The contact
details for the Research & Innovation Department are listed at the end of this guide.
Alternatively, clinical audit aims to improve the quality of local patient care and clinical outcomes through
the peer-led review of practice against evidence-based standards, implementing change where necessary. It
asks the questions ‘are we following best practice?’ and ‘what is happening to patients as a result?’
Clinical audit is initiated by national bodies, commissioners/Clinical Commissioning Groups (CCGs) or service
providers, including local healthcare staff and managers. The methodology is designed to address clearly
defined audit questions that establish whether a specific clinical standard is being met. Results are specific
and local to a particular team or service although the audit tool may be used by more than one team or
service.
A clinical audit project will never involve a completely new treatment or practice, never involve the use of
control groups or placebo treatments, nor does it involve allocating patients randomly to different
treatment groups. It may, however, involve input from patients, service users or carers at a number of
levels, e.g.
 Participation in surveys which help to determine whether standards have been met.
 Involvement in the design of individual clinical audit projects or whole programmes of activity.
Clinical audit projects must be registered with the Clinical Audit Team, and therefore will have been
approved by the relevant Clinical Audit Convenor. The use of survey methodologies as part of a clinical audit
is also subject to approval by the Trust’s Questionnaire Interview & Survey Group (QIS). Whilst clinical audit
projects should be scrutinised for ethical implications, REC approval is not required.
CLINICAL AUDIT & RESEARCH: WHY IT IS IMPORTANT TO KNOW THE DIFFERENCE

As outlined above research projects and clinical audit projects have very different purposes, and therefore
use different methodologies; they are also managed and funded in different ways.
It is sometimes suggested that research is more rigorous than audit, but both research and audit can either
be rigorous (i.e. done according to protocol and producing valid results) or not rigorous enough (i.e. done
carelessly, producing flawed results, and in the case of clinical audit, not leading to improvements in clinical
practice).
Whilst research requires REC approval, clinical audit does not. However, clinical audit should still be
conducted within an ethical framework. By approving and registering a project as a clinical audit, the Trust is
stating that the project fulfils the methodological criteria that allows for patient data to be accessed and
analysed.
Whilst clinical audit projects may be published without ethical approval, e.g. the Quality Improvement
Reports published by the British Medical Journal, journal editors may refuse to publish articles if there are
ethical concerns and REC ethical approval has not been granted. If you want to publish because of the
results of your project, rather than to share the methodology, you should question whether you are
undertaking research, rather than a clinical audit project.
SERVICE EVALUATION
The aim of service evaluation is to judge a service’s effectiveness or efficiency through the systematic
assessment of its aims, objectives, activities, outputs, outcomes and costs. It addresses specific questions
about the service concerned and results are specific and local to a particular team or service and may lead to
service redesign and reconfiguration in that particular area. The evaluation tool may, however, be used by
more than one team or service.
Service evaluation never involves completely new treatment practices, the use of control groups or a
placebo treatment nor does it involve allocating service users randomly to different treatment groups. It
may, however, involve input from patients, service users or carers through their participation in surveys,
which help to determine the effectiveness or efficiency of a service, or through their involvement in the
design of individual projects or whole programmes of activity.
If service evaluation activity is undertaken via the Clinical Audit Team or the Research & Development
Department, it will be subject to the scrutiny and advice of those teams, however it should be noted that
neither team currently provides support in the field of service evaluation. Importantly, whilst service
evaluation projects should be scrutinised for ethical implications, REC approval is not required.
PATIENT, SERVICE USER, CARER ENGAGEMENT

Research, clinical audit or service evaluation projects may all include a patient, service user or carer survey.
In terms of clinical audit, surveys can be a useful tool, where measuring compliance against your audit
criteria requires information that can only be obtained from the patient or service user e.g. ‘Did the doctor
introduce themselves at the beginning of your appointment?’ ‘Did the doctor listen to what you had to say?’
Surveys can be construed as doing something ‘beyond normal clinical management’; therefore it is
important to get advice on the design of your survey as some questions might touch upon potentially
sensitive matters, which would give rise to ethical concerns. It is extremely important that all surveys are
designed to cause minimum possible disruption.
If you are including a patient survey as part of your project it must be submitted to the Questionnaire,
Interview and Survey (QIS) Group for approval. The contact details for the QIS group are listed at the end of
this guide.
UNDERTAKING A CLINICAL AUDIT PROJECT AT UHBRISTOL

Clinical audit at UHBristol is supported by Divisional Clinical Audit Facilitators, centrally located within Trust
Headquarters. Facilitators in turn work closely with Clinical Audit Convenors within Divisions/Specialties.
Further details of roles can be found in the following section:
 Clinical Audit Facilitator - This is the first person you should contact when you have an idea for a project.
Your Facilitator will help you focus your project design and complete the project proposal paperwork
(available on the clinical audit website - website and contact details are listed at the end of this guide).
They will provide support and advice throughout the clinical audit process, e.g. sample selection, data
analysis, presentation writing. However, Facilitators do not assist in data collection.
 Clinical Audit Convenors - Your Divisional/specialty Convenor is a senior clinician responsible for
promoting clinical audit in your area. The Convenor actively supports the Clinical Audit Facilitator in day-
to-day activities and liaises with the Facilitator to agree the annual audit programme and to approve all
clinical audit projects.
 Clinical Audit Group - The Clinical Audit Group is the Trust’s lead group in relation to all matters relating
to the practice of clinical audit. The core function of CAG is to ensure that the Trust has a clear strategy
for Clinical Audit, endorsed by the Quality and Outcomes Committee of the Trust Board, and to monitor
its implementation through regular reporting on activity.
CLINICAL AUDIT STRATEGY

The UHBristol Clinical Audit Strategy states that all healthcare professionals are expected to participate in
clinical audit; it defines what is and is not considered to be clinical audit (as outlined above); it states that
audits relating to the National Agenda should be prioritised; and it places an emphasis on multi-professional
clinical audit and direct/indirect patient engagement.
Clinical audit projects are either unidisciplinary (i.e. involving only one staff group) or multidisciplinary (i.e.
involving more than one discipline or profession). At UHBristol we believe it is important that a clinical audit
project assesses patient care as provided by the whole clinical team, in order to identify how care can be
improved. Therefore if your project has implications for a profession or discipline other than your own,
whether within or outside the clinical area you work in, it is important to ensure that they are represented
on the project team.
If your clinical audit project is looking at the patient journey across different care sectors i.e. ‘interface’
audit, try to include staff representatives from these other care sectors in your project team. An example of
an interface audit would be looking at the process of referral into the hospital from primary care.
You should also consider including a patient, service user and/ or carer representative(s) on your project
team or gaining their views on what they would like to see from the service. Engagement is important as
healthcare is a partnership between clinicians and their patients/ service users. Whilst we strive to provide
the best quality of care as we see it, patients/ service users might want something different. Direct
engagement is best, e.g. through participation of identified individuals on project steering groups or
divisional/specialty audit committees. However, indirect engagement is another possibility, e.g. through the
completion of a survey, usually at the end of an episode of care.
SUMMARY
 Clinical Audit is a quality Improvement process that measures current patient care and outcomes against
agreed standards of best practice.
 Not all ‘audit’ is clinical audit.
 Be aware of the differences between project categories:
 Clinical audit - audit against agreed standards of best practice.
 Research - aims to create new knowledge.
 Service Evaluation - assesses the effectiveness of a service.
 If you are unsure whether the project you wish to undertake is a clinical audit project, your Divisional
Clinical Audit Facilitator will be able to advise.
CONTACT DETAILS / USEFUL INFORMATION
CLINICAL AUDIT
 The UHBristol Clinical Audit website is available via http://www.uhbristol.nhs.uk/for-
clinicians/clinicalaudit/
 Contact details for UHBristol Clinical Audit Facilitators are available via
http://www.uhbristol.nhs.uk/for-clinicians/clinicalaudit/contacts/
 The full range of UHBristol Clinic Audit ‘How To’ guides are available via
http://www.uhbristol.nhs.uk/for-clinicians/clinicalaudit/how-to-guides/
 Copies of UHBristol Clinical Audit Proposal Form, Presentation Template, Report Template, Summary
Form, and Action Form are available via http://www.uhbristol.nhs.uk/for-
clinicians/clinicalaudit/carrying-out-projects-at-uh-bristol/
 The UHBristol Clinical Audit & Effectiveness Central Office can be contacted on 0117 342 3614 or e-
mail: stuart.metcalfe@uhbristol.nhs.uk
 Clinical Audit Training Workshops can be booked through the Clinical Audit & Effectiveness Central
Office as above.
CLINICAL EFFECTIVENESS
 For advice on Clinical Effectiveness (NICE, NCEPOD, PROMS, guidelines) matters contact Stuart Metcalfe,
Clinical Audit & Effectiveness Manager, 0117 342 3614 or e-mail: stuart.metcalfe@uhbristol.nhs.uk
PATIENT EXPERIENCE
 For advice on carrying out surveys, interviews and questionnaires please contact Paul Lewis, Patient
Experience Lead (Surveys & Evaluations), 0117 342 3638 or e-mail: paul.lewis@UHBristol.nhs.uk
 For advice on conducting qualitative and Patient Public Involvement Activities (focus groups,
community engagement, co-design, workshops) please contact Tony Watkin, Patient Experience Lead
(Engagement & Involvement), 0117 342 3729 or e-mail: tony.watkin@UHBristol.nhs.uk
 All surveys that are being carried out for service evaluation or audit purposes should be discussed with
Paul Lewis in the first instance. Patient experience surveys will also usually need to be approved by the
Trust's Questionnaire, Interview and Survey (QIS) Group. Proposals should be submitted to Paul Lewis
using the QIS proposal form. The proposal form and covering letter template is available via
http://www.uhbristol.nhs.uk/for-clinicians/patient-surveys,-interviews-and-focus-groups/
RESEARCH
 For advice on research projects contact the Research & Innovation Department on 0117 342 0233 or e-
mail: research@UHBristol.nhs.uk
 Further information can be found via http://www.uhbristol.nhs.uk/research-innovation/contact-us/
LITERATURE REVIEWS/EVIDENCE
 For advice on literature reviews, NHS Evidence, article/book requests and critical appraisal contact the
Library and Information Service on 0117 342 0105 or e-mail: Library@UHBristol.nhs.uk
SAMPLE SIZES
 The Sample Size Calculator is available via: http://www.uhbristol.nhs.uk/for-
clinicians/clinicalaudit/how-to-guides/
QUALITY IMPROVEMENT
Further information about clinical audit and wider quality improvement is available via the Healthcare
Quality Improvement Partnership (HQIP) - http://www.hqip.org.uk/
A Brief Introduction to the Clinical Audit Cycle
INTRODUCTION
This guide provides a brief introduction to the principles of undertaking a clinical audit project at UHBristol.
Each aspect of the clinical audit cycle is covered in more detail as part of the complete range of ‘How To’
guides as available on the clinical audit website:
 What is Clinical Audit?
 How To: Choose & Prioritise Topics.
 How To: Set an Audit Aim, Objectives & Standards.
 How To: Set an Audit Sample & Plan Your Data Collection.
 How To: Analyse & Present Data.
 How To: Share Your Findings – Clinical Audit Report & Presentation.
 How To: Implement Change Successfully.
 How To: Apply Ethics to Clinical Audit.
WHAT IS CLINICAL AUDIT

“Clinical audit is a quality improvement process that seeks to improve patient care and outcomes through
systematic review of care against explicit criteria…Where indicated, changes are implemented…and further
monitoring is used to confirm improvement in healthcare delivery”
Principles for Best Practice in Clinical Audit (2002, NICE/CHI)
The main stages of the clinical audit process are:
1) Selecting a topic.
Choose
2) Agreeing standards of best practice (audit criteria).
topic
3) Collecting data. Agree /
4) Analysing data against standards. review
5) Feeding back results. standards
6) Discussing possible changes.
Implement Collect data
7) Implementing agreed changes. change if on current
needed practice
8) Allowing time for changes to embed, before re-auditing.
9) Collecting a second set of data. Compare
10) Analysing the re-audit data. data with
11) Feeding back the re-audit results. standards
12) Discussing whether practice has improved.
This process is called the Audit Cycle and is summarised in the diagram above.
UNDERTAKING A CLINICAL AUDIT PROJECT AT UHBRISTOL

Clinical audit at UHBristol is supported by Divisional Clinical Audit Facilitators, centrally located within Trust
Headquarters. Before undertaking a clinical audit project, you should discuss your proposed project with
the relevant Facilitator and complete the clinical audit project proposal paperwork (available on the clinical
audit website - website and contact details are listed at the end of this guide).
 2005 - 2017 UHBristol Clinical Audit Team – Version 4 Page 1 of 6

A Brief Introduction to Clinical Audit
1. CHOOSE A CLINICAL AUDIT TOPIC

Your topic should be chosen systematically. Clinical audit projects take time and resources so the topic that
you choose to address should be of potential benefit to the service as a whole.
Clinical audit projects are best focussed on the processes - e.g. investigations, treatments, or procedures -
which have been shown to result in the best patient outcomes if followed.
Possible sources for your clinical audit project include the:

 National Institute of Clinical Excellence (NICE), National Service Frameworks (NSFs), National
Confidential Enquiries, Patient Safety Initiatives, or Royal Colleges / national professional bodies.
 Publication of conclusive new evidence about clinically effective healthcare; local or regional treatment
guidelines, protocols or frameworks; user views or complaints; adverse incident/near miss reporting
(aka clinical/critical incident reporting); or identified local priorities or concerns, e.g. areas of high
volume, risk or cost.
If you would like to carry out a clinical audit project but are unsure about appropriate topics your divisional
Clinical Audit Facilitator will be able to help you to identify key subjects in your clinical area.
2. FORM A PROJECT TEAM

Clinical audit projects are generally described as being either unidisciplinary (i.e. involving only one staff
group) or multidisciplinary (i.e. involving more than one discipline or profession). It is important that a
clinical audit project assesses patient care as provided by the whole clinical team, in order to identify how
care can be improved. Therefore if your project has implications for a profession or discipline other than
your own, whether within or outside the clinical area you work in, it is important to ensure that they are
represented on the project team.
If your clinical audit project is looking at the patient journey across different care sectors, i.e. ‘interface’
audit, try to include staff representatives from these other care sectors in your project team. An example of
an interface audit would be looking at the process of referral into the hospital from primary care.
Consider including a patient, service user and/ or carer representative(s) on your project team or gaining
their views on what they would like to see from the service. Engagement is important as healthcare is a
partnership between clinicians and their patients/ service users. Whilst we strive to provide the best quality
of care as we see it, patients/ service users might want something different. Direct engagement is best e.g.
through participation of identified individuals on project steering groups or divisional/specialty audit
committees. However, indirect engagement is another possibility, e.g. through the completion of a survey,
usually administered at the end of an episode of care.
It is important that your project is supported by colleagues who have the authority and commitment to see
that any necessary changes indicated by the audit results are put into practice.
3. SET THE AIM, OBJECTIVES AND STANDARDS

To define the aim of your clinical audit project consider Audit Topic
what it is that you hope to achieve, i.e. the overall
purpose of the project. The aim can be written as either a Aim
statement that about what you want to happen as a result
of the audit or as a question that you want your audit to Objective Objective
answer. Statements should be phrased positively, to
ensure that the audit brings about improvements in Standard Standard Standard
practice.
2005 – 2017 UHBristol Clinical Audit & Effectiveness Team – Version 4 Page 2 of 6
Your aim is then broken down into a series of smaller objectives. Objectives are the steps that you need
to take in order to assess whether or not you have achieved your aim. Your objectives can be written as
either specific tasks to be undertaken or as the different aspects of quality that your project will address,
e.g. appropriateness = right treatment for right patient; timeliness = treatment given at right time; or
effectiveness = treatment given in right way or with desired effect
S pecific Standards should be SMART and are more specific than objectives. They are
quantifiable statements detailing the specific aspects of patient care and
easurable
management that you intend to measure current practice against. Standards
greed should always be based on the strongest, most up-to-date evidence of what
constitutes best practice. If standards are available in the form of guidelines, you
elevant
should base your audit on the most widely applicable guidelines available, e.g.
heoretically sound national rather than regional or local guidelines.
If guidelines/ protocols do not exist or existing ones are out of date, you will need to undertake a literature
search to identify best practice. Assistance with this can be provided by the Library and Information Service,
whose contact details are listed at the end of this guide. It is important that there is agreement with your
standards locally before you start. You will find it hard to improve practice if there is disagreement as to
what constitutes best practice.
4. ETHICS & ENGAGEMENT

Unlike research, clinical audit projects do not need to be submitted to a Research Ethics Committee (REC)
for ethical approval. This is one of the key reasons why you must ensure that your project is clinical audit
rather than research. If you think that there are ethical issues with your project you must discuss these with
your divisional Clinical Audit Facilitator.
Clinical audit should always be conducted within an ethical framework. This means abiding by the principles
of the Data Protection Act; e.g. by ensuring patient and staff confidentiality and by ensuring that data is
collected and stored appropriately.
If you are including a patient survey as part of your project it must be submitted to the Questionnaire,
Interview and Survey (QIS) Group for approval. Contact details for the QIS group are listed at the end of this
guide.
5. SELECT AN AUDIT SAMPLE

The sample population will be dependent upon your topic. Occasionally an aspect of treatment or care that
applies to all patients is audited. However, the majority of clinical audit tends to assess the care of a defined
group of patients who share certain characteristics. Typically, the fact that they have the same medical
condition, have received the same form of treatment or were seen within a certain time frame.
It is not always practical to audit all patients within your population, therefore you will need to select an
appropriate sample size. The sample should be large enough so that senior clinicians/managers are willing to
implement changes based on your findings. A ‘snapshot’ is often sufficient (roughly 20-50 cases) for process-
based audit.
It is important that your sample contains current/recent patients. Clinical audit is about improvement. You
cannot change the past but you can change the future.
6. PLAN AND CARRY OUT DATA COLLECTION

Data should only be collected if it is necessary to enable you to measure current practice against your audit
criteria (standards). Any extra data means more time spent on your project without any additional benefit
and is contrary to the principles of the Data Protection Act.
If data is routinely recorded either in the patient’s notes or electronically it is possible to carry out a
retrospective audit, i.e. assessing past episodes of patient care. If the data is not routinely recorded
prospective data collection is required, i.e. assessing the patient care at the time it is given. Data can be
collected using an audit form (proforma) or entered directly onto a computer.
Before you collect the data for your entire sample it is important to pilot your data collection ‘tool’ (form/
spreadsheet/database). The purpose of the pilot is to try out your tool on a small sample to make sure that
it works, especially if someone else is going to be collecting the data for you. The pilot may reveal that some
of your instructions on how to complete the tool or the questions asked are ambiguous, that the tool is
difficult to complete or that you are simply not getting the information you wanted.
7. ANALYSE THE DATA Number of patients

Your data analysis should establish which standards are who meet standard
being met and which are not. If a standard is not being met 100
x
you need to identify why and how practice can be improved
to ensure that the standard is met in the future. You may Number of Number of
also consider if there were other, acceptable reasons for the patients to patients who
whom standard
– meet any listed
standard not being met, i.e. an exception not considered
applies exceptions
during the planning stage.
8. PRESENT THE FINDINGS

The findings of your project should be discussed by the project team and presented to colleagues. If full
compliance with the standards was not achieved an action plan should be developed to address any issues.
Once completed projects should be presented both verbally and written up as a report. Both should address
how well the standards are being met and highlight any problems that need to be addressed. The
presentation gets the message of your audit across to key staff and should generate discussion and
agreement about changes to practice in light of the audit results.
Your final report (template available on the clinical audit website, address provided at the end of this guide)
acts as the official record of what you have done and contains more detail than the presentation. It should
include an Action Plan if improvements are required and all the information needed to plan a re-audit.
9. IMPLEMENT CHANGES AND RE-AUDIT

If an audit shows that current practice needs to be improved, making changes is important. The public has
the right to expect that practitioners will provide care that is consistent with recognised good practice.
Not all changes will be improvements. Do not make changes for change’s sake. At an appropriate time,
repeat the audit (re-audit) to ensure that changes have been implemented and that practice has improved.
A FLOW CHART SUMMARISING THE CLINICAL AUDIT PROCESS AT UHBRISTOL
Discuss and agree project with divisional Clinical Audit Facilitator
Audit topic
Form a project team
Set the aim, objectives and standards
Consider ethics and engagement
Select an audit sample
Plan your data collection and data collection form
Agree project with all areas and staff affected and gain senior clinician support
Complete the UHBristol clinical audit proposal paperwork
Submit proposal to relevant Clinical Audit Convenor/Facilitator for approval
Carry out data collection
Analyse the data
Share your findings: 1. Presentation 2. Report and action plan
Implement changes and re-audit
CONTACT DETAILS / USEFUL INFORMATION
CLINICAL AUDIT
Office as above.
 For advice on Clinical Effectiveness (NICE, NCEPOD, PROMS, guidelines) matters contact Stuart
Metcalfe, Clinical Audit & Effectiveness Manager, 0117 342 3614 or e-mail:
stuart.metcalfe@uhbristol.nhs.uk
PATIENT EXPERIENCE
RESEARCH
For advice on research projects contact the Research & Innovation Department on 0117 342 0233 or e-mail:
research@UHBristol.nhs.uk
Further information can be found via http://www.uhbristol.nhs.uk/research-innovation/contact-us/
For advice on literature reviews, NHS Evidence, article/book requests and critical appraisal contact the
SAMPLE SIZES
QUALITY IMPROVEMENT
How To: Choose & Prioritise Topics
INTRODUCTION
This ‘How To’ guide provides information on how to choose and prioritise clinical audit topics. If you would
like to carry out a clinical audit project but are unsure about appropriate topics, your Divisional Clinical Audit
Facilitator will be able to help you to identify key subjects in your clinical area.
BACKGROUND
Historically there was a move over time towards greater central, i.e. national, control over the content of
local audit programmes as evidenced by an increasing concentration on national and regional agendas.
However, there has been a recent shift in clinical audit strategy which has seen the “reinvigoration” of
clinical audit at a local level.
According to research, most benefit is derived from small-scale clinical audit projects designed by local
teams and focused on local care. The objective is to identify areas where improvements can be made in a
way that does not apportion blame and then to make changes as a multi-disciplinary team. At present the
majority of clinical audit projects carried out at UHBristol are locally determined.
Clinical audit topics should be chosen systematically. Projects take time and resources so the topic that you
choose should be of potential benefit to the service as a whole.
THE NATIONAL AGENDA
UHBristol clinical audit strategy states that the national agenda takes priority. The national agenda
comprises topics from a number of agencies as listed below. The guidelines produced and recommendations
made by these agencies can often provide the basis for a clinical audit project.
1. The Care Quality Commission (CQC) is a non-departmental public body of the United Kingdom
government and is responsible for monitoring performance in the NHS.
2. The National Clinical Audit and Patient’s Outcomes Programme (NCAPOP) is managed by the Healthcare
Quality Improvement Partnership (HQIP). It is responsible for commissioning national audit projects;
engaging stakeholders as part of the reinvigoration of audit, building consensus about how to develop
audit in the future; and supporting clinical audit through the development of materials and resources to
support audit at both a local and national level.
3. National Confidential Enquiries review the management of patients via nationwide confidential surveys
and research. The findings and recommendations produced are widely disseminated in order to
maintain and improve standards of medical and surgical care for the benefit of the public.
4. The National Institute of Clinical Excellence (NICE) is an independent organisation responsible for
providing national guidance on promoting good health and preventing and treating ill health. NICE
produces guidance in three areas of health - public health, health technologies and clinical practice.
5. The National Patient Safety Agency (NPSA) is an arm’s length body of the Department of Health that
contributes to improved, safe patient care by informing, supporting and influencing organisations and
people working in the health sector.
6. National Service Frameworks (NSFs) are targeted to provide clear quality requirements for disease-
specific care. Each NSF or strategy is based on the best available evidence of what treatments and
services work most effectively for patients.
7. National Audits aim to identify and share good practice guidance.
©2005 - 2017 UHBristol Clinical Audit & Effectiveness Team – Version 4 Page 1 of 6
8. The Royal Colleges/ National Professional Body produce written guidance for specific diseases and
conditions. These guidelines provide recommendations on the appropriate treatment and care of
patients as well as recommendations on service provision.
9. Patient Safety Initiatives: The aim of the national “Sign Up To Safety” campaign is to halve avoidable
harm in the NHS and save 6000 lives as a result, over a three-year time period from 2015 to 2018.
Within a number of work-streams, the aim of the initiative is to reduce adverse events and avoidable
risks and to improve the overall safety culture of the organisation.
OTHER SOURCES OF CLINICAL AUDIT PROJECTS
RESEARCH/ GUIDELINES
1. Publication of conclusive new evidence about clinically effective healthcare e.g. information from Royal
College publications, recognised journals, etc.
2. Local or regional treatment guidelines, protocols or frameworks. These will ideally be based on best
evidence (perhaps from national guidelines), or maybe a local consensus of best practice if little or no
evidence is available.
ASPECTS OF CLINICAL GOVERNANCE

Concerns regarding clinical care are often identified through the various facets of clinical governance. These
concerns can often be used to inform a clinical audit project.
1. User views or complaints - There is increasing pressure on Trusts to involve patients in audit. One way of
achieving this is to select topics that are of concern to patients, raised by way of a complaint or via the
Patient Advice and Liaison Service (PALS) or in a focus group.
2. Adverse incident/near miss reporting, aka clinical/critical incident reporting. Incident reporting can
highlight potential topics to audit.
3. Identified local priorities or concerns e.g. areas of high volume, risk or cost - If you are undertaking a
clinical audit project that is based on local priorities or concerns it is important to remember that the
project should be important to the division or Trust, e.g. as identified in the Divisional Clinical Audit
Forward Plan, rather than simply the personal interest of an individual clinician.
STRUCTURE, PROCESS & OUTCOME
What you expect Measurable change in health status

Outcome
What you do Investigations, treatments, procedures

Process
What you need Structure Staff and resources that enable healthcare
PROCESS
Audits of process focus on the clinical care received by patients, e.g. investigations, treatments, or
procedures. Projects are best focussed on the processes, which have been shown result in the best patient
outcomes. For example, if research has shown that Drug X gives better outcomes than Drug Y for patients
with condition A, you would audit “are patients with condition A being given Drug X?”
DISADVANTAGES OF AUDITING STRUCTURE & OUTCOME

Audits of structure look at the resources that enable treatment and care to happen, such as staffing
numbers and mix, as well as environment and equipment. Structural improvements usually involve money
and are better dealt with via business plans or other forms of audit such as financial audit, internal audit or
organisational audit.
Audits of outcome look at the results of our interventions. Outcomes can be defined as measurable changes
in health status; examples include dead/alive (mortality), recurrence of disease, complication rates,
readmission rates, and quality of life measures both generic and disease specific. Measuring outcomes can
be difficult. In terms of mortality, for example, do you look at all deaths that have occurred or only those
that have occurred in the 30 days following surgery? How do you know if the patient died as a result of an
intervention or because of some other cause, such as an undiagnosed condition or compromised immune
system due to current state of ill health? Clinical outcomes are often routinely monitored at the Trust or are
addressed through peer review processes including Mortality & Morbidity (M&M) reporting.
TOPIC IDENTIFICATION
The topic identification table on the following page is a simple mechanism that can be used to generate and
prioritise ideas for clinical audit. This activity is best carried out by all members of your clinical audit team.
The table allows you firstly to identify potential topics to audit, either from the national agenda or reflecting
local needs, and secondly to prioritise these topics.
This model can be adapted to better fit your needs. You could use different criteria; e.g. cost, availability of
evidence about clinical effectiveness or issues that patients have expressed concern about.
Once your list of topics has been prioritised, the list will need to be reviewed to ensure that the projects are
suitable for clinical audit. In some cases there might be a more appropriate way to tackle a problem rather
than through clinical audit; for example, writing a business case for funding or reporting via clinical risk
management (adverse incident reporting). If you are unsure about the suitability of your topic, your
divisional Clinical Audit Facilitator will be able to advise you and if necessary will be able to help you to
identify key subjects in your clinical area.
In summary, in order to progress your topic through clinical audit you must have:
 A specific, focused clinical audit question.
 Published evidence, to provide evidence-based standards.
 An ability and willingness in your clinical team to improve practice in this area.
EXAMINING PATIENT PATHWAYS

Another approach to identifying audit topics is to focus on patient pathways for given clinical conditions.
Every patient passes through a number of points on their journey between their first and last contact with
the Trust, e.g., Admission - Assessment - Diagnosis - Treatment - Review – Discharge.
At each point on this pathway there are aspects of care that could be audited. For example:
 Have patient history and examination been properly recorded following admission?
 Were appropriate investigations carried out?
 Was the treatment appropriate and timely?
 What was the outcome?
 Is the content of discharge summary adequate?
Also consider the interface of this pathway with other organisations; for example, referrals in or discharges
from the hospital. Care often falls down here because of problems in communication.
TABLE 1: Topic Identification
Source of Audit Audit Topic
Patient request/query
Issue of local concern
Score (number of Yes

Multi-disciplinary y/n
Relates to UH Bristol
High Volume y/n
Direct impact on
involvement y/n
Relates to dept.
Direct patient
High Cost y/n
High Risk y/n
priorities y/n
priorities y/n
Interface y/n
Re-audit y/n
patients y/n
responses)
y/n
y/n
Care Quality
Commission
National Clinical
Audit and Patient’s
Outcomes
Programme
National
Confidential Enquiry
NHSLA
NICE Clinical
Guideline
NICE Technology
Appraisal
NICE Interventional
procedure guidance
National Patient
Safety Agency
National Service
Framework
National Audits
Royal Colleges/
National
Professional
Trust's Safer Body
Patient
Initiative
Published research
Local / regional
guidelines
User views or
complaints
Adverse incident/
near miss reporting
SUMMARY
 Focus your efforts where there is greatest potential for improving the quality of care. Do not waste
valuable time looking at areas where realistically you know there is little possibility of making
improvements.
 Get all your stakeholders (colleagues, managers, etc) on board from the start and make sure that they
understand clearly what you are trying to achieve.
 Clinical audit needs to be justifiable in terms of the benefits it will bring about for patients balanced
against the amount of time and resources it takes. For each proposed project topic, ask yourself:
 What is the benefit for the patient of doing this project?
 Will it take a disproportionate amount of time and/or funds to complete?
 All clinical audit projects in UHBristol should be registered and approved. It is worth meeting with your
divisional Clinical Audit Facilitator to discuss your proposal once you have established the topic you
would like to audit.
CONTACT DETAILS/ USEFUL INFORMATION
CLINICAL AUDIT
Office as above.
PATIENT EXPERIENCE
RESEARCH
SAMPLE SIZES
QUALITY IMPROVEMENT
How To: Set an Audit Aim, Objectives & Standards
INTRODUCTION
This ‘How To’ guide provides information on how to set an audit aim, objectives and standards. If you
require any help locating the relevant literature/ evidence-base for your standards, your divisional Clinical
Audit Facilitator may be able to help.
AIM
To define the aim of your clinical audit project consider what it is that you hope to achieve, i.e. the overall
purpose of the project. It should be related to the rationale behind your choice of audit topic and should not
merely be to ‘count the number of’ or ‘examine’, but should focus your audit towards achieving
improvements in practice where necessary.
The aim can be written as a statement about what you want to happen as a result of the audit. Statements
should be phrased positively, to ensure that the audit brings about improvements in practice. For example,
‘To improve the care received by patients who develop leg ulcers.’
The aim can also be phrased as a question that you want your audit to answer. For example, ‘Are we
meeting standards of best practice for the management of leg ulcers?’
OBJECTIVES
Your aim provides a broad structure for your clinical audit project. This may need to be broken down into a
series of smaller objectives. Objectives are the steps that need to be taken in order to assess whether or not
you have achieved your aim. Your objectives can be written as either specific tasks to be undertaken or as
the different aspects of quality that your project will address.
A simple and illuminating question that you could ask your team when designing your project is: “What do
we think that we ought to be doing?” The answers to this question should be based upon the best available
evidence.
More technical forms of this question are outlined below.
ASPECTS OF QUALITY
The majority of clinical audit projects focus on:
 Appropriateness – e.g. Is the right treatment being provided to the right patient? It is generally easier to
assess whether the patients that received the treatment were appropriate rather than whether all the
appropriate patients received the treatment.
 Timeliness – e.g. Was the treatment given at the right time? Other examples include the medical review
of a deteriorating patient within 15 minutes, an appointment for fast-track cancer referral within 14
days, referral to treatment within 18 weeks, or arrival to bed allocation in A&E within 4 hours.
 Effectiveness – Was the treatment given in the right way? With the desired effect?

Other aspects of quality that do not tend to be assessed through clinical audit are:
 Efficiency – Was the treatment given with minimum effort, expense, waste? Efficiency issues are best
resolved through service /quality improvement work. Service improvement aims to improve processes
and systems of care, through process mapping and redesign, in order to make them more efficient.
 Acceptability - Is the treatment acceptable to the patient? Acceptability is usually a focus of research or
patient involvement activity, rather than clinical audit.
 Accessibility - Ease of getting care.
 Equity - Is the treatment available to all patients on an impartial basis? Accessibility and equity issues
are best dealt with by management addressing problems with the structural aspects of care.
EXAMPLE 1 How to Write: Clinical audit topic, aim and objectives
Topic: Leg ulcers
Aim: To improve the care received by patients with leg ulcers
Objectives:
1. To ensure that leg ulcers are treated appropriately
2. To ensure timely treatment of leg ulcers
Patient care cannot be measured against the objectives contained in the example above, as they do not
specify what ‘appropriate’ treatment is or what ‘timely’ treatment would consist of. This is where audit
standards come in; they define exactly how these aspects of care will be measured.
AUDIT STANDARDS
Clinical audit is by definition standards-based (sometimes referred to as ‘criterion-based’). Standards are

more specific than objectives. They are quantifiable statements detailing the specific aspects of patient care
and/or management that you intend to measure current practice against. They seek to ensure that the best
possible care is provided, given available resources, and they are based upon the best available evidence.
“A standard is an explicit statement describing the quality of care to be achieved, which is definable and
measurable”
Using standards to define precisely the care that we are seeking to provide means that we can:
 Accurately inform anyone who might want to use our service, what service it is that we are offering.
 Identify the things we need to enable us to provide our service.
 Monitor and improve our performance.
Your standards should be related to your audit topic and objectives, as per the framework below.
Audit Topic
Aim
Objective Objective
Standard Standard Standard

DEVELOPING AUDIT STANDARDS

If standards are available in the form of guidelines or protocols, you should base your audit on the most
widely applicable guidelines available, e.g. national rather than regional or local guidelines. However, it is
important to bear in mind that guidelines are only as good as the evidence they are based upon. Even some
national guideline statements are only ‘good practice’, without any research evidence-base.
If guidelines/protocols do not exist, or existing ones are out of date, you will need to undertake a literature
search to identify best practice. Assistance with this can be provided by the Trust’s Library and Information
Service. Their contact details are listed at the end of this guide.
It is important that there is agreement with your standards locally before you start. You will find it hard to
improve practice if there is disagreement as to what constitutes best practice.
Standards should always be based on the strongest, most up-to-date evidence of what constitutes best
practice. A generally accepted hierarchy for strength of evidence is:
Stronger (best)
Systematic Reviews of RCTs (randomised controlled trial – eliminates/reduces
bias to give most reliable evidence)
Results of single RCTs
Results of well-conducted non-RCT clinical studies
Expert committee reports; clinical experience of respected authorities
Personal experience and opinion

Weaker
WRITING AUDIT STANDARDS
Standards should be SMART.

S pecific Specific - Clear, unambiguous and jargon-free. A standard should mean only one
thing to all people who read it.
easurable Measurable – Is the information required to answer your standard available? For
example, “information leaflet should be given to patients”. If data is collected
greed retrospectively, how will you know if it’s a failure of practice or a failure of
documentation?
elevant Agreed - By all concerned with delivering that aspect of care.
Relevant - To area of care being audited / concern that has been raised.
heoretically sound Theoretically sound - Based on evidence about best practice, reviewed and
updated as new evidence becomes available.
Once the evidence-base has been identified and what represents best practice in this area has been agreed,
you need to define the standards that you will measure current practice against. Even if you are basing your
audit on a national guideline, you may need to do some work to make the guideline recommendations into
SMART standards.
At UHBristol we recommend that standards are written as per the model below.

EXAMPLE 2 How to write: An audit standard
The standard relates to the first objective of the Leg Ulcer project example on page 2: ‘To ensure that leg
ulcers are treated appropriately’.
Audit Criteria Target Exceptions Source of Evidence
Venous leg ulcers will be 100% ABPI <0.8 UHBristol Policy – Clinical
treated with graduated multi- guidance on the management
layer high compression (ankle: brachial of leg ulcers (2008)
bandaging pressure index)
Taking each aspect of this model in turn:
Audit Criteria - The audit criteria quantify the practice outlined in the objectives, describing in a measurable
way what care should be delivered. The audit criteria should be derived from the evidence-base described in
the fourth column (Source of Evidence). In the UHBristol audit proposal paperwork, we also ask you to
assess the strength of the evidence-base. The stronger the evidence-base, the more likely it is that staff will
agree with your audit standards and therefore they will be more likely to commit to making changes if your
audit shows the standards are not currently being met.
Target - When writing audit standards, it is normal to set the target at 100%, i.e. the standard statement is
something you will always do; the theory being that, if this is best practice, everyone is entitled to receive it.
Reasons for setting the target at a figure lower than 100% may be:
 Setting target at 0% if referring to something you will never do.
 When comparing outcomes against published evidence (e.g. a national benchmark that 92% of patients
achieve post-operative visual acuity of 6/12 or better following cataract surgery)
Exceptions - The exceptions are the justifiable reasons for not providing the level of care specified. For
example, it is not a failure in our provision of care if a patient chooses not to accept care. A common
exception is patient choice; for example, if a patient was not seen within a standard of 4 weeks from referral
because they were away on holiday.
Consensus on the list of exceptions should be achieved before the start of an audit. Be careful that an
exception is not a failure in care in disguise! For example, ‘patient choice’ may mask the fact that the patient
was not given sufficient information about risks and benefits to confidently agree to treatment.
HOW ARE THESE STANDARDS USED TO MEASURE CARE?
At the end of your data collection and analysis you will have three groups of patients:
i) Those who conformed to the standards.

ii) Those who did not conform to the standards but met the exception criteria.
iii) Those who neither conformed to the standards nor met the exception criteria.
As a result of your data collection you might discover that there are other valid reasons for not meeting the
standard, which you had not previously considered as exceptions. It is therefore important for the audit
team to scrutinise the cases in the third category in order to decide whether the reasons for them not
meeting the audit criteria are acceptable or whether they represent failures in care that can and should be
rectified.

EXAMPLE 3 How to write: Clinical audit topic, aim, objectives and standards
Specialty area: Cardiac
Topic: Screening Post Myocardial Infarct Patients for ICD/CRT-D Implant
Aim:
 To assess compliance with local guidance on referral of post-Myocardial Infarct (MI) patients for
further investigation.
Objectives:
 To check that all STEMI/NSTEMI & TNI +ve patients are followed up post discharge.
 To check these patients receive an echocardiogram at least 4 weeks post event.
Standards:
Source & Strength of
Criteria Target Exceptions
Evidence
All patients with a discharge diagnosis of
STEMi, NSTEMi or ACS +ve Troponin, receive
100% Deceased/DNA Local Policy
a follow-up appointment 4-6 weeks post
discharge
Echocardiogram should be performed or
National Policy – NICE
booked when patient attends follow-up 100% None
guidance
appointment
CLINICAL AUDIT
Office as above.

PATIENT EXPERIENCE
RESEARCH
SAMPLE SIZES
QUALITY IMPROVEMENT
 Further information about clinical audit and wider quality improvement is available via the Healthcare

How To: Set an Audit Sample & Plan Your Data Collection
INTRODUCTION
The aim of this ‘How To’ guide is to provide advice on how to set your audit sample and how to design your
data collection methodology and your data collection form. Aspects of this guide are discussed in more
detail in the following ‘How To’ guides:
1. How To: Engage Patients, Service Users & Carers in Clinical Audit.
2. How To: Apply Ethics to Clinical Audit.
1. SAMPLING
WHICH CASES SHOULD YOU AUDIT?
Your sample population will be dependent upon your audit topic. Occasionally an aspect of treatment or
care that applies to all patients is audited, e.g. nutrition. However, the majority of clinical audit tends to
focus upon the care of a defined group of patients who share certain characteristics. Typically the fact that
they have the same medical condition, have received the same form of treatment or were seen within a
certain time frame; e.g. patients over 50 years of age admitted to the BRI for a suspected MI.
In an ideal world you would audit the care received by all your audit population - i.e. every patient seen for a
given condition over an extended period of time, every treatment received and every outcome achieved - in
order to see whether their care met the agreed standards of best practice. However, if the number of
patients in this population is too large this becomes impractical and you will need to look at a sample of your
overall population instead.
HOW MANY CASES SHOULD YOU AUDIT? – THE SNAPSHOT APPROACH
For research projects it is very important that a scientifically valid sample is selected. This is because
research is at its most powerful when its results are generalisable to a larger population, nationally or
internationally. For example, a previously unproven surgical method would not be adopted without
convincing evidence that it worked, otherwise the implications of a change in practice could be catastrophic.
Clinical audit, however, simply asks, ‘what is happening here?’ so the answer does not have to be as
definitive as it would need to be in research.
The sample selected for a process-based clinical audit project should be large enough that senior clinicians
and managers will be willing to implement changes based on your findings. It is important to be pragmatic;
you are not doing research. A ‘snapshot’ sample is usually sufficient for process-based audit, roughly 20-50
cases. This will enable you to measure whether processes are being followed as per the standards set.
Choosing a larger sample size than is necessary takes up extra time and resources without adding value, and
can mean that there is no time and energy left within your project team to address any issues of below-par
practice and bring about improvement.
It is also important that your sample contains current or recent patients. Clinical audit is about
improvement; we cannot change the past but you can change the future. For example, if your audit project
indicates that the patients seen in the previous month were not given the right drug, changes can be
implemented to ensure that future patients are. If, however, your audit project indicates that patients seen
three years ago were not given the right drug, it is less clear whether that is representative of current
practice. It might be that what constituted best practice three years ago was different; it may be that
changes to systems and protocols mean that it is no longer a problem. Rarely do you need to look at practice
 2005-2017 UHBristol Clinical Audit Team – Version 4 Page 1 of 11

How To: Set an Audit Sample & Plan Your Data
Collection
more than 12 months ago unless for a specific reason, usually connected with outcomes rather than
processes, e.g. looking at outcomes of a rare procedure or where a specific follow-up period is required.
HOW MANY CASES SHOULD YOU AUDIT? – THE SCIENTIFIC APPROACH
Whilst a ‘snapshot’ sample is usually sufficient for process-based audit, if you need greater assurance in your
results without looking at every patient in your population, you may need to calculate a sample size that is
representative of the whole population. This is likely to be the case if you are auditing outcomes, to be
assured that the results you get are within the expected range.
Sample size calculations depend on four variables:

 Size of population.
 Degree of accuracy required.
 Degree of confidence required.
 How often you expect your audit criteria to be met.
The following example shows how this works in practice:
“A primary care team is planning an audit of the care of patients with hypertension. There are 300 patients
(size of population) being treated for the disorder, but the team do not have time to review the records of
them all. The audit standard states that patients receiving treatment should have had their blood pressure
checked and the result below 150-90 on three occasions in the past 12 months. The target for meeting this
standard is set at 70%. However, the team are willing to accept 5% inaccuracy (degree of accuracy) due to
sampling. In other words, if the findings give a level of 70%, on 95% of occasions (degree of confidence) the
true value would lie between 65% and 75%. The public domain software programme Epi Info
(www.cdc.gov/epiinfo) was used by the team to calculate the sample size using the above parameters, and
the sample required is found to be 155.”
NICE/CHI Principles for Best Practice in Clinical Audit (2002)
Strictly speaking, a sample size calculation should be carried out for each standard that is being addressed as
part of your clinical audit project. The sample size chosen for your project should be the largest figure that
those calculations produce.
The table below appears in a number of guides to choosing audit sample sizes and assumes an expected
incidence of 50% i.e. that standards will be met 50% of the time. It gives the sample size you will need in
order to be 95% sure (degree of confidence) that the results you obtain from the sample will be within 5%
(degree of accuracy) of the results you would have obtained for your whole population if you had collected
data on all of them. Put another way, there is a 1 in 20 chance that your results will not be representative.
TABLE 1: Sample size

Population size Sample size: 95% confidence; +/- 5%
50 44
100 79
150 108
200 132
500 217
1000 278
2000 322
5000 357
Using this table, if your audit showed that audit criteria X was met in 56% of cases, you could be 95% sure
that criteria X would have been met in somewhere between 51-61% of cases had we looked at the whole
population.

Collection
The required sample size will be proportionately smaller as the population size increases; looking at 357 out
of 5000 patients gives you results with the same degree of certainty as looking at 44 out of a population of
50 patients. This is because the chance of the results being unrepresentative is dramatically reduced as the
population size increases. Imagine you tossed a coin five times and got four heads and one tail; there could
be a pattern emerging, but it is almost certainly just chance that you got four heads. If on the other hand,
you tossed a coin 500 times, and got 400 heads to 100 tails, we could be pretty certain that there was
something rather dubious about the coin.
Remember, sample sizes can vary according to any one of the following:
1. The expected incidence of the thing you are auditing.

2. The confidence level you want. The confidence level does not have to be 95%. It could be 90%, 99% etc.
3. The level of accuracy you are prepared to accept. The level of accuracy could be 5%, 10%, 1% etc.
The table below illustrates how the sample size might vary for a population of 500:
TABLE 2: Sample size
Confidence level Degree of accuracy Expected incidence Sample size

('best guess')
95% +/- 5% 50% 217
90% +/- 10% 50% 176
95% +/- 5% 40% 213
95% +/- 5% 20% 165
95% +/- 5% 5% 64
95% +/- 2.5% 50% 378
95% +/- 2.5% 5% 185
A sample size calculator, which takes into account population size, confidence levels, accuracy and expected
incidence, is available on the UHBristol clinical audit website. The website details are listed at the end of this
guide.
SAMPLING METHODS
Once you have decided to take a sample and have decided on the size of that sample, the next question is
which cases are you going to include in your audit? The majority of clinical audit projects use random
sampling or consecutive sampling.
CONSECUTIVE SAMPLING
Consecutive sampling is often referred to as convenience sampling. It involves choosing the next, or last x
cases - e.g. the next or the last 50 patients - or alternatively, all patients seen over the course of the previous
OR next month.
Consecutive sampling is an example of non-probability sampling and is often the most practical way of
selecting cases for a ‘snapshot’ sample of the population. However, it is important to remember that the
sample produced may differ in character from the overall population and therefore the audit results may
not be representative of the overall care that is given.
SIMPLE RANDOM SAMPLING

In a simple random sample every patient within your audit population has an equal chance of selection. An
easy way of selecting your cases is to use a random number table, as per the table below. You could take
one number at a time from left to right (2, 0, 1, 7, 4, etc) or two at a time reading down the table (20, 74, 04,
22, etc). These cases then form your sample, e.g. the 20th, 74th, 4th, 22nd patients from a list of all the patients
in your population.
Collection
2 0 1 7 4 2 2 8 2 3 1 7 5 9 6 6 3 8 6 1 0 2 1 0 9 6 1 0 5 1 5 5 9 2 5 2 4 4 2 5
7 4 4 9 0 4 4 9 0 3 0 4 1 0 3 3 5 3 7 0 2 1 5 4 4 7 8 6 9 4 6 0 9 4 4 9 5 7 3 8
0 4 7 0 4 9 3 1 3 8 6 7 2 3 4 2 2 9 6 5 4 0 8 8 7 8 7 1 3 7 1 8 4 7 8 4 0 5 4 7
2 2 4 4 8 9 6 5 6 8 9 5 3 2 5 2 3 8 3 7 1 5 1 2 5 4 0 2 0 1 3 7 5 6 8 7 6 5 8 9
Simple random sampling is an example of a probability sampling method. It should result in your sample
being representative of the characteristics of the whole population, due to random selection reducing the
possibility of any systematic bias that would make the selected group different in character from the overall
population. To ensure representative results this method should be used in conjunction with a calculated
sample size.
Two other probability sampling methods that are less frequently used in clinical audit, but worth
mentioning, are:
QUASI RANDOM SAMPLING

Quasi random sampling is also referred to as systematic sampling. If an overall audit population is 1000, the
representative sample size would be 278. Since 1000 ÷ 278 is approximately 4 you would select every fourth
patient from the overall population. To ensure that every patient in your audit population has an equal
chance of being selected, your starting point needs to be picked randomly. In this instance the starting
number must be between 1 and 4. This means that you could be auditing patients 1, 5, 10, 15, etc, or 2, 6,
11, 16, etc. The start point must be random because if you always started with the first patient, the second
patient would never have a chance of being selected. This is an important consideration from a statistical
point of view.
STRATIFIED SAMPLING
Stratified sampling ensures that the proportion of different groupings present in the population is reflected
in the sample. For example, if our patient population is made up of 75% men and 25% women, taking a
simple or quasi random sample runs the risk of selecting only men when it might be that there are particular
aspects of care being audited which relate specifically to women. So, if your overall population was 500
patients, this number would need to be split in a ratio of 3:1 in favour of men, producing a ratio of 375 men
to 125 women. This would result in your representative sample of 217 patients being split 163 men to 74
women. To select your random sample, separate men and women into two groups and randomly select
from both i.e. 74 women from a population of 125, and 163 men from a population of 375.
REDUCING BIAS
It is important to take care to consider and eliminate potential sources of bias in your sample. The sample of
cases you audit needs to be chosen in such a way that you can reasonably draw inferences about the care
given to the whole population.
Beware of daily, weekly or seasonal fluctuations which may skew your data. For example, conducting an
audit in the week of school half-term may not be representative of care given in the rest of the month or
year, due to some staff being off work at these times. In general, the narrower your time frame, the greater
the risk of introducing bias, i.e. that your results will not be representative of how well the standards are
being met for the population as a whole. Taking a sample across a longer time period and thereby increasing
the number of cases may be a better way to ensure your results are representative.
It is also important to make every effort to ensure that every case in your sample is included in your audit, as
missing cases may skew your results. For example, if a set of casenotes cannot be located in file they may be
with the Complaints team, Legal Services or with the consultant for review because of problems in care. This
might be just the kind of case that will highlight areas for improvement in your audit. It is therefore
important to try more than once to find any missing notes.

Collection
Prospective audits often suffer from bias because cases that met the audit criteria were not picked up; that
is, the audit proforma was not completed and therefore not included in results. This may skew results if
missing cases are non-random, e.g. if a particular staff member never completed any forms.
SAMPLE SUMMARY
1. Identify your population characteristics.

2. Find out how many patients make up your population.
3. Decide whether to look at every case or choose a sample.
4. Process based clinical audit projects usually involve a ‘snapshot’ sample, of roughly 20-50 cases.
5. If you have calculated a statistically representative sample size, select cases randomly.
6. Talk to colleagues who will have the final say about any changes in practice about how many cases
they would like to see included in your audit and weigh this against time, resources and
requirements for statistical validity.
2. DATA COLLECTION
QUANTITATIVE & QUALITATIVE DATA
Clinical audit is usually concerned with gathering quantitative data, sometimes referred to as ‘hard data’.
Quantitative data is numerical data that is used to measure variables, e.g. counting the number of times
certain things are done, whether they are done on time, and to what end. In clinical audit this data is linked
to standards of best practice which define what should be done, how often it should be done and what
outcome is expected.
Depending on your audit topic, you might also want to collect some qualitative data. This is typically
achieved by using a survey to capture patients’ or staff experiences or opinions. Qualitative data is
subjective and is sometimes referred to as ‘soft’ data.
WHAT DATA DO YOU NEED TO COLLECT
Data should only be collected that is necessary to satisfy your audit aim. The data items that you collect
should enable you to measure practice against your audit standards. It is best to avoid collecting superfluous
data items just because you consider them to be interesting or useful, as this will result in more time spent
on your project, but with little or no additional benefit. The collection of superfluous data is also contrary to
the principles of the Data Protection Act, which states that data must be adequate, relevant and not
excessive for purpose.
A useful technique to ensure that your data items are necessary is to use a data matrix. The data items are
listed in the first column and a tick is placed in the box(es) according to which standard(s) each data item
relates to. If you do not have a tick against a particular data item, the chances are that you should not be
collecting it. Alternatively, if you feel that the data item is important, you might want to consider whether
you need to rewrite an existing standard or include an additional standard.
EXAMPLE 1: A data matrix table

Data item Standard 1 Standard 2 Standard 3 Standard 4
Sex 
Age 

Collection
WHERE WILL YOU FIND THE DATA
If data is routinely recorded, either in the patient’s notes or electronically, it is possible to carry out a
retrospective audit, i.e. assessing past episodes of patient care. If the data is not routinely recorded,
prospective data collection is required, i.e. assessing patient care at the time it is given.
Both methods have their advantages and disadvantages. Retrospective audit can be quick but can turn into
an audit of ‘how well care is documented’, rather than ‘how well care is given’. Prospective audit can be
used to gather data that you would not otherwise have access to but you may have problems ensuring that
you capture every case particularly if other people are completing your data collection forms. This problem
can be mitigated through the provision of training accompanied by written and/or verbal information on the
implementation of your data collection process prior to the start of your audit. Additionally prospective data
collection might be affected by bias; if people know they are being audited they might alter their behaviour
accordingly and compliance with the standards might therefore be higher than if they had been unaware.
WHO WILL COLLECT THE DATA
It is usually the responsibility of the clinicians involved in the project to undertake the data collection.
However, staff not directly involved in designing the project are often asked to collect data on behalf of the
audit team. For example, nursing staff might be asked to complete data collection forms for the patients on
their individual wards. For this reason, it is important to make sure that explicit instructions appear on the
form, such as when to collect the data and what to do with the data collection form after completion, e.g. to
whom and where it should be sent for analysis.
In addition your questions should be written clearly so that they do not require an explanation. If other staff
members are going to be responsible for the data collection process they will need to receive training
beforehand.
3. DESIGNING A DATA COLLECTION TOOL
You will now need to design a data collection ‘tool’ to obtain your audit data. ‘Audit tool’ is a generic term
covering any form or system used to facilitate the clinical audit process, which may include paper forms,
web-based forms, spreadsheets or databases. They may also be referred to as ‘audit forms’ or ‘audit
proformas’. The word ‘questionnaire’ is usually reserved for surveys, where patients or members of staff are
being asked questions. Many of the pointers for designing data collection tools apply equally to
questionnaires. Qualitative methods of data collection such as focus groups and in-depth interviews
sometimes generate ideas for clinical audit projects, but are not usually employed as part of a clinical audit
itself.
DATA COLLECTION FORMS
The design of your data collection form is very important. It is important to include the following:
 The audit project title and date of the audit. This ensures that your form is easy to identify if it is lost or
to retrieve once it has been archived at the end of your project.
 The contact details of the project lead. This ensures that the lead is easily identifiable and contactable if
there are any questions or queries about how to complete the form or where to return it. Again, this
information also helps to link the form to a particular audit project.
 The instructions for the completing the form. If the form is to be completed by someone other than
yourself, clear instructions should be provided for completing the form, such as ‘tick the box’ or ‘circle
the appropriate answer’. If clear instructions are not provided people will not necessarily record the
information that you require or they might record information that in ambiguous. This might affect the
results as you might not be able to analyse the data easily or exactly.

Collection
 The instructions for returning the form. If someone else is responsible for the data collection, clear
instructions for returning the form are vital. The need for instructions is not applicable, however, if you
are doing the data collection yourself.
 Data items. All data items must be included. It is important to remember to include the data items
relating to your exceptions. Exceptions will, in part, provide information detailing why certain standards
were not met.
Other issues to be aware of when designing your data collection for are that:
 Questions should be clear and unambiguous.
 The format that you want the data recorded in is clear. For example, Time of admission : (24 hr clock)
 Questions should be well spaced out. Importantly whilst you do not want to clutter the form, be careful
not to use a font size that is too small to be legible.
 If you want opinions or views, allow space for these to be recorded.
 Keep your data collection form as succinct as possible.
DATA PROTECTION ISSUES
Once you have collected your data you might want to be able to identify which data collection form
corresponds to which patient. However, in order to comply with the Data Protection Act and to ensure
patient anonymity, personal details, including name, address, date of birth, etc, should not be recorded on
the data collection form. Therefore the best approach is to number each of your forms using a unique
identifier. A separate piece of paper, or ‘code sheet' should then be kept, which links each unique identifier
to the patient’s hospital number. Without this list, data collection forms cannot be linked to specific
patients.
Occasionally it might be necessary to record patient identifiable information on the data collection form. For
example during a prospective data collection exercise the data collection form might have to follow a
patient through their pathway of care. In this instance the only patient identifiable information that should
be recorded on the form is their hospital ID number. Name, date of birth, etc. should never be recorded.
Other data protection issues to bear in mind are:

 Data collected must be adequate, relevant and not excessive.
 Collected data should be kept securely, so that members of the public cannot access it. This relates to
both electronic and hard copies of data.
 Data should not be kept for any longer than necessary. Completed data collection forms should be
destroyed once your project is complete.
EXAMPLE 2: Code Sheet

Date Collection Form
Code sheet

Collection
PILOT YOUR DATA COLLECTION FORM
Before you collect the data for your entire sample it is important to pilot your data collection form. The
purpose of the pilot is to try out your data collection form on a small sample to make sure that it works,
especially if someone else is going to be collecting the data for you. The pilot may reveal that some of your
instructions on how to complete the form or the questions asked are ambiguous, that the form is difficult to
complete, or that you are simply not getting the information you wanted. It is essential that you analyse
pilot data against your standards to ensure that you can measure current practice against them. Now is the
time to put right any problems so that when you do your audit properly, you will end up with the right
information first time.
The pilot test might also indicate where open questions can be reconstructed as tick-box options. We advise
that where possible you avoid using free text on data collection forms. This is where, rather than asking for a
numerical or tick-box answer, you allow the person collecting the data or filling in the questionnaire to
provide a description of something in their own words. This kind of data can be complex to analyse and is
more suited to the kind of qualitative work, e.g. focus groups and in-depth interviews, which may precede
an audit.
EXAMPLE 3: Reconstructing open questions into tick box options
Topic: Discharge delays from a day surgery ward
Question: What is the reason for delayed discharge?
Responses:
 TTAs not ready
 Waiting for pharmacy to do drugs
 Ambulance late Are there any themes?
 Partner couldn’t find anywhere to park
Turning above responses into tick box answers:
Reason for delayed discharge:
Awaiting prescription drugs (TTAs)

Transport delays
Other (please state)_____________________________________________________________________
Always consider adding an ‘other’ option with additional space to elaborate. There’s usually at least one
case that will not fit anywhere else.
PATIENT QUESTIONNAIRES
All structured patient surveys, whether administered by post, in hospital, or via a one-to-one interview,
are subject to approval by the Trust’s Questionnaire, Interview and Survey (QIS) Group. The contact
details for the QIS group are listed at the end of this guide. It offers advice on survey design and is
responsible for monitoring all survey activity at the Trust. Please contact Paul Lewis, the Patient
Involvement Co-ordinator, for advice on structured surveys/ questionnaires. If you require advice on
unstructured interviews and focus groups, this should be discussed directly with Tony Watkin, the Trust’s
Public Involvement Project Lead. The contact details for Paul Lewis and Tony Watkin are listed at the end

Collection
of this guide. Formal Research projects are subject to approval by a Research Ethics Committee (REC)
and therefore do not require QIS approval.
PATIENT QUESTIONNAIRE DESIGN TIPS
 Consider whether the information you are seeking is already available. The QIS group may not approve
projects if it feels that one group of patients is being excessively targeted.
 Include a covering letter with all patient questionnaires. A template covering letter is available on the
QIS intranet site. The intranet details are listed at the end of this guide. The letter should include contact
details should the patient have any queries that they wish to be answered quickly about the
questionnaire, the background to project and reason for contacting the patients, instructions on how to
return the completed questionnaire, a statement on how you will protect the patient’s confidentiality,
ideally this should be achieved through developing completely anonymised questionnaires, and what
you intend to do with the data collected.
 If you use a postal questionnaire a response rate of 60% and above is considered to be reasonably
successful. To improve the response rate it is considered to be best practice to include a pre-paid
envelope for respondents to use.
 If you intend to send a follow-up letter to any non-respondents, your questionnaires will need to be
coded; otherwise you will not know which patients have responded. Therefore your questionnaires
cannot be truly anonymous. In this instance it should be stated in the covering letter that although
responses will not be anonymous, they will be treated confidentially. It is generally considered to be bad
practice to follow up a non-response by telephone as this will put the patient ‘on the spot’.
 Consider how the results will be fed-back to the patient population. For example, will you send out a
copy of the final report, write an article for Voices magazine, or use the hospital notice boards.
 Keep the questionnaire succinct. It is important not to bombard patients with too many questions as
they might choose not to participate if the questionnaire looks too long. Up to 20 questions is usually
advised.
 Use plain English. It is important to consider whether the patients will understand the terms that you
use. In particular avoid using NHS jargon. Sometimes a phrase that may seem obvious to anyone
working within the NHS may be unfamiliar to patients.
 Avoid asking ‘double-barrelled’ questions e.g. ‘how would you rate the efficiency and friendliness of
staff’?
 Avoid using leading questions. Sometimes the way a question is phrased might suggest the answer that
you want to receive. For example the question ‘Do you think that the waiting times in the outpatient
department are good?’ should be replaced with ‘How would you rate the waiting times in the
outpatients department?’ Tick-box options could be used to answer this question e.g. Excellent, Good,
Average, Bad, Very Bad, No opinion.
 If appropriate a ‘not sure’ or ‘no opinion’ option should be included. If these options are not included
and the response options do not fit the patient’s views, they might just leave the question blank or add
a free text response that might be harder to analyse.
 Filters should be used to direct responses efficiently e.g. ‘if Yes, go to question 5’. Filters make the
process of completing the questionnaire as quick and efficient as possible.
 Include space for additional comments that the patient may wish to add. This can often be a source of
useful data.
 Your sample, if a postal survey, should be checked against the latest hospital records, as sending
questionnaires to deceased patients is a frequent error that can cause distress to relatives. Similarly,
getting patients’ names or other details slightly wrong can also cause offence.

Collection
USING LIKERT SCALES FOR QUESTION RESPONSES
Sometimes you might want to find out the patient’s opinion on a mater, in which case the responses
might be presented as:
Strongly agree Agree Undecided Disagree Strongly disagree
(Please tick the appropriate box)
With an odd number of choices, responses may tend towards the middle. With even number of choices
you will force a decision towards a positive or negative answer.
SUMMARY DATA COLLECTION & DESIGNING A DATA COLLECTION FORM
 A small, convenience sample of current cases may be all that is needed.

 Beware of potential bias in results.
 Do not collect excess data or patient or staff identifiable information.
 Always pilot data collection forms.
FURTHER READING
“An introduction to statistics for local clinical audit and improvement”, HQIP, 2015
http://www.hqip.org.uk/resources/introduction-to-statistics-for-clinical-audit-and-qi
Includes a section on populations and sampling
“Guide to Ensuring Data Quality in Clinical Audits”, HQIP, 2011

http://www.hqip.org.uk/resources/hqip-guide-to-ensuring-data-quality-in-clinical-audits/

Collection
CLINICAL AUDIT
Office as above.
PATIENT EXPERIENCE
RESEARCH
SAMPLE SIZES
QUALITY IMPROVEMENT

How To: Analyse & Present Data
INTRODUCTION
The aim of this ‘How To’ guide is to provide advice on how to analyse your data and how to present it. If you
require any help with your data analysis please discuss with your divisional Clinical Audit Facilitator who will
be happy to help.
1. HOW TO ANALYSE DATA

CALCULATING COMPLIANCE WITH CLINICAL
Audit data comes in three different forms, ‘tick-box’, AUDIT STANDARD
numerical or free-text. Each requires different methods
of analysis, but in each case the aim is to establish which Number of patients
standards are being met (% compliance) and which are who meet standard
not (% non-compliance). If a standard is not being met 100
you need to identify why and consider how practice can x
be improved to ensure that the standard is met in the Number of Number of
future. You may also consider if there were other, patients to patients who
acceptable reasons for the standard not being met, i.e. whom standard
– meet any listed
an exception not considered during the planning stage. applies exceptions
A. TICK-BOX DATA
It is likely that the majority of the data that you have obtained from your data collection form will relate to
yes/no options or tick-box options from a specified list of alternatives. This is known as ‘categorical’ or
‘nominal’ data; data that can be sorted according to non-overlapping (mutually exclusive) categories, where
each subject in a sample can only fit into one category. For example:
 Staff grade: Consultant, Registrar, Specialist Nurse
 Age group: 16-20, 21-25, 26-30, etc.
 Standard met: Yes / No
In such cases, it is usual practice to add up the number of answers recorded for each option and express the
total as a raw number and as a percentage.
EXAMPLE 1:
 Sample size: 50 patients
 Audit criteria: All patients should attend a pre-operative clinic
 Question: Did the patient attend a pre-operative clinic?
 Results: Yes = 32 and No = 18.
A good way of expressing this data is:
 All patients should attend a pre-operative clinic. n=50

 Yes = 32 (64%)
 No = 18 (36%)
The ‘n=50’ indicates how many patients were in the audit sample and is used to calculate the percentages,
i.e. 32/50 = 64%.
 2005 – 2017 UHBristol Clinical Audit Team – Version 4 Page 1 of 8

It is important to remember that yes/no options do not allow for ‘not applicable’ answers. Taking the
example used above, it is possible that certain patients did not meet the standard because they had an
emergency operation. In this instance the answer to the question ‘Did the patient attend a pre-operative
clinic?’ would have been ‘not applicable’. To reflect this, a variation of the percentage calculation is needed:
EXAMPLE 2:
 Audit criteria: All patients should attend a pre-op clinic 32 32
 Exception: emergency operation 100
x
 Results: Yes = 32, No = 5 and N/A (emergency) = 13 =
50 – 13 37
 32 patients attended a pre-op clinic

 18 did not, but 13 of these were emergencies (exceptions)
 Therefore 32/37 (86%) met the standard
 The difference between 64% in the first example, where no exceptions were taken into account, and
86% in this example is significant enough to influence our thinking about how well we are doing with
meeting this standard, so it is important to remember your exceptions!
B. NUMERICAL DATA
Some of the data items you collect are likely to be numerical values, e.g.
 Temperature: 34°, 35°, 36°, 37°, 38°, etc.
 Days post-op: 1, 2, 3, 4, 5, 6, 7, 8, etc.
 Age: 16, 17, 18, 19, 20, 21, etc.
Lists of numbers like this can be summarised using measures of central tendency and dispersion:
 Measures of central tendency look at the middle/common values in a list of data items: the mean,
median and mode.
 Measures of dispersion look at how spread the data is: the range.
MEASURES OF CENTRAL TENDENCY

Length of Stay Number of patients discharged
(days) Ward 1 Ward 2 Ward 3
The mean is the average value, calculated
1 4 4 1
as:
Sum of all the values ÷ Number of values 2 8 7 3
3 12 17 3
The table to the right shows data about 4 18 10 4
length of stay (LOS) on three wards. 5 20 7 10
6 18 4 15
7 12 2 4
8 8 2 2
9 3 5 0
10 0 9 0
For Ward 1, the mean is:
(1x4) + (2x8) + (3x12) + (4x18) + (5x20) + (6x18) + (7x12) + (8x8) + (9x3) = 511 = 4.96
4 + 8 + 12 + 18 + 20 + 18 + 12 + 8 + 3 103
The mean LOS on ward 1 is therefore 5 days (rounded to nearest whole day).
If the same formula were used to calculate the means for wards 2 and 3, you will find that for each ward,
the mean LOS is 5 days. However, the mean is not always the best measure of central tendency.

The LOS for all three wards is illustrated in Chart 1 below. The mean suggests that the data is the same for
all three wards, however the chart indicates that this is not the case. The problem with the mean is what it
does not tell us.
Length of Stay
Number of patients discharged

25
20
Ward 1
15
Ward 2
10 Ward 3
0
1 2 3 4 5 6 7 8 9 10
Day
Chart 1
The data collected for Ward 1 is almost perfectly symmetrical, with the graph illustrating that the data
follows the shape of a ‘bell curve’. Data that conforms to this shape is known as ‘parametric’ data. In this
instance the mean is an appropriate measure of central tendency.
The data for Wards 2 and 3 is non-parametric; their graphs do not form a symmetrical curve. Describing
their notable features, Ward 2 has a significant proportion of patients with a LOS of 3 days together with a
number of patients staying 9 or 10 days. Ward 3 has a peak LOS of 6 days. It can be seen that using the mean
alone with non-parametric data is not very informative. The median and mode can help to convey the
missing information.
The mode is the most commonly occurring value. For Ward 2 this is 3 days and for Ward 3 it is 6 days. This
should be obvious from both the raw data and the graph. If the highest occurrence is shared by more than
one value you could either state them all as modal values or none. For example, if for Ward 3 there were 10
patients discharged on both day 5 and day 6 you could either say there were 2 modal values of 5 and 6, or
that there was no mode.
The median is the mid-point of all the values. For Ward 2, we have data on 67 patients. If we made a list of
LOS, placed in order from the lowest to the highest, the mid-point would be the 34th value, i.e. there are 33
values below and above this. The 34th value relates to a patient who was discharged after 5 days, so this is
the median. For Ward 3, we have data on 42 patients, i.e. there is no single mid-point. In this case, take the
average of the 21st and 22nd value (there are 20 values below and above these two values). The 21 st value
relates to a patient who was discharged after 5 days and the 22nd value relates to a patient who was
discharged after 6 days, so the median is 5.5 days (5+6 divided by 2).
Unless you are well versed in statistics, we would advise that you use all three measures of central tendency
or show the information using a graph. In general, quote median rather than mean for non-parametric data.
Not all lists of numerical data should be analysed in this way and obtaining figures for compliance with your
standards is still the principal aim of analysing clinical audit data. For example, if your standard is “The
patient will be considered medically fit for surgery if temperature <38°C” and you collect a list of
temperature data, it would not be meaningful to present the mean, median and mode temperature. What
you are interested in here is the percentage of surgical cases with temperature <38°C.

MEASURES OF DISPERSION
As well as stating the mean, median and mode, it is also good practice to provide some indication of how
spread the data is. The range states the lowest and highest values. In our example:
Ward 1 has a range of 1-9 days

A more subtle way of expressing dispersion is to use quartile range. This involves listing your values from
lowest to highest, as per calculating the median, and then dividing the values into four equal parts or sub-
ranges. The range you are interested in lies between the second and third quarter (or ‘quartile’).
So, for example, some more LOS data:
Ward C: 1 2 3 5 | 5 6 6 7 | 7 7 7 9 | 9 11 20 38
In this case the range is 1-38 days, but the quartile range is 5-9 days.
The quartile range is useful in taking out outlying data (data some distance away from the median), as in the
case of Ward C above and Ward 2 in our first example. Ward 2 has the largest range but a comparable
quartile range to Wards 1 and 3:
Ward 1 - range 1-9 days; quartile range 4-6 days

ANALYSING DATA AGAINST STANDARDS

If your standard statement was ‘Patients should be discharged by the end of their 5th day following surgery’,
using LOS data for ward 2, you find that 45 out of the 67 discharged patients had a LOS of 5 days or less.
You would write this as 45/67 (67%) patients met the standard.
C. FREE-TEXT DATA
If you include an open question in your data collection form, you will obtain free-text data. In order to
analyse this data you should group comments into themes or categories, i.e. as if you were creating
categorical tick-box options for the data collection form. You might also want to consider reproducing some
comments verbatim in your report if they are particularly pertinent.
DRAWING CONCLUSIONS
The end stage of your analysis is concluding how well the standards were met and, if applicable, identifying
reasons why the standard was not met in all cases. These reasons might be agreed to be acceptable (and
could potentially therefore be added to the exception criteria for the standard in future) or will show what
needs to be your focus for improvement. In theory, any case where the standard (criteria or exceptions) was
not met in 100% of cases suggests a potential for improvement in care. In practice, where standard results
are close to 100%, it may be agreed that any further improvement will be difficult to obtain and that other
standards with lower results should be the priority targets for action. This decision will depend on the topic
area; in some ‘life or death’ type cases it will be important to achieve 100%, while in other areas a lower
(but still high) percentage might be considered acceptable.

2. DISPLAYING DATA
USING TABLES
The simplest way to present data is in a table and this is the best way to show compliance with your audit
standards (see Table 1); in fact, our audit report template will ask you to provide this regardless of what else
you do in the way of analysing and displaying your data.
Table 1: compliance with audit standard Table 2: category of caesarean sections in audit
Standard Target Result Category of urgency Frequency Percentage
All prescriptions should
100% 80% (40/50) Category 1 12 21%
include frequency of dose
Category 2 27 47%
Category 4 (elective) 18 32%

USING CHARTS
The purpose of a chart is the visualisation of data, i.e. providing useful information in a graphical form. It is
simple enough to enter figures into a spreadsheet and hit the “insert chart” button, but in order for charts
to be meaningful and useful to your audience a little more thought and preparation is often necessary.
The key principles to think about are:

- What message am I trying to convey?
- Do I need to represent the data graphically to get the message across?
- What kind of chart will deliver my message in the clearest way?
- What information could I include in order to anticipate and answer my audience’s questions?
 Bar/column charts – For categorical data –

Generally used to show frequency, e.g.
number of patients meeting the standard /
not meeting the standard, or the number of
patients seen by different staff groups.
For example, if your standard stated that all

patients in A&E who meet certain criteria
should be seen by a consultant, you might
want to show your audience what grade of
staff saw the patients if it was not the
consultant (see Chart 2, right).
Chart 2
 Versions of bar charts (stacked or

comparative) – You can show more than
one standard and/or more than one
audit per chart, either by displaying bars
with different values next to each other
(Chart 3) or by stacking bars on top of
each other (Chart 4).
Chart 3

Chart 4 Chart 5
 Pie charts - For categorical data – Used to show proportion of parts compared to a whole, e.g.
percentage compliance with a standard, or the types of device fitted in a sample of patients whose
devices failed (see Chart 5). As a general rule, keep the number of slices low (advice differs on the
maximum, but above 6 slices your chart may become hard to interpret) and avoid using this type of
chart if the values are all very similar, as small differences in the size of slices are difficult to see.
Another occasion to avoid using a pie chart is when the parts do not add up to a meaningful whole; for
example, if you wanted to illustrate ‘type of treatment’ in cases where patients are likely to have had
more than one treatment each. In this instance the pieces of the pie would add up to the total number
of ‘treatments’ rather than the total number of patients, which could be misleading and is probably not
very meaningful in itself. A bar chart could be used instead.
 Line charts – Can be used to show change

over time, e.g. ongoing compliance with a
regular monthly audit of hand hygiene or
equipment checks
Chart 6
CREATING GOOD CHARTS
Good charts should focus on getting your message across rather than creating fancy and distracting images.
Clutter should be avoided and the charts clearly labelled.
EXAMPLE CHART 1:
A chart might be considered unnecessary
to illustrate this result. Consider whether Did member of staff take personal protective
equipment with them?
people need to see a graphical
representation of your data; in this case
simply expressing the data as ‘37/40
members of staff (93%) took personal Yes
protective equipment on domiciliary 3

visits’ should be sufficient. In a project
with a lot of standards producing a chart No
37
for every single one may confuse rather
than clarify the results. People may not
remember which image related to which
standard.

EXAMPLE CHART 2:
What’s wrong?
 The 3D effect makes it difficult to read how
many “yes” or “no” answers there are.
 The title needs more detail.
 The axes should be labelled. In this case the x
axis might not need labelling according to
what other titling is put on chart, but the y
axis needs to be labelled to be meaningful.
 In this instance the legend is not needed, as
there is only one data series (unlike in the
stacked or comparative bar charts shown in
charts 3 and 4)
 There is a lot of white space on the page. It
looks unprofessional.
Revised version
 It has been changed to a 2D chart.
 Titles have been added.
 There is better use of the space available.
 The scale has been extended a little way past
the highest bar.
 The sample size number has been added for
quick reference.
FURTHER READING
“An introduction to statistics for local clinical audit and improvement”, HQIP guide, 2015
http://www.hqip.org.uk/resources/introduction-to-statistics-for-clinical-audit-and-qi

CLINICAL AUDIT
Office as above.
PATIENT EXPERIENCE
RESEARCH
SAMPLE SIZES
QUALITY IMPROVEMENT

How To: Share Your Findings -
Clinical Audit Report & Presentation
INTRODUCTION
Once completed, clinical audit projects should be both presented verbally and written up as a report. Both
should state how well the standards are being met and highlight any problems that need to be addressed. The
aim of this ‘How To’ guide is to provide advice on sharing your clinical audit results, addressing both report
writing and designing a presentation.
1. BASIC CONTENT: REPORT AND PRESENTATION
1. TITLE PAGE/ SLIDE

 Name of the organisation and name of division/specialty
 Project title
 Project lead/s (and name of the person who wrote the report, if different)
 Date of report/ presentation
2. BACKGROUND
This section explains the rationale for doing the audit, i.e. why it is a priority for quality improvement. The
evidence-base for the audit topic should be summarised, with full references provided at the end of the
report. If you convened a team to undertake the audit, this is a good point to explain how this was organised
and who was involved.
3. AIM, OBJECTIVES & STANDARDS

This section sets out the aim, objectives and standards of your clinical audit project.
 Aim - Defines what you hope to achieve, i.e. the overall purpose of the project.
 Objectives - Defines the individual steps that need to be taken in order to achieve your aim.
 Standards - The quantifiable statements detailing the specific aspects of patient care and/or management
that you measure current practice against. You should specify the audit criteria, target, exception(s) and
source(s) of evidence.
4. METHODOLOGY
This section should outline:
 The population for your audit project, e.g.
 “Patients aged over 50 years of age admitted to the BRI for a suspected MI”.
 Whether it is a retrospective or prospective audit, e.g.
 “A prospective audit assessing the first 30 patients aged over 50 who are admitted to the BRI for a
suspected MI from 01/10/17”.
 “A retrospective audit looking at all patients aged over 50 who were admitted to the BRI for a
suspected MI during September 2017”.
 How these patients were identified, e.g. from Medway, laboratory systems, radiology database etc.
 Sample size.
 Time period audited.
 The data collection method, e.g.
 “Data was collected from patients’ case notes using a data collection form (see Appendix A)”.
 “Patients were asked to complete a patient survey following their consultation (see Appendix A)”.
 Who was responsible for data collection.
 The method of data input (if appropriate) and analysis, e.g. data was input into and analysed using
Microsoft Excel.
How To: Share Your Findings - Clinical
Audit Report & Presentation
5. RESULTS
The results for each standard should be presented in this section to establish which standards are being met
and which are not. If you find a standard is not being met you need to identify why and how practice can be
improved to ensure that the standard is met in the future. You may also consider if there were other,
acceptable reasons for the standard not being met, i.e. an exception not considered during the planning
stage.
The results for each standard can be presented using graphs to further illustrate, if appropriate.
In your clinical audit report and presentation you should state how many patients were included in the audit;
this is your initial ‘n’ number. If your data is incomplete you need to explain why, e.g. you were unable to find
every set of patient notes.
Ensure you include both the number and percentage of cases meeting each standard, as well as the ‘n’
number (i.e. the number of patients audited for each standard), e.g. 45/50 (90%).
Data may be presented as pie charts or bar graphs. Be selective in your use of charts; only illustrate the key
findings in this way so as not to overburden readers. Use the most appropriate chart for each piece of data,
e.g. pie charts to show proportions, or bar charts for easy comparisons. In reports, charts should be
accompanied by a table showing the raw figures; these will be needed when it comes to re-auditing and
comparing results.
IMPORTANT: Individual healthcare staff should not be identifiable in your report. If, for example, you are
comparing the results of three consultant firms, you could call them A, B and C. For confidentiality reasons it
is also important that patients are not identified.
6. CONCLUSIONS
This section should list the key points that flow from your results. Ensure your conclusions are supported by
the data, or if your data points to no firm conclusions, say so. Make objective, factual statements, not
subjective ones, i.e. do not say ‘it is obvious that…’ or ‘clearly, what is happening is…’.
Where appropriate, recommendations for change should be made. Make sure these are realistic and
achievable. If money is needed to implement your recommendations, you should have already investigated
whether there are suitable funds available.
THE REPORT SHOULD ALSO CONTAIN
7. ACTION PLAN
Your action plan should be finalised after you have presented your project e.g. at a clinical audit presentation
meeting. It is as a result of your presentation that your recommendations will either be accepted or revised.
Following this, an action plan should be agreed, preferably at that meeting, to confirm what changes will be
implemented, who will be responsible and when they will be implemented by. Your action plan should be
included in either the body of the report, or if the report has already been completed, as an addendum to the
full report. If appropriate a date for a re-audit should be included in order to complete the audit cycle.
8. REFERENCES
Full references for any literature discussed in your background section as well as for the source(s) of evidence
that your standards are taken from must be provided. It is important to be consistent in your referencing. The
Vancouver style, favoured by the BMJ and other journals, numbers references in the order in which they
appear in the text. For each reference provide the names and initials of all authors followed by the title of the
article, the title of the journal, the year of publication, the volume number and the first and last page
numbers. References to books should give the names of any editors, place of publication, publisher and year.

9. APPENDICES
A copy of your data collection form should be included in the appendix.
2. PRESENTATION
Your presentation should get the message(s) of your clinical audit project across to key staff, generate
discussion and then agreement about changes to practice (in light of the audit results).
You might want to formally present your project at a divisional clinical audit presentation meeting before a
large audience. Alternatively your project might be presented more informally at a staff meeting or in 1-to-1
meetings with key staff, or by distributing copies of your audit report. You should discuss this with your
divisional Clinical Audit Facilitator in order to choose the method(s) most appropriate for your project.
Presentations should be ‘punchy’ in style, compared to audit reports. Think about them as a four stage
process:
1. Planning
2. Writing
3. Practicing
4. Delivering
PLANNING YOUR PRESENTATION
 What are your objectives? Are you giving an ‘influencing’ presentation (perhaps you are using the results
of an audit project to try to lever additional resources) or a factual presentation (simply imparting
information)? Are you using the presentation as an opportunity to open up a dialogue?
 Who is your audience? How much knowledge will they have about your subject? You might have a mixed
audience, in which case you will need to be sensitive to all their needs so as not to lose some or bore
others. Knowing your audience is also important if you are thinking about using anecdotes or injecting
humour into your presentation. If you are in any doubt about the appropriateness of using anecdotes or
jokes, do not use them.
 How much time do you have? Remember to keep it short and simple (KISS). What are the points you must
get across, as opposed to the things you could say? Presentations should only take a maximum of 10 – 15
minutes; padding and waffle will just detract from your message. If necessary you could save time by
distributing some background information prior to the meeting or as a handout. If you do this, explain in
your presentation that you are assuming the audience has / will read the paper.
 What visual aids are you going to use? It is usual nowadays to project slides directly from a laptop PC via
an LCD projector, however other visual aids like posters or flip-charts may be more appropriate to the
meeting you attend. Make sure any equipment you need will be available. Please contact the Clinical
Audit Central Office if you want to book our lap-top and projector (contact details are listed at the end of
this guide).
WRITING YOUR PRESENTATION
 Use Microsoft PowerPoint where possible. Your divisional Clinical Audit Facilitator can help you with this if
needed.
 Do not put too much information onto a slide. Sitting through a presentation with lots of detailed
information written in a small font size can be frustrating. Use your slides to convey your key messages,
using bullet points to punctuate your talk. Do not read straight from the slides, talk around the text on the
slide; this reinforces the impression that you know what you are talking about and helps maintain your
audience’s interest. The bullet points should be sufficient to prompt you if you lose your thread halfway
through the talk. Try using a 1-6-6 template for each slide: have no more than one main idea on each
slide, expressed in no more than six lines of text, with no more than six words per line; this is only a
guideline, use your judgement as to whether a completed slide can be easily read and understood or not.

 Use graphs and tables where appropriate. Some of your audience will like to see data shown in tables,
whilst others will prefer the visual impact of graphs. Try to use a balance of these methods, according to
what feels most appropriate for your data.
 Only use abbreviations and acronyms if your audience will understand them or if you have written them
out in full in the first instance (best practice). Think about fonts and colours. Choice of font can affect the
tone/ impression of your presentation. Using colour can add interest and help to emphasise key points,
but do not overdo it. Be aware that some colours do not go well together, e.g. yellow tends not to show
against a white background. A white background is often best when printing out handouts from a
PowerPoint presentation. LCD projectors allow use of animation to add interest to your presentation, e.g.
bullet points and images can be made to fly onto the screen and then fade away afterwards. Use these
facilities sparingly; any visual aids should enhance your presentation, while too many clever tricks may
distract the audience from your message.
PRACTICING YOUR PRESENTATION
 Pretend it is for real. Give your presentation to a friend or colleague, or even to the bathroom mirror. The
more realistic you can make it, the better. Have your presentation materials to hand and try not to stop if
you make a mistake - if you make a mistake on the day, you have to keep going. If the equipment you are
going to use on the day is not available for you to practice with, at least make sure that you are shown
how to use it on the day. It may also be an idea to visit the room you will be presenting in if you have
never been there before.
 Time yourself. Time usually goes more quickly than you imagine when delivering a presentation. Make
sure you can keep to your allotted time slot.
 Some people find it helpful to have ‘prompt cards’ to talk from, i.e. small cards with key points written
down as reminders of how you are going to structure your talk. A hard copy of the presentation materials
may serve the same purpose, although this is likely to be bulkier. If you are using a projector and laptop,
you can have the laptop screen in front of you to use as a prompt.
 Try to anticipate questions. If you were listening to your talk, what are the obvious questions you would
ask? If you are aware of any flaws in your project, head this off in your presentation by including
explanations and/or counter-arguments.
 Different techniques for different equipment. If you are using a laptop with LCD projector, you can set up
the presentation so that bullet points or images appear one at a time, so as to focus the audience’s
attention.
DELIVERING YOUR PRESENTATION ON THE DAY
 Dress appropriately, remembering who your audience is.

 At the start of your presentation, explain why you are there and give a brief summary of what you will be
talking about. If you are confident, you could use a rhetorical device to capture your audience’s attention,
e.g. an anecdote that illustrates your theme, a rhetorical question (‘How often have you thought….?’, etc),
or a shock statement (‘This audit shows that we’re in the bottom 5% in the country for….’)
 Aim to:
1. Tell your audience what you are going to tell them (simple bullet points at start of presentation).
2. Tell them (the main content of your presentation).
3. Tell them what you have told them (summarise key points at the end).
 Remember to smile. It will help you to relax and show that you are enthusiastic about your subject. And
be confident, even if you are presenting to a room full of the ‘great and good’, you know more about your
project than your audience. You are the expert; you are the one who has something to say.
 If you have handouts, decide whether you will distribute these at the start or the end of your presentation
and tell your audience.
 Tell your audience whether you want to take questions as you go or at the end of the presentation. When
you are asked a question, take your time answering (silence is never as long as it feels) and if you do not
know something, be honest and do not bluff. If it becomes obvious that you are bluffing it will undermine
the credibility of your presentation. You can always say ‘I do not have that information to hand but will
find out and get back to you’. Do not allow yourself to become flustered. Speak clearly and loudly enough
to reach everyone in the room. Consider whether or not there is a sound system or microphone that you
can use. Varying the pace of your speech can be effective, monotone presentations can send an audience
to sleep.
 Address everyone in the room. If looking at people puts you off, try looking just above their heads. If you
need to look at the screen, stand near it and speak sideways. If you turn your back people will not be able
to hear you.
 Try not to fiddle, scratch or jangle coins. It is easily done, but can be very distracting for the audience.
 Be careful if you have a tendency to repeat certain words and phrases, e.g. finishing every sentence with
‘right?’, saying ‘er’, ‘kind of’ and so on. These can be really distracting. Try not to shift your balance from
one foot to the other. Moving around the stage can make things more dynamic or it can be a distraction.
You want the audience to concentrate on what you are saying, not on you. Finish positively, give a quick
summary of your key points and positively invite questions.
AFTERWARDS
 At the end of your presentation consider whether or not you have achieved your objectives. If you need
the audience to reach a decision about something before the end of the meeting, make sure the chairman
of the meeting is fully briefed. For instance, if your results show that you are not meeting the standards,
e.g. a performance of significantly less than 100%, you need agreement as to what changes in practice will
be made and when a re-audit will be done to confirm improvement. Plans made at the end of your
presentation will help with the formulation of your action plan, which should be included in your audit
report. Alternatively your audit results may suggest new exceptions to the audit standards. You might
want to get agreement to update the standard to include these exceptions.
 Be brave and ask your colleagues for constructive feedback, so you know what worked well and what to
improve on for next time.
3. TIPS FOR REPORTS
The finalised report acts as the official record of what you have done and contains more detail than your
presentation. It should include all the information needed to plan a re-audit. Clinical audit reports should be
written up in sufficient detail for a reader to be able to ascertain how the audit was conducted.
 Think about who is going to read your report and gear the content and style accordingly. With this, also
think about what you are trying to achieve, e.g. if you need to persuade people of the need for action,
make sure your report reads persuasively and puts a good case.
 Make it look professional and interesting. Use colours and fonts sensibly and consistently. Arial is an easily
readable font.
 Your report should be written in plain English and have a logical flow to it. Make the structure explicit with
section headings and paragraphs. Use page numbers.
 Every word in your report should count for something. Do not embellish your report unnecessarily, e.g.
‘With this in mind…’ or ‘All things considered…’ An audit report should be largely descriptive, i.e. a
statement of fact. However, when analysing the responses to open questions it might be appropriate to
include the opinions and anecdotes of the respondents.
 When using abbreviations and acronyms it is good practice to write these out in full in the first instance.
 It is often a good idea to ask someone else to proof-read your report before you distribute it. They can
check for errors and ensure that the report is easy to understand and that it flows well.

SUMMARY
 All clinical audit projects should be both presented verbally and written up as a report.
 Both the report and presentation should include:
1. Title Page/ Slide
2. Background
3. Aim, Objectives and Standards
4. Methodology
5. Results
6. Conclusions
 The report should also contain:
1. Action Plan
2. References
3. Appendices
 The presentation should be used to get the message(s) of your clinical audit project across to key staff and
should generate discussion and agreement about changes to practice in light of the audit results.
 It should be more ‘punchy’ than the report.
 It is important to prepare for your presentation. Think about:
1. Planning
2. Writing
3. Practicing
4. Delivering
 The finalised report acts as the official record of the project.
 The report should include all the information needed to plan a re-audit.

CLINICAL AUDIT
Office as above.
PATIENT EXPERIENCE
RESEARCH
SAMPLE SIZES
QUALITY IMPROVEMENT

How To: Implement Change Successfully
INTRODUCTION
“The most important part of the audit cycle is making change” - Baker et al (1999)
“A clinical audit which does not result in improved services and increased patient safety is a waste
of resources.” - Healthcare Quality Improvement Partnership (2014)
The aim of this ‘How To’ guide is to provide advice on how to implement change successfully. If an audit
shows that current practice needs to be improved, making changes is important. The public has the right to
expect that practitioners will provide care that is consistent with recognised good practice. However, it is
important to bear in mind that not all changes are necessarily improvements. Do not make changes for
change’s sake. At an appropriate time, repeat the audit (re-audit) to ensure that changes have been
implemented and that practice has improved.
PLANNING AHEAD
Clinical audit is recognised as an effective means of changing clinical practice to bring about improvements
in patient care, management and outcomes; this does not mean that change is easy. It is the most difficult
part of the clinical audit cycle, and the point at which projects are most likely to lose momentum.
To maximise your chance of success, design your project from the outset with the following in mind:
 Ensure staff are motivated to improve practice. If the audit does not interest anyone else, or if you are
doing an audit simply because you have to, you are less likely to bring about change.
 Involve all the key players. If all of the people who will have the final say about changes in practice are
involved with the project from the very beginning the likelihood that the proposed changes will be
agreed and implemented will be increased.
 If there are additional costs associated with the proposed change, ensure that management
understands and support the proposal. If you do not get this agreement before starting your project, it is
less likely that you will be able to get the funds you require to make the change.
 Use robust methodology in your project. If people are confident in the validity and reliability of your
results they will be more likely to make the changes indicated by the results.
Be aware that change may be perceived positively or negatively.
MAKING RECOMMENDATIONS FOR CHANGE
Before you take any steps to make change, it is worth considering whether you fully understand the
fundamental reasons for the shortfalls identified by the audit. There may be an opportunity to use root
cause analysis (RCA) and related techniques to explore this further.
5 WHYS
This technique is straightforward enough: it involves repeatedly asking the question “why?” (5 times is a rule
of thumb) in order to get to the root cause of a problem. For example:
 The patient's diagnosis of skin cancer was considerably delayed. Why?
 The excision biopsy report was not seen by the surgeon. Why?
 The report was filed in the patient's notes without being seen by the surgeon. Why?
 It was the receptionist’s job to do the filing. Why?
 The junior doctors were busy with other tasks. Why?

 The root cause - that the doctors' other tasks were seen as more important than filing. The
system has now been changed. A copy of all biopsy reports is now sent to the consultant surgeon
responsible for the patient and no reports are filed unless they have been signed by a doctor.
FISHBONE DIAGRAMS
Also known as ‘Cause and Effect’ diagrams,

these are a good way of breaking-down all
the contributory causes of an observed
effect, including less-obvious factors. This can
help you decide which factors are most-likely
to be causing the problem and decide what
you can tackle most effectively.
PROCESS MAPPING
Depending on the complexity of your audit, it may

be valuable to look in much greater depth at patient
journeys through your service. For example, if your
audit has identified that patients are not receiving
blood test results in a timely manner, you may be
able to use process mapping to establish where
problems arise and whether there are steps that can
be eliminated or redesigned in order to make the
pathway more efficient.
All these exercises could involve wider consultation

with multi-disciplinary teams; the more you engage
at this point with the staff members who will
ultimately be adopting proposed changes, the more
likely they are to be successful.
[Source for all above examples: www.institute.nhs.uk]
THE CHANGE PROCESS
There are three main stages to the change process. These are summarised below:
1. Initiation - The process leading up to the change.
2. Implementation – The first experiences of change.
3. Continuation – The changes become embedded.
INITIATING CHANGE
You will need to analyse the situation before you think about suggesting changes.
 Do people recognise the need for change? The presentation of your audit results should be used to
notify people of the need for change and to ‘sell’ to them your recommendations for change.
 Sometimes people will readily recognise the need for change, perhaps there have been a series of
critical incidents in a particular area, whereas on other occasions you may need to highlight the
importance of change.
 Willingness to change varies from person to person. For example, someone who has been working in a
particular clinical area for a short period of time might be more open to the idea of change than
someone who has been working there for a longer period.

 You may need to sell your proposal. An important factor to bear in mind is that, whilst clinicians will be
interested in what the proposed changes might mean for their patients, they will probably be most
concerned about the implications for them personally.
 People respond to different stimuli when it comes to thinking about change. For some, a shared vision
of the future will suffice. Others will want to be persuaded by facts and figures.
 There may be individuals who will only change practice if a reward or penalty is at stake. Use power or
influence where you can.
 The majority of any group will accept changes in response to the action of opinion leaders, i.e. people
who are well respected. It is therefore important to have opinion leaders on your side; this is particularly
true if potential barriers to the proposed change are cultural ones relating to existing routines or
practices.
USEFUL TOOLS FOR CHANGE ANALYSIS

Before implementing change, you may need to devise a strategic plan. There are a number of useful tools
available to help you do this. Three of the most popular tools are detailed below, which will help you to
anticipate different reactions and counter potential resistance.
1. TROPICS - This is a good way to get a feel for the nature of a particular change and plan an appropriate
strategy:
T Time scales - Defined? Short or long term?

R Resources - What will be needed?
O Objectives - Are these quantifiable?
P Perceptions - Does everyone see this issue the same way?
I Interest - Who has an 'interest' in making change happen/keeping things the same?
C Control - Who holds the power?
S Source – Who is driving this proposal, internal or external source?
Note: Externally generated ideas for change, i.e. from a different organisation or department, tend to create
most resistance. Staff feel as though they have less control.
2. Stakeholder Analysis - This is a framework for

thinking about where your colleagues might
stand in relation to the proposed changes and
the most appropriate approach for you to take
with them.
3. Forcefield Analysis - This is a way of visually mapping out the forces that are likely to help or hinder you.
You can use different length or different thickness lines to show the varying strengths of the forces.
Driving Restraining
Forces Forces
By identifying the pros and cons you can develop strategies to reduce the impact of the opposing,
restraining forces and strengthen the supporting, driving forces. As a rule of thumb, it is better to reduce
restraining forces, which can be rational or emotional, rather than increase driving forces. Driving and
restraining forces might relate to an individual, staff group or to the organisation as a whole.

They can include the following questions:

 Have past experiences of similar situations/changes affected people's views?
 Is management supportive?
 How unacceptable/undesirable is the current situation perceived to be?
 Are there fears about increased workload?
 Is there pressure for change from patients?
 Are there national policy requirements we must comply with?
Forcefield Analysis can be used effectively in combination with TROPICS and/or Stakeholder Analysis.
IMPLEMENTING CHANGE
Assuming you have won your colleagues over to the idea of the proposed change, you will now want to
implement it. You may need to plan the implementation phase, even if it is simply a question of purchasing a
piece of equipment.
You may need to break down the changes into manageable tasks and achievable targets. Crucially, it also
means communicating, e.g. informing staff about what is going on and consulting them for their own ideas.
Different objectives require different methods of communication. Sending out a newsletter about a new
clinical guideline is, on its own, unlikely to change clinical practice. It is therefore important to consider
whether or not there is a need for training and development, e.g. organising briefings/ workshops.
A useful rule to remember WEAK ACTIONS STRONG ACTIONS

is that a strong action Raise staff awareness Remove barriers to doing the work effectively
makes it easier to do the Remind staff Redesign the work
right thing and harder to do Provide training Supervise, monitor and feed back
the wrong thing. Write a new policy Use IT or technological solutions
MODEL FOR IMPROVEMENT
You might decide to pilot the change that you are planning to make; this is particularly
important if you need to demonstrate the benefits to previously unconvinced staff. One
way of approaching this is to use the Model for Improvement, which is based on
answering three fundamental questions:
 What are we trying to accomplish?

 How will we know that a change is an improvement?
 What changes can we make that will result in an improvement?
These three questions combined with the Plan-Do-Study-Act (PDSA) cycle form the basis
for the model.
 Plan – decide how to test the change that

you want to make – start small in the first
cycle
 Do – carry out the test
 Study – observe and learn from the test
 Act – determine what should happen next based on the results
of the test – make modifications to the proposed change
where indicated and repeat the cycle

You will usually require a number of short test cycles to refine a change idea. Each subsequent test can be
on a wider-scale than the previous one, until you are ready to fully implement the change; you could, for
example, start with one nurse on one ward testing a new care plan as your first cycle, make amendments
using the learning from that and expand to the whole ward in the second cycle, before implementing it
division-wide with a much better insight into how it works in practice and with the support of the staff who
will be using it, who should feel more involved in and able to influence the change process.
CONTINUATION OF CHANGE
Even if you manage to get changes implemented, it does not mean they will stay implemented. People
sometimes slip back into the old ways of working. Once again, communication is crucial, both with the
members of staff who are implementing the change and with management. Provide staff with evidence that
the changes have had a positive impact through a re-audit. If other staff members are slow to come on
board with the changes, can management encourage them to move their position?
Agree /
REASONS WHY CHANGE SOMETIMES FAILS
review
standards
Change can fail at any one of the stages outlined above; frequent Implement Collect data
reasons for failure include a lack of resources, a lack of motivation, change if on current
inadequate management of the process, or poor communication. needed practice
Compare
Usually factors that hinder change can be addressed by planning data with
your clinical audit project properly: standards
 Create a multi-professional/multi-disciplinary audit team with a representative from each staff group
involved in the care being audited; this will increase ownership of the problems and improve motivation
for change.
 Ensure you have involved people with authority to agree changes.
 If you are likely to need resources to implement changes, ensure that management is in agreement with
the aims of the project and will provide funds if necessary.
Prompts reminding you to consider the issues above are included in the UHBristol Clinical Audit Proposal
paperwork, which you must complete before you start any clinical audit project. A copy of the clinical audit
proposal form is available on the clinical audit website or from your divisional Clinical Audit Facilitator -
details for both are listed at the end of this guide. The clinical audit proposal form serves not only to register
your project, but also to improve the planning and design of your project, which in turn will increase the
likelihood that it will lead to improvements in practice, for the benefit of patients.
NOTIFYING THE CLINICAL AUDIT DEPARTMENT OF CHANGES IN PRACTICE
At UHBristol, project leads are asked to complete a Summary form and Action Plan form on completion of
their audit project (both are available on the clinical audit website or from your divisional Clinical Audit
Facilitator – details for both are listed at the end of this guide); these forms detail what results have been
found and what action is planned to address any areas needing improvement. The Action Plan format helps
to break down change into manageable tasks of who is doing what action, and when.
You should keep your divisional Clinical Audit Facilitator informed about the progress made with your action
plan, so that the Trust has a record of what improvements have been made to care as a result of following
the Clinical Audit process, and can therefore demonstrate the benefits of UHBristol’s Clinical Audit
programme.

SUMMARY: CHANGE & RE-AUDIT
 Get people on board with your proposal.

 Write and implement an action plan.
 Consider piloting change first and review.
 Re-audit to confirm improvement
REFERENCES AND FURTHER READING
1. “Taking action to improve quality: skills and tools to train and support colleagues”, HQIP, 2014
2. “Guide to Quality Improvement”, West of England Academic Health Science Network
3. NHS Institute, Quality and Service Improvement tools

www.institute.nhs.uk – now archived on the NHS England website
4. "Guide to quality improvement methods", HQIP, 2015

http://www.hqip.org.uk/resources/guide-to-quality-improvement-methods/
5. "Using root cause analysis techniques in clinical audit", HQIP, 2016

http://www.hqip.org.uk/resources/using-root-cause-analysis-techniques-in-clinical-audit/
CLINICAL AUDIT
Office as above.

PATIENT EXPERIENCE
RESEARCH
SAMPLE SIZES
QUALITY IMPROVEMENT

How To: Involve Patients, Service Users &
Carers in Clinical Audit
INTRODUCTION
The aim of this ‘How To’ guide is to provide advice on how to involve patients, service users and carers in
the clinical audit process. The Healthcare Quality Improvement Partnership (HQIP) state that the
involvement and engagement of patients, service users and carers in clinical audit is vital and is a marker of
high quality audit. If you are thinking about involving patients, service users and/or carers in your project it
is important that you discuss this with your divisional Clinical Audit Facilitator at the planning stage. Aspects
of this guide are discussed in more detail in:
 How To: Set an Audit Sample and Design your Data Collection & Data Collection Form.
 How To: Apply Ethics to Clinical Audit
THE ENGAGEMENT AGENDA
The idea of public and patient engagement within the

NHS is to enhance the quality of healthcare services by “Clinical audit must include patients. They,
ensuring that they are oriented, planned and delivered to as well as clinicians, are the true
meet patient needs and interests. In particular, involving professionals in illness. The doctor may be
patients in clinical audit provides a mechanism by which highly qualified to diagnose and give
NHS organisations can show that what is being measured treatment. However, only the patients truly
actually matters to patients. know the pain, physical and psychological,
and the stress of their illness. The patient’s
The Department of Health has published a range of voice is so valuable as part of the audit”.
guidance and legislation that places the engagement
agenda at the heart of improving quality within the NHS, Iain Thomas, MINAP representative, Member
with the aim of ensuring that patient involvement is of the SW London Cardiac and Stroke Network
incorporated into the way in which NHS organisations HQIP, Patient & Public Engagement (PPE), PPE
make decisions and improve services. Key Department of in Clinical Audit 2009
Health documents are:
 Patient and Public Involvement in the New NHS (1999) - This document places involvement at the
centre of NHS policy, stating that health organisations “need to work with patients, service users, their
carers and local communities to develop ways of improving health and making services and care
responsive to individuals’ and local communities’ needs”.
 Patient and Public Involvement Forums (PPIFs) (2003) – A PPIF was established as an independent
statutory body for each NHS trust and PCT in England. The aim of the PPIFs was to improve the quality of
NHS services by representing the views and experiences of patients, their carers and families, through
monitoring and reviewing NHS delivery, seeking the views of the public about those services and making
recommendations to the NHS accordingly.
 NHS Act, 2006 - Section 242 of the Act, states that all NHS organisations have a duty to involve by
ensuring that arrangements are in place so that the ‘persons to whom those services are being or may
be provided are, directly or through representatives, involved in and consulted on the planning of the
provision of those services, the development and consideration of proposals for changes in the way
those services are provided, and decisions to be made by that body affecting the operation of those
services’.

 Health and Social Care Act (2007) and Local Involvement Networks (LINks) - The Act reiterated that
patient and public engagement should be part of the “core business” of every NHS organisation and
replaced PPIFs with LINks. LINks are independently run by agencies or charitable groups with the aim of
giving local citizens a voice and the authority to monitor local services and hold providers to account.
 High Quality Care for All: NHS Next Stage Review (2008) - Lord Darzi’s report emphasised the need for
high quality care for patients and the public and the need to work in partnership. By stating that
‘measuring and valuing what matters most to patients, the public and staff is the way in which we will
enable the NHS to make progress towards high quality care’.
 World Class Commissioning (2009) - World class commissioning aims to use best practice from across
the world to help PCTs deliver better health and well-being for their population. Competency three
states that this should mean to; ‘proactively build continuous and meaningful engagement with the
public and patients to shape services and improve health’.
 The NHS Constitution (2009) - The NHS Constitution pledged that patients have the right to be involved
in discussions and decisions about their healthcare, and to be given information to enable them to
do this; and the right to be involved, directly or through representatives, in the planning of healthcare
services, the development and consideration of proposals for changes in the way those services are
provided, and in decisions to be made affecting the operation of those services.
“The patient’s voice is so important but do not forget the carer. There are about six million voluntary
carers, relatives and friends of all ages, in the UK. They have more experience than a doctor of a
patient’s pain, problems and care needs. They are the unrecognised figures in dealing with patients,
their views should be brought into the open through audit”.
Sirkka Thomas, Former Health Visitor and Cardiac Nurse, now carer for her husband
HQIP, Patient & Public Engagement (PPE), PPE in Clinical Audit 2009
THE BENEFITS OF ENGAGEMENT IN CLINICAL AUDIT
HQIP states that ‘Patients need to be involved and engaged in the selection of initiatives to improve quality;
in the governance of these initiatives, such as a clinical audit; in the collection of data for the audit; as part
of the dissemination of the products to ensure, as consumers, they are provided with re-assurance and
possible choice of healthcare provider, where such a choice is realistic or possible’.
HQIP, Patient & Public Engagement (PPE), PPE in Clinical Audit 2009
Clinical audit is about improving the quality of clinical services given to patients. At UHBristol we believe that
clinical audit should be focused on what matters most to our patients, service users and their carers. For
example, if outcomes are being audited, patients should be enabled to inform us about what outcome
measures actually matter to them.
THE BENEFITS
The benefits of engagement as specified below are based upon those listed by HQIP in Patient & Public
Engagement (PPE), PPE in Clinical Audit 2009.
 Those who have lived with, or cared for, someone with a particular illness are “experts” in the condition.
They have a subjective and valid view point based on actual experience about where quality could, and
should, be improved.
 Through participation and engagement patients and service users can enhance clinical audit by
achieving services better focused on their needs.
 A patient’s or service user’s direct experience of care gives them a different perspective about
ways to improve the quality of a service compared to a clinician. Patients might make different
choices about which aspects of care should be included as standards to be measured through the
clinical audit process.

 A patient can address the concerns they have raised by assisting with the development of the
action plan. The patient’s perspective will provide staff with a clearer picture about what changes
are needed and help to ensure that services are being provided in the way people want to see
them.
 Clinical audit results can provide patients and the public with more knowledge about the quality of care
in a particular healthcare setting; enabling them to make more informed choices about where they
obtain treatment. Public reporting should raise public confidence in the NHS as it boosts trust and
satisfaction. Involvement in clinical audit should also provide patients with a deeper insight into care
that is provided. However, it is important to remember that the communication of audit results should
be done in such a way to ensure it is accessible to the public and easy to understand, i.e. written simply
whereby the data and conclusions are easy to understand.
 Being involved in improving the quality of care through the clinical audit process can bring health
benefits to patients; feeling satisfaction after having influenced their care, being listened to, and from
the social interaction and engagement that this offers.
 As a direct result of listening to local people and developing clinical audits that are responsive to local
needs, patients and the public should feel a greater ownership of their local health services.
 Patient involvement will give clinical audit insight into the preferences of patients as to suitable
indicators of the quality of care provided. It will further assist in the identification of what is needed to
maintain and improve care in the future.
METHODS OF INVOLVEMENT - THE THEORY
In 1969, Sherry Arnstein developed her ‘ladder of

participation’ concept to provide a framework for
involvement. The ladder illustrates the different levels at
which a patient/ service user may wish to participate and
the level of involvement an organisation has reached.
The ladder is a useful visual tool and has been frequently

adapted. Models commonly show information, consultation
and partnership. In terms of clinical audit this might mean:
 Information – Informing patients of the results of a
clinical audit. This is the lowest level of engagement.
 Consultation – Sending out a patient survey. This is often mistaken as a high level of involvement but is
usually placed in the middle of the ladder and considered lower than engaging or partnering the patient.
 Partnership – Joint working between clinicians and patients, service users and careers to develop a
clinical audit project from the very beginning, participating throughout the process and with potential
for some clinical audit projects being patient-led. This is the highest level of engagement.
HQIP CRITERIA AND INDICATORS OF BEST PRACTICE IN CLINICAL AUDIT

The table below illustrates HQIP’s ‘Criteria and Indicators of Best Practice in Clinical Audit’. HQIP believe that
a key element in achieving good quality clinical audit is through patient representation and participation.
HQIP have adapted the NHS Centre for Involvement’s (NCI) set of six organisational standards and included
additional criteria in order to give more in depth guidance for specific engagement in clinical audit.
CRITERIA INDICATORS HOW

Patients or their The patient group to whom the clinical
representatives audit standards apply is clearly defined.
are involved in The clinical audit standards take full For example, the clinical audit
the clinical audit account of patient priorities and incorporates Patient Related Outcome
if appropriate. patient-defined outcomes. Measures (PROMS).

Patients/carers are recognised as key If appropriate and feasible, patient

stakeholders in the clinical audit representatives and relevant patient
process. organisations are involved in clinical audit
governance, treated as stakeholders, and
where appropriate, in all stages of the
clinical audit cycle as equal members of
the clinical audit team.
Patients who are members of the Not all patients and/or patient
clinical audit team are fully informed organisations will be members of the
about what is expected from them in clinical audit team but, as relevant
terms of participation, commitment stakeholders, should still be kept informed
and workload. and engaged.
If required, patients who are members
of the clinical audit team are given
basic clinical audit training to enable
them to contribute effectively to the
clinical audit process.
Patients are kept informed throughout All communications should use plain
the clinical audit process about English avoiding the use of jargon and
timescales, progress, results and acronyms.
actions.
METHODS OF INVOLVEMENT – IN PRACTICE
Patients, service users and carers should not be involved in a clinical audit project simply as a tick-box
exercise. At the outset of your project it is important to consider whether or not involvement will add value
to your project design.
Most commonly, involvement in clinical audit is achieved through a survey; this is likely to be a
questionnaire asking for details about what happened to the patient or service user. Surveys should only ask
for information that cannot be collected from another source and that is related to processes or the
outcomes of care, i.e. were standards of best practice being met. For instance, were they given the correct
patient information leaflet?
Asking patients about their satisfaction with our services and/or their ideas for improvement is a valid and
important aspect of patient involvement, but tends to sit outside the clinical audit cycle of measuring
against standards.
Surveys are not the only means by which patients, service users and carers can be involved in clinical audit
projects and programmes. Consider the following:
GIVING INFORMATION
Information can be made available locally to staff members, patients, service users and carers on planned
clinical audit projects, current projects and completed projects. A summary can be provided outlining each
clinical audit topic, the background, aims, objectives and standards; and in the case of completed projects a
summary of the results and any actions implemented can also be included. This can be achieved via
newsletters, leaflets or via posters located on the ward/ hospital notice boards.
If individuals have taken part in a survey, it is good practice to offer to send them a copy of the finalised
report.

GETTING INFORMATION
Information can be obtained from patients, service users, carers or members of the public via the following
methods:
1. An area of concern raised by way of a complaint or via the Patient, Advice & Liaison Service (PALS) can
be investigated using clinical audit.
2. A topic of concern highlighted via a focus group can be investigated using clinical audit.
3. Self-completed questionnaires – Questionnaires can be completed at the end of an episode of care.
4. Structured one-to-one interviews – Interviews can be face-to-face or via the telephone. The interviewer
asks a number of pre-determined questions and allocates the answers to one of a pre-determined set of
possible responses.
FORUMS FOR DEBATE

Links can be developed with existing groups in order to give patients, service users and/or carers an
opportunity to comment on your local clinical audit programme.
PARTNERSHIP
Joint working between clinicians and patients, service users and careers can ensure that a partnership
approach to clinical audit is instigated from the very beginning of a clinical audit process and continued
throughout. There should also be some scope for some clinical audit projects to be patient-led.
Partnership working can be achieved through the involvement of patient, service user and/or carer
representative(s) on your clinical audit committee or the clinical audit project team. This way, you could get
direct input into:
 Identifying and prioritising topics.
 Designing projects.
 Setting process standards and key outcome measures.
 Collecting data.
 Analysing and interpreting results.
 Disseminating findings.
 Agreeing and monitoring action plans.
EXCUSES FOR NOT INVOLVING PATIENTS

 One or two patients are not representative of all patients.
 Patients will not understand clinical audit.
 There are too many problems about confidentiality.
None of these problems are insurmountable. One or two patients on a committee may not be
representative of all patients, but is this a good enough reason not to do it? After all, are the clinicians on
your committee representative of all clinicians? Patients can be given basic training in audit. They can sign
confidentiality undertakings. The real issues are about motivation, time and resources.
CHECKLIST FOR INVOLVEMENT ON COMMITTEES / ON PROJECT TEAMS

 Try to find people who are used to committee/ project work. You could consult with patient
organisations to help identify appropriate representatives.
 Try to ensure that the patients you involve have links to wider networks, for wider consultation.
 Do not select a current or former patient of a clinician who sits on your committee. This is unfair to both
parties.
 Be clear about why you have involved patients and what you are hoping that they will contribute.
 Be clear about expectations.
 Provide training as appropriate.
 Ensure you can pay travel and carer costs.
 Schedule meetings at convenient times for patients.
 Think about physical access to meeting rooms.
 Avoid jargon.
QUESTIONNAIRE, INTERVIEW & SURVEY GROUP
All structured surveys, staff or patient, administered by post, in hospital, or via a one-to-one interview, are
subject to approval by the Questionnaire, Interview and Survey (QIS) Group. The QIS group offers advice on
survey design and is responsible for monitoring all survey activity at the Trust. Please contact Paul Lewis, the
Patient Involvement Facilitator for advice on structured surveys/ questionnaires. If you require advice on
unstructured interviews and focus groups, this should be discussed directly with Tony Watkin, the Trust’s
Public Involvement Project Lead. The contact details for QIS, Paul Lewis, and Tony Watkin are listed at the
end of this guide. Formal Research projects are subject to approval by a Research Ethics Committee (REC)
and therefore do not require QIS approval.
SUMMARY
In essence, the engagement of patients and public, both nationally and locally in clinical audit needs to
involve:
 Engagement in the strategic direction of clinical audit.
 Consultation in respect of standards to be audited.
 Active participation in collection and analysis of clinical audit data.
 Engaging the public in communication activity about clinical audit, partly as patients or potential
patients of treatment, and also through involvement in governance.
To achieve this, involvement needs to underpin the whole conception of clinical audit, from beginning to
end. It needs to be integral to the whole project.
FURTHER READING
"A guide to patient and public involvement in quality improvement", HQIP, 2016
http://www.hqip.org.uk/resources/patient-and-public-involvement-in-quality-improvement/
CLINICAL AUDIT
Office as above.

PATIENT EXPERIENCE
RESEARCH
SAMPLE SIZES
QUALITY IMPROVEMENT

How To: Apply Ethics to Clinical Audit
INTRODUCTION
The aim of this ‘How To’ guide is to provide advice on how to apply ethics to clinical audit. Unlike research,
clinical audit projects do not need to be submitted to a Research Ethics Committee (REC) for ethical
approval; this is one of the key reasons why you must ensure that your project is clinical audit rather than
research. If you think that there are ethical issues with your project you must discuss these with your
divisional Clinical Audit Facilitator at the planning stage. Aspects of this guide are discussed in more detail in:
 What is Clinical Audit?
 How To: Set an Audit Sample and Design your Data Collection & Data Collection Form.
ETHICS, INFORMATION GOVERNANCE AND CLINICAL AUDIT
The starting point when considering ethics in relation to clinical audit is to remember that your audit project
should benefit patients and not do harm.
Clinical audit must always be conducted within an ethical framework. At a practical level, this means
ensuring patient and staff confidentiality and ensuring that data is collected and stored appropriately. As
someone involved in clinical audit, you should be aware of the following pieces of legislation and national
guidance:
CALDICOTT PRINCIPLES (1997)

The Caldicott Committee was established by the Chief Medical Officer to review all patient identifiable
information which passes from NHS organisations to other NHS or non-NHS bodies for purposes other than
direct care, medical research, or where there is a statutory requirement for information.
The Committee considered that, whilst no single data item could be relied upon to identify an individual
with certainty, there were many items being transferred between organisations which, when taken
together, could permit identity to be inferred. They concluded that all items of information which related to
an attribute of an individual should be treated as potentially capable of identifying patients to a greater or
lesser extent, and should be appropriately protected to safeguard confidentiality.
The principles and recommendations made by the Committee emphasised the need for controls over the
availability of patient identifiable information and access to it. In particular a Caldicott Guardian, appointed
in each NHS organisation, with specific responsibilities to oversee an ongoing process of audit, improvement
and control.
The seven Caldicott principles that applied to the handling of patient identifiable information are:
 Justify the purpose(s) of using confidential information.
 Only use it when absolutely necessary.
 Use the minimum that is required.
 Access should be on a strict need-to-know basis.
 Everyone must understand his or her responsibilities.
 Understand and comply with the law.
 The duty to share information can be as important as the duty to protect patient confidentiality.
 2005 – 2019 UH Bristol Clinical Audit Team – Version 4.1 Page 1 of 7

GENERAL DATA PROTECTION REGULATION (2016)

The European General Data Protection Regulation (GDPR) came into force in 2018 and its specific
application to the Law of England and Wales is covered by the updated Data Protection Act 2018. This
legislation regulates the processing of information relating to individuals, including the obtaining, holding,
use or disclosure of such information. It relates to personal data; that is, data which relates to a living
individual who can be identified from that data, or when combined with other information possessed or
likely to come into possession.
Anyone processing personal information must comply with six enforceable principles of good information
handling practice. These say that data must be:
1. Fairly, lawfully and transparently processed;

2. Processed for specified, explicit and legitimate purposes;
3. Adequate, relevant and limited to what is necessary for the purpose for which they are being processed;
4. Accurate and (where necessary) up to date;
5. Not kept in a form which permits identification of data subjects for longer than is necessary;
6. Kept securely.
A seventh principle of accountability requires individuals and organisations to take responsibility for what
they do with personal information and how they comply with the other principles. You must have
appropriate measures and records in place to demonstrate compliance.
Health information (i.e. data about a person’s physical or mental health or condition) is classified as ‘special
category data’. The legislation allows for this to be lawfully processed where necessary for medical
purposes, when undertaken by a health professional or a person who owes an equivalent duty of
confidentiality. This includes the purposes of preventative medicine, medical diagnosis, medical research,
the provision of care and treatment and the management of healthcare services.
NHS CONFIDENTIALITY CODE OF PRACTICE (2003)

This contains practical guidance for NHS staff and others on how to treat patient information with respect in
the context of a modern health service. It is endorsed by the Information Commissioner, the General
Medical Council and the British Medical Association and is consistent with Data Protection requirements.
It aims to ensure that all patient information is processed fairly, lawfully and as transparently as possible so
that the public:
 Understand the reasons for processing personal information.
 Give their consent for the disclosure and use of their personal information.
 Gain trust in the way the NHS handles information.
 Understand their rights to access information held about them.
It states that there are situations where consent cannot be obtained for the use or disclosure of patient
identifiable information, yet the public good of this use outweighs issues of privacy. Section 60 of the Health
and Social Care Act 2001 currently provides an interim power to ensure that patient identifiable
information, needed to support a range of important work such as clinical audit, record validation and
research, can be used without the consent of patients. It also states that where reasonable efforts are made
to ensure that patients understand how their information is to be used to support their healthcare, consent
can be implied, providing that “need to know” principles are enforced.
The General Medical Council has also published its own Confidentiality Code of Practice (see References and
Further Reading below).

HQIP GUIDE TO MANAGING ETHICAL ISSUES (2017)

The Healthcare Quality Improvement Partnership have published guidance on managing ethical issues in
quality improvement or clinical audit projects, which recommends that projects should meet the following
criteria:
 “Favourable benefit/risk balance” — The project should maximise benefits to patients and patient care
and limit any risks for patients, such as breaches in confidentiality or privacy;
 “Scientifically valid” — The project must be well-designed with sound methodology, and evidence
should be provided that any changes in practice, processes or systems implemented as a result of the
project show benefits for patients and patient care;
 “Equitable and reflecting priorities” — Within an organisation, audit activity should cover all clinical
services, patient conditions and professional groups, and a system should exist to set priority topics;
 “Value” — The anticipated improvement should justify the effort in the use of time and resources
 “Awareness of conflict of obligation to patients” — Efforts to reduce cost of care through improvement
projects should be carefully considered so as not to compromise quality of care.
WHAT ARE THE IMPLICATIONS FOR CLINICAL AUDIT?
Essentially, the documents outlined above allow for the use of patient data for the purposes of clinical audit,
that is, to review and improve healthcare. The legal basis for Clinical Audit is Article 6(1)(e) of the GDPR: the
processing is necessary for you to perform a task in the public interest or for your official functions, and the
task or function has a clear basis in law. The legal basis for processing special category data is 9(2)(h):
processing is necessary for the purposes of preventive or occupational medicine, for the assessment of the
working capacity of the employee, medical diagnosis, the provision of health or social care or treatment or
the management of health or social care systems and services.
Audit projects will usually involve clinical members of staff who already provide care for the group of
patients concerned. All Trust staff are automatically bound by common law and contractual duty of
confidentiality. However, data should not be reviewed by non-Trust staff unless a similar duty of
confidentiality is in place, for example in the case of medical students.
Note that UHBristol has published detailed information on its public facing website about how the Trust
collects, uses and shares information, including its use for clinical audit. This can be found at:
http://www.uhbristol.nhs.uk/privacy/.
Specific implications for the audit process are as follows:

 Data Collection - In order to comply with data protection rules and to ensure patient anonymity,
personal details, including hospital ID number, name, address, date of birth etc, should not be recorded
on the data collection form. The best approach is to number each of your forms using a unique
identifier. A separate piece of paper, or ‘code sheet', should then be kept as a key, linking each unique
identifier to the patient’s hospital number. Without this list, data collection forms cannot be linked to
specific patients. The key would need to be destroyed when the audit analysis is complete. This also
applies to data held on databases. Occasionally it might be necessary to record patient identifiable
information on the data collection form. For example, during a prospective data collection exercise the
data collection form might have to follow a patient through their pathway of care. In this instance the
only patient identifiable information that should be recorded on the form is their hospital ID number;
Name, date of birth, etc. should never be recorded. The hospital number should be removed and each
patient coded at the earliest opportunity, i.e. once all data has been collected.
 Adequate, relevant and limited to what is necessary. Ensure data is being collected for the stated
purpose of clinical audit (rather than research) and collect only the data you need to answer your audit
objectives and standards. HQIP’s guide on information governance (2017) recommends clearly defining

the purpose of the project, so that it is clear why you are holding and using the data, and precisely
defining the target population, as it would be excessive to collect data on patients outside that.
 Data should be kept securely - This relates to both electronic and hard copies of data. Any patient-
identifiable clinical audit data must be kept securely. Audit proformas should be locked in a filing
cabinet, electronic files should be password-protected or kept on a secure, restricted access server and
only clinical professionals and audit team with a duty of confidentiality should be able to access the
data. Person-identifiable information should not be stored on personal devices not authorised under the
Trust’s Bring Your Own Device (BYOD) Policy. Identifiable data should only be emailed within the Trust
network or from one NHS.net address to another (more details on secure email can be found in a Trust
guideline, the link for which is provided in the references section at the end of this guide). Situations
should be avoided that might allow public access to personal data, such as leaving patient files in the
boot of your car. Your car could be stolen with the files in it!
 Data should not be kept for any longer than necessary - Records relating to Clinical Audit must be
retained for five years following their creation. After this time, they should be reviewed and securely
disposed of if they are no longer needed.
CLINICAL STAFF CONFIDENTIALITY
It is also important to remember that clinical audit reports and presentations should not identify clinicians
by name unless the people concerned have agreed to this.
DO YOU NEED ETHICS APPROVAL FOR AUDIT?
Whilst research needs to be submitted to REC for ethical approval, clinical audit does not. However, whilst
clinical audit by definition does not involve anything being done to patients beyond their routine clinical
management and therefore does not require formal ethical approval, it should still be conducted within an
ethical framework. By approving and registering a project as a clinical audit, the Trust is stating that the
project fulfils the methodological criteria that allows for patient data to be accessed and analysed.
However, before assuming that you do not require ethical approval, it is important to consider:
1. IS YOUR PROJECT REALLY A CLINICAL AUDIT PROJECT?

Decisions about whether projects need ethical approval often hinge on the question of whether they really
are clinical audit, or whether they are actually research. Remember that clinical audit asks the questions
“are we following best practice?” and “what is happening to patients as a result?”
Clinical audit projects never involve:

 A completely new treatment or practice;
 The use of control groups or placebo treatments;
 Any disturbance to the patient beyond that required for routine clinical management;
 Allocating patients randomly to different treatment groups.
Clinical audit may, however, involve input from patients at a number of levels, e.g.
 Patients may be asked to participate in surveys which help to determine whether standards have been
met;
 Patients may be involved in the design of individual audit projects or indeed whole programmes of
activity, e.g. as members of steering groups.
Sometimes healthcare professionals undertake what they mistakenly think is ‘clinical audit’, when what they
are really doing is research. Research always requires ethical approval; calling research by any other name
does not remove this requirement.

If you are unsure whether or not the project that you want to undertake is clinical audit or research, your
divisional Clinical Audit Facilitator will be able to advise. Very occasionally liaison might be required between
your divisional Clinical Audit Facilitator, the Research & Innovation Department and the local REC. If the
conclusion is that you are undertaking research, you must formally submit the appropriate paperwork to the
local REC for ethics approval.
If on the other hand your project contains an element of both clinical audit and research, you will need to
obtain formal ethical approval for the research component of the project from REC and submit a completed
copy of the clinical audit proposal form to your divisional Clinical Audit Facilitator for the clinical audit
component.
2. DOES YOUR PROJECT INCLUDE A PATIENT SURVEY?

Patient surveys can be construed as doing something to patients ‘beyond normal clinical management’. It is
therefore important to take advice on the design of patient surveys. Planned questions could touch upon
potentially sensitive matters, giving rise to ethical concerns. Any patient surveys should be designed in such
a manner as to cause minimum possible disruption to patients.
All structured surveys, staff or patient, administered by post, in hospital, or via a one-to-one interview,
undertaken for clinical audit are subject to approval by the Questionnaire, Interview and Survey (QIS) Group.
The group offers advice on survey design and is responsible for monitoring all survey activity at the Trust.
Please contact Paul Lewis, Patient Experience and Involvement Team Manager for advice on structured
surveys/ questionnaires. If you require advice on unstructured interviews and focus groups, this should be
discussed with Tony Watkin, the Trust’s Public Involvement Project Lead. The contact details for QIS, Paul
Lewis, and Tony Watkin are listed at the end of this guide.
Formal Research projects involving questionnaires are subject to ethical approval by the local REC and
therefore do not require QIS approval.
When undertaking a survey it is important to consider the patient's, service user’s or carer’s rights, dignity
and time. Should you wish to undertake a survey as part of your clinical audit project there are a number of
ethical questions that you should consider. These are:
 Is the information that you are seeking already available? The QIS group may not approve projects if it
feels that one group of patients is being excessively targeted.
 Is it necessary to carry out a survey? Surveys should only ask for information that cannot be collected
from another source and that is related to processes or the outcome of care, i.e. were standards of best
practice being met.
 Will it add value to the clinical audit project? Carrying out a survey unnecessarily, or asking more
questions than necessary, will not add value to your project. Keep your questionnaire succinct. It is
important not to bombard patients with too many questions as they might choose not to participate if
the questionnaire looks too long; up to 20 questions is usually sufficient.
 Have you checked your sample? Your sample, if postal survey, should be checked against the latest
hospital records, as sending questionnaires to deceased patients is a frequent error that can cause
distress to relatives. Similarly, getting patients’ names or other details slightly wrong can also cause
offence.
 What written information will be given to the subject to explain what the survey is about? Include a
covering letter with all patient questionnaires. A template covering letter is available on the QIS intranet
site. The intranet details are listed at the end of this guide. The letter should include contact details
should the patient have any queries that they wish to be answered quickly about the questionnaire, the
background to project and reason for contacting the patients, instructions on how to return the
completed questionnaire, a statement on how you will protect the patient’s confidentiality, ideally this
should be achieved through developing completely anonymised questionnaires, and what you intend to
do with the data collected.

 How will you ensure that patients freely consent to taking part? Patients must not be coerced into
taking part and must have the option to decline or withdraw at any time without detriment or
antagonism.
 Is your survey addressing a sensitive topic? Where patients are being asked questions about their clinical
care, it is important that questions are not phrased inappropriately/insensitively, as this might
inadvertently cause harm or distress.
 Will the survey interfere with the treatment of the patient? Your survey should not cause the patient to
reflect negatively upon their course of treatment, thereby jeopardising clinical outcomes.
If you are in any doubt about whether your survey raises ethical concerns please discuss with your divisional
Clinical Audit Facilitator.
3. ARE YOU PLANNING TO PUBLISH?

Clinical Audits are usually published because the topic and/or methodology may be of interest to a wider
audience, for instance, demonstrating how an audit cycle was successfully followed after initially poor
results against standards, by implementing changes and demonstrating an improvement in practice with a
re-audit.
Whilst clinical audit projects may be published without ethical approval, e.g. the Quality Improvement
Reports published by the British Medical Journal, journal editors may refuse to publish articles if there are
ethical concerns and REC ethical approval has not been granted. If you want to publish because of the
results of your project, rather than to publish the methodology for use by others, you should question
whether you are undertaking a research activity, rather than a clinical audit project, that should have been
submitted to REC for ethical approval. If you intend to publish your clinical audit project this should be
discussed with your Clinical Audit Facilitator at the beginning of your project.
REFERENCES AND FURTHER READING
 General Data Protection Regulation (2016)

https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32016R0679
 Data Protection Act (2018)
http://www.legislation.gov.uk/ukpga/2018/12/contents/enacted
 Caldicott Committee Report (1997)
http://webarchive.nationalarchives.gov.uk/20130124064947/http://www.dh.gov.uk/prod_consum_dh/
groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_4068404.pdf
 NHS Confidentiality Code of practice (2003)
https://www.gov.uk/government/publications/confidentiality-nhs-code-of-practice
Note that figure 5 (p19) contains practical advice on keeping information secure, and figure 6 (p20)
nicely summarises the relevant Caldicott principles.
 GMC Confidentiality Code of Practice (2017)
https://www.gmc-uk.org/ethical-guidance/ethical-guidance-for-doctors/confidentiality
 Health & Social Care Act (2001), Section 60
http://www.legislation.gov.uk/ukpga/2001/15/contents
 “What we do with your personal information”, UHBristol Patient Information leaflet (2018)
http://nww.avon.nhs.uk/dms/download.aspx?did=19101
 “Guidance on sending confidential information by email”, UHBristol guideline (2016)
http://nww.avon.nhs.uk/dms/download.aspx?did=17132
 “Managing ethical issues in quality improvement or clinical audit projects”, HQIP guide (2017)
https://www.hqip.org.uk/resource/guide-to-managing-ethical-issues-in-quality-improvement-or-
clinical-audit-projects/
 “Information governance in local quality improvement”, HQIP guide (2017)
http://www.hqip.org.uk/resources/information-governance-in-local-quality-improvement

CLINICAL AUDIT
Office as above.
PATIENT EXPERIENCE
Experience and Involvement Team Manager (Surveys & Evaluations), 0117 342 3638 or e-mail:
paul.lewis@UHBristol.nhs.uk
RESEARCH
SAMPLE SIZES
QUALITY IMPROVEMENT

Clinical Audit - UH Bristol Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Clinical Audit - UH Bristol Guide

Uploaded by

Copyright:

Available Formats

1_what_is_clinical_audit_v4 2

WHAT IS CLINICAL AUDIT

The key messages being that:

The main stages of the clinical audit process are:

THE HISTORY OF CLINICAL AUDIT

WHAT CLINICAL AUDIT IS NOT

Other forms of audit can include:

CLINICAL AUDIT & RESEARCH: WHAT IS THE DIFFERENCE?

CLINICAL AUDIT & RESEARCH: WHY IT IS IMPORTANT TO KNOW THE DIFFERENCE

PATIENT, SERVICE USER, CARER ENGAGEMENT

UNDERTAKING A CLINICAL AUDIT PROJECT AT UHBRISTOL

CLINICAL AUDIT STRATEGY

CONTACT DETAILS / USEFUL INFORMATION

WHAT IS CLINICAL AUDIT

The main stages of the clinical audit process are:

UNDERTAKING A CLINICAL AUDIT PROJECT AT UHBRISTOL

 2005 - 2017 UHBristol Clinical Audit Team – Version 4 Page 1 of 6

1. CHOOSE A CLINICAL AUDIT TOPIC

Possible sources for your clinical audit project include the:

2. FORM A PROJECT TEAM

3. SET THE AIM, OBJECTIVES AND STANDARDS

4. ETHICS & ENGAGEMENT

5. SELECT AN AUDIT SAMPLE

6. PLAN AND CARRY OUT DATA COLLECTION

7. ANALYSE THE DATA Number of patients

8. PRESENT THE FINDINGS

9. IMPLEMENT CHANGES AND RE-AUDIT

A FLOW CHART SUMMARISING THE CLINICAL AUDIT PROCESS AT UHBRISTOL

Discuss and agree project with divisional Clinical Audit Facilitator

Complete the UHBristol clinical audit proposal paperwork

Submit proposal to relevant Clinical Audit Convenor/Facilitator for approval

Carry out data collection

Analyse the data

Share your findings: 1. Presentation 2. Report and action plan

Implement changes and re-audit

CONTACT DETAILS / USEFUL INFORMATION

THE NATIONAL AGENDA

OTHER SOURCES OF CLINICAL AUDIT PROJECTS

ASPECTS OF CLINICAL GOVERNANCE

STRUCTURE, PROCESS & OUTCOME

What you expect Measurable change in health status

What you do Investigations, treatments, procedures

DISADVANTAGES OF AUDITING STRUCTURE & OUTCOME

EXAMINING PATIENT PATHWAYS

TABLE 1: Topic Identification

Source of Audit Audit Topic

Issue of local concern

Score (number of Yes

CONTACT DETAILS/ USEFUL INFORMATION

More technical forms of this question are outlined below.

 2005 - 2017 UHBristol Clinical Audit Team – Version 4 Page 1 of 6

EXAMPLE 1 How to Write: Clinical audit topic, aim and objectives

Topic: Leg ulcers

Aim: To improve the care received by patients with leg ulcers

Clinical audit is by definition standards-based (sometimes referred to as ‘criterion-based’). Standards are

Standard Standard Standard

 2005 - 2017 UHBristol Clinical Audit Team – Version 4 Page 2 of 6

DEVELOPING AUDIT STANDARDS

Results of single RCTs

Results of well-conducted non-RCT clinical studies

Expert committee reports; clinical experience of respected authorities

Personal experience and opinion

WRITING AUDIT STANDARDS

Standards should be SMART.