You are on page 1of 13

International Dental Journal 2019; 69: 252–264

CONCISE REVIEW
doi: 10.1111/idj.12453

A systematic review of quality measures used in primary


care dentistry
Matthew J. Byrne1 , Martin Tickle1, Anne-Marie Glenny1, Stephen Campbell2,
Tom Goodwin1 and Lucy O’Malley1
1
Division of Dentistry, School of Medical Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK;
2
NIHR Patient Safety Translational Research Centre, Division of Population Health, HSR & Primary Care, School of Health Sciences, Faculty
of Biology, Medicine and Health, University of Manchester, Manchester, UK.

Introduction: ‘Quality’ in primary care dentistry is poorly defined. There are significant international efforts focussed on
developing quality measures within dentistry. The aim of this research was to identify measures used to assess quality in
primary care dentistry and categorise them according to which dimensions of quality they attempt to measure. Methods:
Quality measures were identified from the peer-reviewed and grey literature. Peer-reviewed papers describing the devel-
opment and validation of measures were identified using a structured literature search. Measures from the grey literature
were identified using structured searches and direct contact with dental providers and institutions. Quality measures were
categorised according to domains of structure, process and outcome and by disaggregated dimensions of quality. Results:
From 22 studies, 11 validated measure sets (comprising nine patient satisfaction surveys and two practice assessment
instruments) were identified from the peer-reviewed literature. From the grey literature, 24 measure sets, comprising 357
individual measures, were identified. Of these, 96 addressed structure, 174 addressed process and 87 addressed outcome.
Only three of these 24 measure sets demonstrated evidence of validity testing. The identified measures failed to address
dimensions of quality, such as efficiency and equity. Conclusions: There has been a proliferation in the development of
dental quality measures in recent years. However, this development has not been guided by a clear understanding of the
meaning of quality. Few existing measures have undergone rigorous validity or reliability testing. A consensus is needed
to establish a definition of quality in dentistry. Identification of the important dimension of quality in dentistry will allow
for the production of a core quality measurement set.

Key words: Quality, measurement, improvement, indicators

process involving dental professionals, patients and


INTRODUCTION
other stakeholders to develop and maintain goals and
Improving quality in primary dental care is a goal of measures to achieve optimal health outcomes’2. This
international interest1,2. To improve the quality of statement highlights the importance of measurement
health care, a definition of quality and the criteria of quality in the process of improving patient care.
used to assess quality must be established3. Most defi- However, the constituent parts of this and other defi-
nitions of quality in primary care relate to medicine. nitions need to be considered further to ensure that all
Within medicine, several different definitions have of the key dimensions of quality are captured. Fur-
been described4–13. Dentistry has a number of signifi- thermore, the view that dentistry is so different from
cant differences from medicine, which has led to sug- other areas of health care that it merits its own defini-
gestion of the need for a specific definition of quality tion must be tested.
in dentistry to be developed3. Whilst some specific Many quality assurance and improvement schemes
definitions for quality in dentistry have been have been attempted but are hampered by a weak evi-
offered14,15, there is no agreed definition or concep- dence base17. The Institute of Medicine (IoM) in the
tual framework available describing what quality USA has stated that quality measures in dentistry ‘lag
means for primary dental care1,3,16. The World Dental far behind’ those in medicine and other health profes-
Federation (FDI) has defined quality as ‘an iterative sions18. The IoM suggests that construction of quality
252 © 2018 FDI World Dental Federation
Systematic review of quality measures

measures would help to improve oral health and Inclusion criteria


reduce inequalities18. A first step in this process would
be to develop a comprehensive understanding of what Types of study
type of measures have been developed and what
Any cross-sectional or longitudinal study concerning
dimensions of quality they are attempting to measure.
the development or validation of a quality measure in
Quality is complex and multi-dimensional; therefore,
primary care dentistry or that uses a previously
a structured approach to terminology usage is neces-
reported measure.
sary to avoid confusion. We propose the following
terminology for categorising quality measures:
Types of measure
Domains The types of measure included were those that may
The Donabedian system-based model of quality states be used by a dentist, patient or other stakeholder to
that quality measures may address one or more of assess quality in a primary care setting.
three domains, namely: the structures that contribute Studies were only selected if they presented the pro-
to the delivery of care; the processes of care; and the cess by which the measure was developed and vali-
outcomes of care6. Measures of structure include dated in sufficient detail. Measures that had their
assessments of the provision of facilities, staff and validity confirmed in later studies were also included.
training. Process measures assess what the practitioner
actually does, and outcomes measurements assess the
Exclusion criteria
impact of an intervention6. Evidence is required to
show that the measurement and improvement of pro- Non-peer-reviewed studies or opinion pieces, studies
cesses will lead to an improvement in outcomes19, for published in a language other than English, studies
example, if placement of fissure sealants (process) published pre-1970 and studies that did not describe
leads to prevention of caries (outcome). development or validity testing of the measure were
excluded.
Measures of oral health-related quality of life
Dimensions
(OHRQoL), being primarily epidemiological tools,
Quality may be disaggregated into different dimen- were excluded.
sions. These dimensions each give a partial view of A structured systematic literature search was pro-
quality7. The IoM definition of quality identifies duced for use in MEDLINE (Appendix 1) and
dimensions of safety, effectiveness, timeliness, patient- adjusted for other databases. The Databases used
centredness, efficiency and equity13. There is no con- were MEDLINE via OVID (1946 to present), Psych-
sensus on the dimensions of quality that are most per- info via OVID (1806 to present), EMBASE via OVID
tinent to dentistry. (1980 to present), Health and Psychosocial Instru-
An ideal measure set in dentistry would address ments via OVID (1985 to present) and Social Policy
each of the dimensions of quality across the domains and Practice via OVID.
of structure, process and outcome. The screening process was managed using an End-
The aim of this research was to identify measures note Library; references were initially screened accord-
used to assess quality in primary care dentistry. ing to title and abstract by MB, AG, MT, SC and LO,
Measures were categorised according to the IoM’s and irrelevant papers were removed. The remaining
dimensions of quality and the Donabedian domains of papers were double screened by MB and AG, with
the IoM. The identification of these measures will disagreements discussed with MT. Full-text papers
assist in the production of a framework to support were obtained for the remaining included studies. Ref-
further development of quality measures in dentistry. erence lists of these studies were hand searched to
obtain key sources that described the development or
validation of the measures. Key journals were directly
METHODS
searched to identify articles that may have been miss-
ing from the literature search strategy.
Peer-reviewed literature
A systematic search strategy was used to identify mea-
Grey literature
sures from the published literature (Appendix 1). Each
identified measure was assessed for internal and exter- Searching of the grey literature was completed using
nal validity. The constituent parts of each measure the OpenGrey database20 and through hand searching
were assessed for the Donabedian domain and IoM of the websites of large dental providers, dental asso-
disaggregated dimension of quality that they measure. ciations, insurers and government bodies in English-
© 2018 FDI World Dental Federation 253
Byrne et al.

speaking nations (the USA, Canada, the UK, Australia Number of records Number of additional records
and New Zealand). Dental insurers and large corpo- identified through identified though other
database searching: sources:
rate dental providers were contacted directly and n = 543 n=7
asked to state any quality measures they use. The
Number of records after duplicates removed
National Quality Forum’s ‘Environmental Scan, Gap n = 501
Analysis & Measure Topics Prioritization’ was con-
sulted, as this project had similar goals of measure
identification as the present study17. A number of the Number of records screened Number of records excluded:
n = 501 n = 285
measures identified within this study are no longer in
use and were thus omitted. As measures from the grey
Number of full-text papers Number of full-text papers
literature are actively measuring and affecting clinical assessed for eligibility: excluded
practice, exclusion criteria were limited to non-English n = 216 n = 194
references and publication before 1970.

Number of Records included in qualitative synthesis


Evaluation and categorisation of measures n = 22

As there is no established measure to assess the qual- Figure 1. PRISMA flow diagram of literature screening and identifica-
ity of quality measures within primary care dentistry, tion process.
the measures were individually assessed to determine
their internal and external validity. Measures that did Table 1 summarises the contents of each measure
not demonstrate face validity were excluded. Evalua- and describes the internal validity and reliability test-
tion of the internal consistency of measures, such as ing of the included measures. Further studies that
Cronbach’s alpha scores, was extracted. An a < 0.5 make use of the measure are described. Satisfaction
suggests unacceptable internal consistency of the mea- was assessed in the nine patient satisfaction scales
sure21. Measures of test–retest reliability, such as intr- using Likert-style scores. These measures were to be
aclass correlation coefficients and Spearman’s rank completed by the patient receiving treatment or by
correlation coefficients, were also extracted. Each their parent/guardian. Similar ordinal rating scales
measure was categorised according to the Donabedian were used by the measures described in the two prac-
domains of Structure, Process and Outcome and then tice assessment tools. These measures were designed
assessed against the six disaggregated dimensions of to be completed by a dentist or manager in the dental
quality of the IoM definition of quality. practice.
All the included measure sets showed face validity.
Method of analysis and synthesis Cronbach’s alpha was used to describe internal valid-
ity in all of the measures. All were above the accept-
Structured tables were used to describe the data from able level of a = 0.721, except the Dental
each measure narratively to give an overview of the Management Survey Brazil (DMS-BR)22, with
domains of quality evaluated using each measure and a = 0.632 (suggesting questionable internal consis-
the dimensions of quality they address. tency), and components of the Dental Satisfaction
Questionnaire (DSQ)38. The validity of the DSQ is
RESULTS demonstrated by Chapko44; the measurement concept
of ‘Cost’ was suggested as unreliable (a = 0.47). All
Measures from the peer-reviewed literature other measures in this set had a > 0.6. Test–retest
reliability was reported in five of the measures, using
A flow diagram of the screening process is presented either intraclass correlation coefficients or Spearman
in Figure 1. A total of 543 papers were generated rank correlation coefficients, all of which showed
from the structured literature search. Seven additional acceptable values.
papers were identified following further searches. The patient satisfaction scales broadly considered
Removal of 49 duplicates gave 501 papers for initial patient satisfaction with the care received. For these
analysis. Of these, 285 were excluded as they were measures, satisfaction can be considered as an out-
irrelevant. The full text of the remaining 216 papers come of care. Where specific questions address the
was assessed. Following this, 22 papers met the inclu- procedural aspects of care, the domain of process is
sion criteria. Within these papers, 11 individual mea- also measured. The Burdens in Prosthetic Dentistry
sure sets were identified: nine were patient satisfaction Questionnaire (BiPD-Q) 27 and Burdens in Oral Sur-
scales and two were practice assessment instruments gery Questionnaire (BiOS-Q)29 measures were specifi-
for use by a dentist or practice manager. cally related to satisfaction with processes of care and

254 © 2018 FDI World Dental Federation


Table 1 Contents of each measure set identified in the peer-reviewed literature outcomes of internal validity and reliability testing of the measure sets
identified in the peer-reviewed literature
Measure set name Key Further Description of measure contents Items Type of Internal Test–retest Domain IoM
and abbreviation reference references using (n) measure consistency reliability dimensions of
measure (Cronbach’s quality
alpha a) assessed

Dental Gonzales22 N/A Self-assessment tool for use by dentists 6 Practice a = 0.632 Intraclass Structure Safety,
Management and practice managers to assess the assessment correlation efficiency
Survey Brazil quality of safety and organisational tool

© 2018 FDI World Dental Federation


coefficients = 0.93
(Dimension 6) aspects of dental care delivery and 0.94
(DMS-BR)
Survey of Goetz23 N/A Self-assessment tool of structural 20 Practice a = 0.775 Intraclass Structure Safety,
Organisational elements of the delivery of dental care, assessment correlation patient-
Aspects of Dental with focus on teamwork, leadership tool coefficients = 0.732 centredness
Care (SOADC) and the implementation of change
within a practice
Dental patient Cheng Wong25 Patient satisfaction scale on the quality 16 Patient a = 0.94 Intraclass Outcome Patient-
feedback on et al.24 of information provided by the dentist satisfaction correlation centredness
consultation skills to patients in consultations and the survey coefficients = 0.89
(DPFCS) atmosphere of trust generated
Tool Developed Bahadori26 N/A Patient satisfaction scale of the 30 Patient a = 0.71–0.91 Not reported Structure, Effectiveness,
from Parasurman structures and processes of primary satisfaction process patient-
and Zalathml dental care. Focus on the settings in survey centredness,
Construct of which dental care is delivered and timeliness,
quality communication between dentists and efficiency
patients
Burdens in Reissman27 Hacker28 Patient satisfaction scale of the 25 Patient a = 0.87 Not reported Process Patient-
Prosthetic perceived burdens of the processes of satisfaction centredness
Dentistry dental treatment during prosthetic survey
Questionnaire dental procedures
(BiPD-Q)
Burdens in Oral Reissman29 N/A Patient satisfaction scale of the 16 Patient a = 0.84 Intraclass Process Patient-
Surgery perceived burdens of the processes of satisfaction correlation centredness
Questionnaire dental treatment during oral surgical survey coefficients = 0.90
(BiOS-Q) procedures
Tool after Kikwilu30 N/A Patient satisfaction scale of the 11 Patient a = 0.849 Spearman rank Outcome Effectiveness,
‘Consensus perceived quality of the setting of satisfaction correlation timeliness
workshop for delivery of dental care and perceptions survey coefficients = 0.751
selecting essential of treatment quality and –0.923
oral health communication
indicators in
Europe’
Tool developed Keller31 N/A Patient satisfaction scale of the 23 Patient a = 0.74 Not reported Outcome Timeliness,
from Consumer perceived quality of information, satisfaction effectiveness
assessment of communication and dental care survey
Healthcare received by dental plan holders
Providers and
systems

(continued)
Systematic review of quality measures

255
Byrne et al.

dimensions of the procedural elements of prosthetic dentistry and

effectiveness
Effectiveness,

Effectiveness,

centredness,
centredness

centredness
assessed

Timeliness,
quality oral surgery. Structure was measured in the practice
IoM

patient-

patient-

patient-
self-assessment tools [DMS-BR22 and Survey of
Organisational Aspects of Dental Care (SOADC)23]
and concerned the training of staff and provision of

Outcome
Outcome
safety equipment and data management. The ques-
Domain

Outcome

Structure
tionnaire developed by Bahadori et al.26 asked
patients to rate the importance of a number of struc-
tures and processes within a dental clinic, including
the state of facilities and the communication skills
exhibited by the dentist.
Test–retest
reliability

The practice assessment measures (DMS-BR22,


Not reported

Not reported

Not reported

SOADC23) assessed the dimensions of (i) safety, with


measures addressing the use and provision of personal
protective equipment and (ii) efficacy, using measures
of ability of practice members to work as a team. The
Chapko) = 0.46

patient satisfaction surveys covered a range of dimen-


(Cronbach’s
consistency

a = 0.83 and
alpha a)
Internal

sions, for example: safety – satisfaction with cleanli-


a = (from

ness of facilities30; effectiveness – satisfaction with


a = 0.92

–0.78
0.84

treatment received33; patient-centredness – perception


of dentist caring about patient32; timeliness – satisfac-
tion with waiting times to see a dentist31; efficiency –
satisfaction

satisfaction

satisfaction
measure

patient satisfaction of cost38; and equity – patient per-


Type of

survey

survey

survey

ception of dentist acceptance of them as a person33.


Patient

Patient

Patient

Measures from the grey literature


Items
(n)

10

10

10

In total, 24 collections of quality measures sets were


identified, with a total of 357 individual quality mea-
communication, information given and

communication of oral health, rapport

Patient satisfaction scale, assessing ease


Patient satisfaction scale regarding the

sures contained therein. Table 2 describes the mea-


Description of measure contents

surement sets qualitatively in terms of what attributes


with dentist and comfort during

of access, communication and

they attempt to measure, evidence of validity testing


environment of care deliver

and categorisation of measures according to domain


Patient Satisfaction scale,

and IoM dimensions. The majority of measures within


thoroughness of care

these sets (n = 196/357) followed a numerator/denom-


inator format, wherein the patients receiving a treat-
ment process or reporting an outcome were classed as
treatment

the numerator and a target population was identified


as the denominator. These types of quality measure
are presented as a percentage. A further 36 measures
were patient satisfaction measures using Likert-style
Mascarenhas43
references using

ordinal rating scores. The Denplan Excel Quality


Stouthard37
Hakeberg36
measure

measures68 contain 122 checklist-style ‘yes/no’


Milgrom40
Further

Brennan42
Olausson34

Chapko44
Skaret41

responses. The only measures that described validity


Sun35

Lee39

or test–retest reliability were those developed by the


Dental Quality Alliance49–51. These measures have
been developed according to the National Quality
Davies and
Corah and
Larrsson32

O’Shea33
reference

Forum measure development guidance69.


Ware38
Key

Approximately half (48.7%, 174/357) of measures


Table 1 continued

assessed processes of care provision. Common themes


for process measures were the provision of fluoride,
Dental Satisfaction
Satisfaction Scale
Measure set name

Quality from the


and abbreviation

fissure sealants and annual reviews. There was a high


Questionnaire

Questionnaire

degree of repetition of these concepts across a number


Perspective

Dental Visit
Patient’s

of measure sets. A total of 24.4% (87/357) measured


(DVSS)

(DSQ)

outcomes, with patient satisfaction ratings making up


52.9% (46/87) of these outcome measures. The
256 © 2018 FDI World Dental Federation
Systematic review of quality measures

Table 2 Qualitative description of measure sets identified from the grey literature summarising evidence of valida-
tion, and categorisation of measure sets according to domains and dimensions of quality
Measurement collection Items Description of measure Domains Dimensions Validation
(n)

Dental Quality and Outcomes 15 Measures of patient satisfaction of Process, Safety Not reported
Framework45 their dental state and dental Outcomes Effectiveness
practice, clinical effectiveness and Patient-
patient safety centredness
Timeliness
NICE oral health in care 9 Measure of the oral health needs Structure, Patient- Not reported
home46 of nursing and care home Process, centredness
residents and the provision of Outcome Effectiveness
care Equity
NICE oral health promotion in 15 Measures of access to health- Structure, Patient- Not reported
the community47 promotion resources within the Process, centredness
community Outcome Effectiveness
Equity
Dental Assurance Framework 12 Claim data-based assessment of Process, Effectiveness Not reported
Policy48 provision of fluoride varnish, Outcomes Patient-
sealants and radiographs, rate of Centredness
extractions, endodontics, patient Timeliness
reattendance and patient
satisfaction
Dental Quality Alliance Adult 3 Process indicators of the evaluation Process Effectiveness Face validity gained through
Measures49 and ongoing care of patients with consensus of members. Data
periodontitis and provision of element and convergent validity
topical fluoride in patients with testing undertaken
elevated risk
Dental Quality Alliance 12 Measures of the utilisation of Process, Effectiveness RAND-UCLA method used to
Paediatric measures50 services; provision of sealants, Outcomes gain consensus of face validity
fluoride, prevention, treatment; of measure concept. Data
continuity of care; emergency element collection validity
department visits and follow-up assessed using Kappa statistics
cost
Dental Quality alliance 2 Measures of utilisation of Process Effectiveness Face validity gained through
Electronic Paediatric preventive and treatment services consensus of members. Data
measures51 element and convergent validity
testing undertaken
Child and adolescent Health 3 US national survey. Measures of Process, Effectiveness Not reported
Measurement Initiative utilisation of treatment services, Outcomes
National Survey of Children’s preventive services and presence
Health52 of toothache, bleeding gums,
decay and cavities
Child and adolescent Health 2 US national survey, assessment of Process Effectiveness Not reported
Measurement Initiative need and utilisation of preventive
National Survey of Children services
with Special Health Care
Needs, 2009/201053
National Network for Oral 15 Practice based dashboard to Structure, Effectiveness Not reported
Health Access Dental measure caries at recall, risk process Efficiency
Dashboard54 assessment, provision of sealants, and Patient-
topical fluoride, self-management outcome centredness
goal setting and review,
completion of treatment plans,
recall rates, recommendations,
and practice finances
Agency for Healthcare 21 National patient survey to assess Outcomes Patient- Not reported
Research and Quality Patient patient’s assessment of care from centredness
Experience Measures for the dentist and staff, access to dental Timeliness
CAHPSâ Dental Plan care, dental plan costs and
Survey55 services and patient satisfaction
Australian Council on 13 Measures of use of radiographs in Process, Effectiveness Not reported
Healthcare Standards56 new patients, retreatment rates, Outcomes
extraction of deciduous teeth,
complications following
extractions

(continued)

© 2018 FDI World Dental Federation 257


Byrne et al.

Table 2 continued
Measurement collection Items Description of measure Domains Dimensions Validation
(n)

California Department of 15 Measures of the use of preventive Process Effectiveness Not reported
Health Care Services57 services, sealants, fluoride varnish,
treatment services, continuity of
care
Indian Health Service58 3 Measure of receipt of topical Process, Effectiveness Not reported
fluoride dental sealants and access Structure
to oral health care
HRSA HIV/AIDS bureau 5 Measures of provision of oral Process Effectiveness Not reported
performance measures59 health education, periodontal
screening, treatment planning and
completion and taking dental and
medical history in patients with
HIV
HRSA Oral Health Quality 9 Treatment plan completion, use of Process Effectiveness Not reported
Improvement initiative60 services, provision of oral health
education, sealants, fluoride,
periodontal screening
Q-METRIC61 1 Measure of availability of services Structure Effectiveness Not reported
CMS-146 Measures62 7 Measures of use of dental services, Process Effectiveness Not reported
preventive services, treatment
services, sealant
MCH Title V National 3 Measure of percentages of: Process Effectiveness Not reported
Performance Measure for children with decay/cavities,
Oral Health Summary63 pregnant women receiving dental
care and children receiving
preventive dental care
Oral Health Disparities 17 Measures of rates of perinatal and Process Effectiveness Not reported
Collaborative Pilot early childhood caries, treatment Timeliness
Measures64 plan construction and completion,
paediatric dental examination and
treatment plan. Fluoride varnish
application, continuity of care
and fluoride assessment
Permanente dental associates65 32 Measures of use of fluoride, Process, Effectiveness Not reported
sealants, clinical incidents, Outcomes Patient-
examination rate, continuity of centredness
care, specialist care referral, Safety
percentage of specialty care Efficiency
completed by general dentist
NCQA 2017 State of Health 1 Measure of Medicaid members Process Effectiveness Not reported
Care Quality HEDIS measure who attended for a dental visit
annual dental visits66
Denplan Excel Patient Survey67 12 Patient-reported outcome measures Outcomes Patient- Not reported
of satisfaction with their dental centredness
health and satisfaction with their
dental care provision
Denplan Excel Quality 129 122 Checklist-style questions, Structure, Safety Not reported
programme68 rating of aspects of the structures Process Effectiveness
and processes of dental care Efficiency
delivery against quality standards.
Seven percentage measures of
process

Consumer Assessment of Healthcare Providers and Systems, CAHPS; Centers for Medicare & Medicaid Services, CMS-146; the Healthcare
Effectiveness Data and Information Set, HEDIS; Health Resources & Services Administration, HRSA; Maternal and Child Health, MCH;
National Committee for Quality Assurance, NCQA; National Institute for Health and Care Excellence, NICE; Quality Measurement, Evalua-
tion, Testing, Review and Implementation Consortium, Q-METRIC.

remaining outcome measures included measures of effectiveness, patient-centredness, safety and timeliness
longevity of restorations, rates of complications and of treatment. There were few measures of efficiency
new disease presence at recall. In addition, 26.8% or equity.
(96/357) of the measures assessed structure. Of these
96 measures, 84 (87.5%) were derived from the ‘Den-
Quality measures gap analysis
plan Excel’ quality measures, which quantified provi-
sion of equipment and staff within practices. The The common themes identified in both the peer-
measures predominantly assessed dimensions of reviewed and grey literature measures are compiled in
258 © 2018 FDI World Dental Federation
Table 3 Classification of all Peer-Reviewed and Grey Literature measures identified according to dimensions and domains
Domains

Structure Process Outcome

Dimensions
Safe Dentist : Nurse ratio Recording of medical history Patient satisfaction with cleanliness of facilities
Evidence of staff training/certification Evidence of incident reporting being carried out Patient satisfaction with quality improvement initiatives
Evidence of ensuring that suppliers/contractors are Number of serious incidents
certified
Building set up to allow decontamination away from
clinical areas

© 2018 FDI World Dental Federation


Evidence of complaints handling procedures
Evidence of data protection and handling procedures
Cleanliness of practice
Evidence of practice infection-control measures
Use of single-use equipment where feasible
Certification of buildings and surgery safety
Evidence of medical emergency equipment
Effective Percentage of patients receiving oral health examination Patient rating of comfort in daily function
Percentage of patients receiving soft-tissue screening Patient rating of comfort during visit
Percentage of patients receiving emergency treatment Patient rating of ease to eat
Percentage of patients receiving planned treatment Patient rating of appearance of teeth
Percentage of patients receiving preventive advice Patient rating of comprehensiveness of examination/
treatment
Number of patients having radiographs taken Patient satisfaction with treatment received
Percentage of patients receiving dental follow-up after emergency Patient rating of quality of treatment
department visit for dental cause
Extraction to endodontics ratio Periodontal health: number of sites with bleeding on
probing
Evidence of caries and periodontal risk assessment being carried out Caries: number of decayed teeth
Percentage of patients receiving treatment for caries Caries: prevalence of early childhood caries
Prevention: fillings ratio Caries: new caries at recall of patient
Number of referrals to medical care Number of patients who are caries free
Number of referrals to secondary dental care Plaque on children’s teeth
Percentage of patients with a recording of Basic Periodontal Referral to secondary care for paediatric tooth
Examination extraction
Percentage of patients having comprehensive periodontal examination Emergency department visits from dental-related cause
Percentage of patients with history of periodontitis undergoing course of Patient-reported oral health-related quality of life
periodontal therapy
Percentage of patients with fluoride needs assessment Proportion of endodontic teeth that required
retreatment
Percentage of patients receiving preventive advice Proportion of sealants that require retreatment
Provision of fissure sealants in high-risk groups Number of deciduous teeth extracted
Provision of fluoride therapy in high-risk groups Complications following treatment
Extractions following endodontics
Proportion of fillings that subsequently required
retreatment/endodontics/extraction
Longevity of restorations

(continued)
Systematic review of quality measures

259
Table 3 continued

260
Domains

Structure Process Outcome


Byrne et al.

Patient- Percentage of patients with named regular dentist Evidence of assessment of needs Patient satisfaction with communication – listening to
centred patient concerns
Access to hygienist Development of personalised treatment plans Patient satisfaction with communication – showing
concern
Access to Out of Hours care Setting self-management goals for patients Patient satisfaction with communication – explaining
treatments
Comfort of dental practice Time spent with patients Patient satisfaction with communication – treatment/
preventive advice
Aesthetics of dental practice Percentage of patients seeing same dentist/dental team at consecutive Patient satisfaction with communication – giving
visits appropriate level of information
Ease of payment Percentage of patients having treatment by their regular dentist Patient perception of dentist’s acceptance of them as a
person
Percentage of patients who are able to see their own dentist for Patient satisfaction with courtesy and respect of dental
emergency treatment team
Extractions/endodontics completed by patient’s general dentist Patient satisfaction with time spent with dental team
Length of treatment sessions (comfort to patient) Patient satisfaction with helpfulness of staff
Patient rating of ‘atmosphere’ of dental environment
Percentage of patients who would recommend to friend
Patient rating of trust
Patient satisfaction of dental team’s ability to respond
to their needs
Patient satisfaction with written information
Patient satisfaction with dentist
Patient rating of comfort of treatment
Patient-reported pain
Patient retention – number of patients who stay with
practice over time period
Timely Percentage of group of interest who have access to Timeliness of treatment plan completion Patient satisfaction with waiting times – to get standard
dentist appointment
Percentage of group of interest who have access to Timeliness of administrative claims Patient satisfaction with waiting times – to get
oral health education emergency appointment
Patient satisfaction with waiting times – in surgery
Proportion of high-risk patients who have been able to
access dentist
Efficient Generalist: specialist ratio in primary care Patient satisfaction with cost
Quality of interpersonal relationships between Patient rating of structure of dental appointments
members of dental team
Dental team satisfaction with their leadership Percentage of treatment plans completed
Dental team satisfaction with their ability to make Number of patients failing to attend
changes
Responsiveness of practice and team to making Percentage of patients reattending within 3 months
changes
Stress of team members within dental practice Average cost of treatment per patient
Use of modern equipment Number of dental encounters/hour
Longevity of restorations
Equity Evidence of local arrangements to assess health needs Percentage of group of interest that receive preventive advice/treatment Patient perception of dentist’s acceptance of them as a
in non-dental setting person
Local arrangements to identify high needs groups Patient rating of ease of access
Local arrangements to ensure access for high-needs
groups
Ease of access – car parking, disabled access

© 2018 FDI World Dental Federation


Systematic review of quality measures

Table 3. This table identifies the broad constituent ele- dentistry. The majority of these measures showed
ment of quality that the measure attempts to capture acceptable levels of internal validity; however, their
(IoM Dimensions) and the nature of the measure (Don- usefulness in delivering a clear picture of quality and
abedian’s Domains). The categorisation of domain refers to support quality improvement is unclear.
to whether the measure assesses the structures of dental The gap analysis categorised the measures according
care delivery, the processes that are undertaken or out- to the IoM dimensions of quality and identified impor-
comes that result from the delivery of care. As such, a tant areas of care in which specific dental measures were
measure of the provision of a treatment or preventive absent. Some of these gaps are possibly populated by
programme to a population is categorised as a process measures used successfully in other areas of health care.
measure. This analysis shows a proliferation of measures It could also be the case that the IoM dimensions fail to
developed for the assessment of process and outcomes describe significant elements of quality that are impor-
within the dimensions of effectiveness and patient-cent- tant to dentistry, such as cosmetic care, functional
redness. Outcome measurement is predominantly improvement and the discomfort and anxiety associated
achieved via patient satisfaction measures. Significant with dental procedures. Further dimensions described in
gaps in measures across the domains and dimensions of the literature include tangibility26,70, responsiveness26,70,
quality are evident. empathy26,70, accessibility7,12, coordination and continu-
ity of care12, comprehensiveness12, technical quality5,
acceptability71, legitimacy71, optimality71, relevance72,
DISCUSSION
appropriateness11 and ‘caring function’11.
This systematic review describes 11 measure collections The majority of the measures within the grey litera-
(167 total measures) from the peer-reviewed literature ture focussed on processes of care. Core themes of flu-
and 24 measure collections (357 total measures) from oride prescription, provision of fissure sealants and
the grey literature that may be used to assess quality in dental attendance were identified from this search.
primary care dentistry. This study is the first known However, without corresponding measures of result-
review of quality measures in primary care dentistry ing outcomes, these process measures may have lim-
that uses a systematic review design, with the use of a ited utility19. The majority of outcome measures
priori inclusion and exclusion criteria. identified in this search assessed patient satisfaction.
Despite the structured searching methodology, it is However, a review of patient satisfaction measures
difficult to ensure that a comprehensive list of measures suggests that these are highly affected by disconfirma-
has been captured, particularly those which appear in tion and attribution bias and are inherently unreli-
the grey literature. The pace at which new measures are able73. It has been suggested that the patient’s
being developed outside the peer-reviewed literature perception of technical competence is based on the
means that further measures are likely to have been communication and caring nature of the dentist rather
produced since searching was completed. Whilst this than the work they actually carry out33. As the pri-
search may not include every measure that is available, mary stakeholders of primary care dentistry, it is intu-
it does display the major trends of how quality is cur- itively important that patients are provided with a
rently measured and therefore viewed by the dental service that provides satisfaction; however, measures
profession. Only English language sources were used, of patient satisfaction give only a limited indication of
as the cost associated with the translation of foreign the overall quality of care provided.
language measures and papers would be prohibitive. Consideration also needs to be given to the reasons
This provides a view of quality measurement domi- for collecting data on quality; the various stakeholders
nated by developed countries. Quality measures will in primary dental care have different priorities for
necessarily reflect the context and the priorities in their use. For example, policy makers may want to
which care is delivered; measures formulated in the use such data to improve equity and access for a
developed world may not be as relevant in less devel- defined population. Dentists may wish to use quality
oped communities. FDI called for an international con- measurement to fulfil a personal and professional
sensus on understanding quality in dentistry2, and the desire to improve the care for their patients or as a
influence of local context and priorities should not be way of marketing themselves or their practice.
underestimated in working towards this goal. Patients may wish to use measures to compare the
Using valid and reliable tools to measure quality is quality of care provided by different dentists. Provi-
vital in order to support day-to-day quality assessment ders of dental care may wish to use quality data for
and improvement of dental care. The 11 measure sets performance management of their dentists or as a
from the peer-reviewed literature22–24,26,27,29,30,31,32,33,38 way of remunerating and incentivising their dentists.
and three measure sets from the grey literature50,56,57 Linking quality improvement to remuneration runs
represent measures available to researchers to assess the risk of inducing unintended behaviour change,
dimensions of quality in relation to primary care leading to alterations in the provision of care74; this
© 2018 FDI World Dental Federation 261
Byrne et al.

may include services de-registering high-risk patients 9. Soni Raleigh V, Foot C. Getting the Measure of Quality:
Opportunities and Challenges. London: King’s Fund; 2010.
who threaten to lower their quality rating.
The majority of measures currently used within pri- 10. Royal College of General Practitioners. What sort of doctor? J
R Coll Gen Pract 1981 31: 698–702.
mary care dentistry have been developed by private
11. Caper P. Defining quality in medical care. Health Aff 1988 7:
companies, such as insurers or corporate care provi- 49–61.
ders, and are not published in peer-reviewed journals. 12. Health Services Research Group. Quality of care: 1. What is
Two issues arise from this: first, time and resources quality and how can it be measured?. CMAJ 1992 146: 2153–
are being used in the development of measures that 2158.
have already been developed elsewhere; and, second, 13. Institute of Medicine Committee on Quality of Health Care in
America. Crossing the Quality Chasm: A New Health System
measures are being used without the scrutiny of the for the 21st Century. Washington, DC: National Academies
wider scientific community. The authors believe that Press; 2001.
international collaboration is required to develop 14. Tickle M, O’ Malley L, Brocklehurst P et al. A national survey
valid and reliable measures that can be applied across of the public’s views on quality in dental care. Br Dent J 2015
219: E1.
a range of contexts.
15. Poorterman JH, Van Weert CM, Eijkman MA. Quality assur-
ance in dentistry: the Dutch approach. Int J Qual Health Care
CONCLUSIONS 1998 10: 345–350.
16. Mills I, Batchelor P. Quality indicators: the rationale behind
This research has highlighted the lack of valid, reliable their use in NHS dentistry. Br Dent J 2011 211: 11–15.
measures used to assess quality. Current efforts in pro- 17. National Quality Forum. Oral Health Performance Measure-
ducing quality measures are being conducted with little ment: Environmental Scan, Gap Analysis & Measure Topics
Prioritization. Washington, DC: National Quality Forum; 2012.
academic grounding, leading to measures of limited
18. Institute of Medicine and National Research Council Commit-
validity and utility that fail to capture some key elements tee on Oral Health Access to Services. Improving Access to
of quality. International collaboration is required to Oral Health Care for Vulnerable and Underserved Populations.
agree on a definition of quality and a shared understand- Washington, DC: National Academies Press; 2011.
ing of what quality means for dentistry. Consensus 19. Bader JD. Challenges in quality assessment of dental care. J Am
would guide the production of valid, reliable measures Dent Assoc 2009 140: 1456–1464.
within an agreed framework for the benefit of patients. 20. OpenGrey [Internet] [cited 29/03/2018]. [Available from: http://
www.opengrey.eu.]
21. DeVellis R. Scale Development: Theory and Applications.
Acknowledgements Thousand Oaks, CA: Sage; 2003.
22. Gonzales PS, Martins IEF, Biazevic MG et al. Dental Manage-
No further acknowledgements are made. No financial ment Survey Brazil (DMS-BR): creation and validation of a
support was provided for this project. management instrument. Braz Oral Res 2017 31: e26.
23. Goetz K, Hasse P, Szecsenyi J et al. Questionnaire for measur-
ing organisational attributes in dental-care practices: psychome-
Competing interests tric properties and test-retest reliability. Int Dent J 2016 66:
93–98.
The authors have no competing interests to declare. 24. Cheng BS, McGrath C, Bridges SM et al. Development and
evaluation of a Dental Patient Feedback on Consultation skills
(DPFC) measure to enhance communication. Community Dent
REFERENCES Health 2015 32: 226–230.

1. European Regional Organisation of the Federation Dentaire 25. Wong HM, Bridges SM, McGrath CP et al. Impact of promi-
Internationale. ERO-FDI WG Quality in Dentistry Report nent themes in clinician-patient conversations on caregiver’s
2007-2010. 2010 [cited 03/04/2018]. [Available from: www.ero perceived quality of communication with paediatric dental vis-
dental.org/ddc/ddid97.] its. PLoS ONE 2017 12: e0169059.

2. FDI World Dental Federation. Quality in Dentistry. 2017 [cited 26. Bahadori M, Raadabadi M, Ravangard R et al. Factors affect-
03/04/2018]. [Available from: http://www.fdiworlddental.org/ ing dental service quality. Int J Health Care Qual Assur 2015
resources/policy-statements-and-resolutions/quality-in-dentistry.] 28: 678–689.

3. Campbell S, Tickle M. What is quality primary dental care? Br 27. Reissmann DR, Hacker T, Farhan D et al. The Burdens in Pros-
Dent J 2013 215: 135–139. thetic Dentistry Questionnaire (BiPD-Q): development and vali-
dation of a patient-based measure for process-related quality of
4. Maxwell RJ. Quality assessment in health. Br Med J (Clin Res care in prosthetic dentistry. Int J Prosthodont 2013 26: 250–
Ed) 1984 288: 1470–1472. 259.
5. Blumenthal D. Part 1: Quality of care–what is it? N Engl J Med 28. Hacker T, Heydecke G, Reissmann DR. Impact of procedures
1996 335: 891–894. during prosthodontic treatment on patients’ perceived burdens.
6. Donabedian A. The Definition of Quality and Approaches to its J Dent 2015 43: 51–57.
Assessment. Chicago, IL: Health Administration Press; 1980. 29. Reissmann DR, Semmusch J, Farhan D et al. Development and
7. Campbell SM, Roland MO, Buetow SA. Defining quality of validation of the Burdens in Oral Surgery Questionnaire (BiOS-
care. Soc Sci Med 2000 51: 1611–1625. Q). J Oral Rehabil 2013 40: 780–787.
8. Heath I, Rubinstein A, Stange KC et al. Quality in primary 30. Kikwilu EN, Kahabuka FK, Masalu JR et al. Satisfaction with
health care: a multidimensional approach to complexity. BMJ urgent oral care among adult Tanzanians. J Oral Sci 2009 51:
2009 338: b1242. 47–54.

262 © 2018 FDI World Dental Federation


Systematic review of quality measures

31. Keller S, Martin CGC, Evensen CT et al. The development and 52. The Child and Adolescent Health Measurement Initiative.
testing of a survey instrument for benchmarking dental plan Guide to Topics & Questions Asked: National Survey of Chil-
performance using insured patients’ experiences as a gauge of dren’s Health. 2016 [cited 03/04/2018]. [Available from: http://
dental care quality. J Am Dent Assoc 2009 140: 229–237. childhealthdata.org/learn/NSCH/topics_questions/2016-nsch-
32. Larsson BW, Bergstrom K. Adolescents’ perception of the quality guide-to-topics-and-questions.]
of orthodontic treatment. Scand J Caring Sci 2005 19: 95–101. 53. The Child and Adolescent Health Measurement Inititiative.
33. Corah NL, O’Shea RM, Pace LF et al. Development of a Guide to Topics & Questions Asked: National Survey of Chil-
patient measure of satisfaction with the dentist: the Dental Visit dren with Special Health Care Needs, 2009/2010. 2010 [cited
Satisfaction Scale. J Behav Med 1984 7: 367–373. 05/04/2018]. [Available from: http://childhealthdata.org/learn/
NS-CSHCN/topics_questions/3.3.0-2009-10-national-survey-of-
34. Olausson M, Esfahani N, Ostlin J et al. Native-born versus for- cshcn–topics-questions.]
eign-born patients’ perception of communication and care in
Swedish dental service. Swed Dent J 2016 40: 91–100. 54. National Network for Oral Health Access. The Dental Dash-
board. 2015 [cited 03/04/2018]. [Available from: http://www.
35. Sun N, Burnside G, Harris R. Patient satisfaction with care by nnoha.org/resources/dental-dashboard-information/.]
dental therapists. Br Dent J 2010 208: E9; discussion 212–213.
55. Agency for Healthcare Research and Quality. Patient Experience
36. Hakeberg M, Heidari E, Norinder M et al. A Swedish version Measures for the CAHPSâ Dental Plan Survey Rockville, MD.
of the Dental Visit Satisfaction Scale. Acta Odontol Scand 2000 2011 [cited 03/04/2018]. [Available from: https://www.ahrq.gov/
58: 19–24. cahps/surveys-guidance/dental/about/survey-measures.html.]
37. Stouthard ME, Hartman CA, Hoogstraten J. Development of a 56. Australian Council on Healthcare Standards. Australasian Clini-
Dutch version of the Dental Visit Satisfaction Scale. Commu- cal Indicator Report, 18th ed. Sydney: Australia; 2017.
nity Dent Oral Epidemiol 1992 20: 351–353.
57. California Department of Health Care Services. FFS Perfor-
38. Davies AR, Ware JE Jr. Measuring patient satisfaction with mance Measures California. 2018 [cited 03/04/2018]. [Avail-
dental care. Soc Sci Med A 1981 15: 751–760. able from: http://www.dhcs.ca.gov/services/Pages/FFSPerforma
39. Lee CT, Zhang S, Leung YY et al. Patients’ satisfaction and nceMeasures.aspx.]
prevalence of complications on surgical extraction of third 58. Indian Health Service. GPRA/GPRAMA resource guide. 2013 [cited
molar. Patient Prefer Adherence 2015 9: 257–263. 03/04/2018]. [Available from: https://www.ihs.gov/california/tasks/
40. Milgrom P, Spiekerman C, Grembowski D. Dissatisfaction with sites/default/assets/File/GPRA/GPRAResourceGuide_v2.pdf.]
dental care among mothers of Medicaid-enrolled children. 59. Health Resources & Services Administration. HIV/AIDS Bureau
Community Dent Oral Epidemiol 2008 36: 451–458. Performance measures. 2017 [cited 03/04/2018]. [Available
41. Skaret E, Berg E, Raadal M et al. Reliability and validity of the from: https://hab.hrsa.gov/sites/default/files/hab/clinical-quality-
Dental Satisfaction Questionnaire in a population of 23-year- management/oralhealthmeasures.pdf.]
olds in Norway. Community Dent Oral Epidemiol 2004 32: 60. Anderson JR, editor. HRSA’s Oral Health Quality Improve-
25–30. ment Initiative. National Oral Health Conference; 2010; St
42. Brennan DS, Gaughwin A, Spencer AJ. Differences in dimen- Louis, MO. Conference Presentation [cited 03/04/2018]. [Avail-
sions of satisfaction with private and public dental care among able from: http://www.nationaloralhealthconference.com/docs/
children. Int Dent J 2001 51: 77–82. presentations/2010/Jay%20Anderson%20-%20Improving%20Oral
%20Healthcare%20in%20Safety%20Net%20Setti.pdf.]
43. Mascarenhas AK. Patient satisfaction with the comprehensive
care model of dental care delivery. J Dent Educ 2001 65: 61. National Quality Measures Clearinghouse. Availability of ser-
1266–1271. vices: the number of dental providers who have provided any
dental procedure to at least one child, per 1,000 eligible chil-
44. Chapko MK, Bergner M, Green K et al. Development and vali- dren. Rockville, MD: Agency for Healthcare Research and
dation of a measure of dental patient satisfaction. Med Care Quality (AHRQ); 2015 [cited 03/04/2018]. [Available from:
1985 23: 39–49.
https://www.qualitymeasures.ahrq.gov/summaries/summary/49899/
45. Department of Health. Dental Quality and Outcomes Frame- availability-of-services-the-number-of-dental-providers-who-have-
work. London, UK: Department of Health; 2016. provided-any-dental-procedure-to-at-least-one-child-per-1000-eligible-
46. National Institute for Health and care Excellence. Oral Health children?q=dental.]
in Care Homes: Quality standard. London: National Institute 62. Medicaid. 2700.4 Instructions for Completing Form CMS-416:
for Health and care Excellence; 2017. Annual Early and Periodic Screening, Diagnostic, and Treat-
47. National Institute for Health and Care Excellence. Oral Health ment (EPSDT) Participation Report 2017 [cited 03/04/2018].
Promotion in the Community: Quality Standard. London: [Available from: https://www.medicaid.gov/medicaid/benefits/
National Institute for Health and Care Excellence; 2016. downloads/cms-416-instructions.pdf.]
48. NHS England. Dental Assurance Framework; [2014 cited 03/ 63. Association of State & Territorial Dental Directors. MCH Title
04/2018]. [Available from: https://www.england.nhs.uk/wp- V National Performance Measure for Oral Health Details and
content/uploads/2014/05/dental-assurance-frmwrk-may.pdf] Recommended Actions. Reno; 2015 [cited 03/04/2018]. [Avail-
able from: http://www.astdd.org/docs/mch-npm-combined-
49. Dental Quality Alliance User Guide for Adult Measures Calcu- summary-and-detailed-overview-03-20-2015.pdf.]
lated using administrative Claims data. 2018 [cited 03/04/
2018]. [Available from: https://www.ada.org/~/media/ADA/Scie 64. The Health Resources and Services Administration’s Health
nce%20and%20Research/Files/DQA_2018_Adult_Measures_ Disparities Collaborative. Oral Health Disparities Collaborative
User_Guide.pdf?la=en.] Implementation Manual 2012 [cited 03/04/2018]. [Available
from: http://www.nnoha.org/nnoha-content/uploads/2013/09/
50. Dental Quality Alliance. User Guide for Pediatric Measures OHDC-Implementation-Manual-with-References.pdf.]
Calculated Using Administrative Claims Data 2018 [cited 03/
04/2018]. [Available from: https://www.ada.org/~/media/ADA/ 65. Snyder J. Quality Measurement Models 2015 [cited 03/04/
Science%20and%20Research/Files/DQA_2018_Pediatric_%20 2018]. [Available from: http://www.ada.org/en/science-research/
Measures_User_Guide.pdf?la=en.] dental-quality-alliance/2015-dqa-conference.]
51. Dental Quality Alliance. Electronic Pediatric Measures. 2016. 66. National Care Quality Alliance. Annual Dental Visits 2017
[cited 03/04/2018]. [Available from: https://www.ada.org/en/sc [cited 03/04/2018]. [Available from: http://www.ncqa.org/
ience-research/dental-quality-alliance/dqa-measure-activities/elec report-cards/health-plans/state-of-health-care-quality/2017-table-
tronic-pediatric-measures.] of-contents/dental.]

© 2018 FDI World Dental Federation 263


Byrne et al.

67. Denplan. Denplan Excel Patient Survey 2016 [cited 03/04/


2018]. [Available from: https://www.hilltondentistry.co.uk/pdfs/ APPENDIX 1. SEARCH STRATEGY
excel-patient-survey-questionnaire.pdf.]
The Following search strategy was used in MEDLINE:
68. Denplan. Excel Practice Assessment Survey. 2016.
1 exp *”Quality of Health Care”/
69. Dental Quality Alliance. Procedure Manual for Performance
Measures Development and Maintenance. Chicago, IL: Ameri- 2 ((quality or performance$ or satisf$) adj5 (indicat
can Dental Association; 2016. $ or activit$ or framework$ or tool$ or matrix or
70. Parasuraman A, Zeithaml VA, Berry LL. A conceptual-model matrice$ or measure$ or guideline$ or accreditat$
of service quality and its implications for future-research. J or indices or index or certificat$ or regulat$ or
Mark 1985 49: 41–50.
assur$ or standard$ or control$ or audit$ or met-
71. Donabedian A. The seven pillars of quality. Arch Pathol Lab
Med 1990 114: 1115–1118. ric or improv$ or criteri$ or assess$ or valid$ or
72. Maxwell RJ. Dimensions of quality revisited: from thought to
scale$ or checklist$)).ti,ab. not (quality adj2
action. Qual Health Care 1992 1: 171–177. life).mp. [mp=ti, ab, sh, hw, ot, kw, bt, id, cc, ac,
73. Newsome PR, Wright GH. A review of patient satisfaction: 1. de, md, sd, so, pt, an, tn, dm, mf, dv, fx, nm, kf,
Concepts of satisfaction. Br Dent J. 1999;186:161–165. px, rx, ui, sy]
74. Brocklehurst P, Price J, Glenny AM et al. The effect of different 3 1 and 2
methods of remuneration on the behaviour of primary care den- 4 exp Dentists/
tists. Cochrane Database Syst Rev 2013 11: CD009853.
5 ((dentist$ or (dental adj5 practitioner$)) not stu-
dent$).ti,ab.
Correspondence to:
6 exp Dental auxiliaries/
Matthew Byrne,
7 ((dental and (hygienist$ or therapist$)) not student
NIHR Academic Clinical Fellow in Primary Care
$).ti,ab.
Dentistry,
8 (“oral health practitioner” or “dental assistant$”
Division of Dentistry,
School of Medical Sciences, or “dental auxil$” or “dental hygiene practitioner
Faculty of Biology, Medicine and Health, $” or “community dental health co-ordinator$”
or “oral health co-ordinator$”).ti,ab.
University of Manchester,
9 4 or 5 or 6 or 7 or 8
Oxford Road,
10 3 and 9
Manchester M13 9PL, UK.
11 Remove duplicates from 10
Email: matthew.byrne-6@postgrad.manchester.ac.uk

264 © 2018 FDI World Dental Federation

You might also like