Periodontology 2000, Vol.
37, 2005, 12–28 Copyright
Blackwell Munksgaard 2005
Printed in Denmark. All rights reserved
Evidence-based periodontology,
systematic reviews and research
quality
I A N N E E D L E M A N , D A V I D R. M O L E S & H E L E N W O R T H I N G T O N
Periodontology has a rich background of research
and scholarship. A simple MEDLINE search of ÔPeri-
odontal DiseasesÕ OR ÔPeriodontitisÕ alone from 1966
to 2003 brings up more than 45,000 hits. Therefore,
efficient use of this wealth of research data needs to
be a part of periodontal practice. Evidence-based
periodontology aims to facilitate such an approach,
accelerating the introduction of the best research into
patient care.
This chapter will review the concepts of evidence-
based periodontology, introduce the systematic
review as a research tool and examine how evidence-
based periodontology can both inform on and benefit
healthcare in periodontology. Finally, we will exam-
ine the strengths and limitations of different research
designs and their appraisal. We hope that the infor-
mation in this chapter will provide a basic under-
standing of the concepts that will be relevant to
reading and enjoying the other chapters in this vol-
ume of Periodontology 2000.
What is evidence-based
periodontology?
Evidence-based periodontology is the application of
evidence-based health care to periodontology. A
useful definition of evidence-based health care has
been proposed by Muir Gray: ÔAn approach to
decision making in which the clinician uses the
best evidence available, in consultation with the
patient, to decide upon the option which suits that
patient bestÕ (8). Therefore, evidence-based period-
ontology is a tool to support decision making and
integrating the best evidence available with clinical
practice.
12
PERIODONTOLOGY 2000
The highest quality evidence will be used if it exists,
but if it does not, lower levels of evidence will be
considered. Lower levels of evidence usually means
research designs more prone to bias and therefore
with less reliable data. However, the nature, strengths
and weaknesses of the evidence will be made clear to
the reader. In addition, wherever possible, the data
presentation supplies more clinically relevant infor-
mation, including the probability of achieving a cer-
tain effect such as a benefit, and considering possible
adverse effects.
What evidence-based
periodontology is not
Evidence-based periodontology is not simply sys-
tematic reviews of randomized controlled trials,
although this can be an important aspect. Evidence-
based periodontology is an approach to patient-care
and nothing more. The expectations that are some-
times laid on it can be inappropriate. It cannot pro-
vide answers if research data do not exist (other than
using expert opinion) and it cannot substitute for
highly developed clinical skills. Therefore, it can
never be cookbook healthcare or use statistics in
isolation to drive clinical care. Instead it is the com-
prehensive integration of appropriate research evi-
dence, patient preference and clinical expertise
(Fig. 1).
This can be illustrated with data from a recent
systematic review on periodontal plastic surgery for
root surface coverage in localized Miller Class I and II
defects (25). The data from the systematic review
demonstrated that connective tissue grafts were sig-
nificantly better than guided tissue regeneration in

Clinician’s Patient
skills preferences
and views
Evidence-based
periodontology
Best evidence
available
Fig. 1. How evidence-based periodontology fits into
healthcare. Reproduced with permission from Clarkson,
J, Harrison, JE, Ismail, AI, Needleman, IG, Worthington, H,
eds. Evidence Based Dentistry for Effective Practice.
London: Martin Dunitz, 2003 (20).
reducing recession (mean difference 0.43 mm,
95%CI [0.62,0.23], chi square for heterogeneity 7.8
(df ¼ 5) P ¼ 0.17). This indicates that the pooled
difference between six studies included in the review
is 0.43 mm, with a 95% confidence interval from 0.62
to 0.23. The chi-square test indicates that there is no
evidence of any heterogeneity between the studies
(they could theoretically all be measuring the same
difference). So, does this mean that only connective
tissue grafts should be used in the treatment of
localized recession defects? Clearly, this would not be
appropriate. The data show that both GTR and con-
nective tissue grafts can work. For the selected out-
come, which was recession reduction, connective
tissue grafts produce 0.43 mm greater effect; the
result is both reasonably precise (judged by the
confidence interval) and the studies from which
the data were taken were similar (no evidence of
heterogeneity).
However, recession reduction might not be the
only outcome of interest. The two surgical proce-
dures are very different. One requires the harvesting
of a soft tissue graft from the palate and the other
does not. There are no data available examining
patient preferences, but it is likely that some indi-
viduals will prefer a procedure that does not involve
two surgical sites, even if it does not reduce
recession to the same extent. It is also possible that
aesthetics are different following the two procedures
Evidence-based periodontology
and this might inform on the decision. However,
surprisingly, no data are available on patient views
on aesthetics comparing the two procedures.
Therefore, this evidence-based approach to man-
agement of recession has produced the best avail-
able evidence, shown how precise this estimate
actually is, and highlighted the limitations of the
evidence, in this case the lack of data on some
outcomes that are relevant to the decision making
process.
Clinical relevance
One of the barriers to the application of research
findings in clinical practice is the way that results are
often presented. Typically, a mean value will be
published, based on a statistical analysis comparing
experimental groups. Such a value in conjunction
with its associated 95% confidence interval is useful
to determine whether there is a statistically signifi-
cant difference between groups and will often be a
requirement of a study designed for regulatory
approval. However, this type of analysis is not
designed to provide information about the probabil-
ity of achieving a certain outcome were the reader to
apply it in practice. Such an outcome could include
achieving a health benefit or preventing further dis-
ease. For instance, in a meta-analysis from a sys-
tematic review on guided tissue regeneration (GTR)
for periodontal infrabony defects, the additional
benefit of using GTR over access flap surgery was a
1.1 mm gain in clinical attachment (21, 22). This
should, however, not be interpreted as the additional
benefit to be expected every time that GTR is used
instead of access flap surgery.
One approach to analysing and presenting data in a
more clinically useful format is to calculate the
number needed to treat (NNT). This is the number of
patients that would need to be treated to achieve a
stated benefit (NNTb) or to avoid a stated harm
(NNTh). It is derived from a dichotomous outcome
such as the proportion of sites achieving at least
2 mm gain in attachment. For the GTR meta-analysis,
and using this benefit, the NNTb is eight. In other
words, for every eight patients treated with GTR, you
can expect one to have at least 2 mm more gain in
clinical attachment than if you had used an access
flap (95% confidence interval [4,33]). For detailed
guidance regarding the use and calculation of
the NNT the reader is recommended to the elec-
tronic journal ÔBandolierÕ: http://www.jr2.ox.ac.uk/
bandolier/booth/painpag/NNTstuff/numeric.htm.
13
Needleman et al.

bias and therefore vary in their reliability and will be

Evidence-based periodontology vs.
traditional periodontology
High quality research and the use of evidence are
fundamental to both evidence-based periodontology
and traditional periodontology. The differences
between these approaches emanate from how
research informs clinical practice. Evidence-based
periodontology uses a more transparent approach to
acknowledge both the strengths and the limitations
of the evidence. An appreciation of the level of
uncertainty or imprecision of the data is essential in
order to offer choices to the patient regarding
treatment options. Evidence-based periodontology
also attempts to gather all available data and to
minimize bias in summarizing the data. These
aspects are key to decision making and are high-
lighted in Table 1.
Furthermore, evidence-based periodontology
acknowledges explicitly the type or level of research
on which conclusions are drawn. The research hier-
archy is discussed in more detail later in this chapter.
However, one aspect that influences the reliability of
the data is the control of bias. Bias is a collective term
for factors that systematically distort the results of
research away from the truth. Different research
designs offer different possibilities for the control of
discussed further below.
The components of evidence-based
periodontology
An overview of the components is given in Fig. 2.
Evidence-based periodontology starts with the
recognition of a knowledge gap. From the know-
ledge gap comes a focussed question that leads on
to a search for relevant information. Once the rele-
vant information is located, the validity of the
research needs to be considered in two broad areas.
Firstly, is the science good (internal validity)? Inter-
nal validity focuses on the methodology of research.
Secondly, can the findings be generalized outside of
the study (external validity)? External validity might
be affected by the way treatment was performed. For
instance, if the time spent on treatment was exten-
sive it might not be practical to provide this therapy
outside of a research study. Another example could
relate to the use of many specific inclusion criteria
in a trial which could make it difficult to generalize
the findings to a wider group of patients. The
question the reader should ask is whether their types
of patients are so different from the study that it is

Recognize clinical knowledge gap

Table 1. Comparison of evidence-based periodon-
tology vs. traditional periodontology
Evidence-based Traditional
periodontology periodontology Develop into a focussed question

Similarities
• High value of clinical
skills and experience
Search for evidence
• Fundamental importance
of integrating evidence
with patient values Reject
Differences if invalid Evaluate the evidence
• Uses best evidence • Unclear basis or poor
available of evidence
• Systematic appraisal • Unclear or absent
of quality of evidence appraisal of quality Integrate into practice
of evidence
• More objective, more • More subjective, more
transparent and less opaque and more
biased process biased process
• Greater acceptance • Greater tendency to Evaluate the effects
of levels of uncertainty black and white
Fig. 2. The steps of evidence-based periodontology.
conclusions
Reproduced with permission from Clarkson, J, Harrison,
Reproduced with permission from Clarkson, J, Harrison, JE, Ismail, AI, JE, Ismail, AI, Needleman, IG, Worthington, H, eds. Evi-
Needleman, IG, Worthington, H, eds. Evidence Based Dentistry for Effective dence Based Dentistry for Effective Practice. London:
Practice. London: Martin Dunitz, 2003 (20).
Martin Dunitz, 2003 (20).

14

reasonable to expect differences in outcomes. After
locating and appraising the research, the results
then need to be applied clinically, or at least inclu-
ded in a range of options. Finally, the results in
clinical practice need to be evaluated to reveal
whether the adopted technique achieved the
expected outcome.
The example of gingival recession mentioned ear-
lier can be used to illustrate this approach. The
uncertainty might relate to whether to change from
using connective tissue grafts for recession defects to
guided tissue regeneration and can be translated into
a focussed question. Here the patient or problem
group could be refined more closely to localized
recession defects and perhaps Miller Class I or II, as
we might reasonably expect these lesions to respond
differently from more advanced lesions. The inter-
vention is guided tissue regeneration and the com-
parison, connective tissue grafts. The outcomes would
include change in recession or possibly the chance of
achieving complete root coverage. Since the proce-
dure is primarily for aesthetics, a patient-centred
assessment of aesthetics should be an outcome. As
always, there must be a consideration of adverse
effects and these might include pain, postoperative
infection, and severe bleeding postoperatively.
Reassembling this structure into a focussed
question would lead to ÔIn patients with localized
Miller Class I or II recession defects, what is the
effect of guided tissue regeneration vs. connective
tissue grafts on change in recession, chance of
complete defect coverage and aesthetics and what
are the adverse effects?’ For this particular research
question, the randomized controlled trial is best able
to address the change in recession outcome. For the
other outcomes, other research designs might have
been used, such as observational studies. Preferably,
we would like to find a systematic review that will
have completed the searching and study appraisal
for us.
The search quickly identifies a systematic review
(25). The review has a research question that is
appropriate to our question and demonstrates a
statistically superior effect of connective tissue grafts
compared with guided tissue regeneration. The
review also acknowledges certain limitations. In
terms of the validity of the meta-analysis, the
reviewers urge caution as publication bias could be
affecting the overall result, but this could not be
tested due to the low number of studies. Publication
bias is discussed later in this chapter. Another limi-
tation was that there were no data on aesthetics or
adverse effects.
Evidence-based periodontology
Therefore, having reviewed the data, it is clear that
there is good evidence to indicate that connective
tissue grafts have a greater effect on change in
recession than guided tissue regeneration, although
there are several limitations to this evidence. Clinical
recommendation is tempered by the lack of data on
aesthetics and adverse effects and the possible
exaggeration of benefit through publication bias. This
information can then inform on the case presenta-
tion to the patient and a choice of options discussed
and agreed. The outcome of treatment can then be
evaluated to see whether the desired endpoint was
achieved and this helps to refine the case presenta-
tion discussion in future.
Systematic reviews
One important element of evidence-based period-
ontology is the systematic review. Systematic reviews
are a research design termed Ôresearch synthesisÕ.
That is, they use research methodology to pool data
from multiple studies that address a particular
hypothesis. A systematic review can be defined as a
review of a clearly formulated question that attempts
to minimize bias using systematic and explicit
methods to identify, select, critically appraise and
summarize relevant research.
The description of systematic reviews as providing
the highest level of evidence is widespread but also
raises expectations that may or may not be fulfilled. A
realistic understanding of what a systematic review
can provide is important for the appropriate use of
this type of evidence (Table 2). More detailed infor-
mation on systematic reviews exist (5, 19), and guides
to conducting them are freely available (1, 13).
As with all research, a systematic review starts from
an hypothesis. This is derived from a focussed ques-
tion which is set to answer a particular area of
uncertainty. For instance, for the systematic review
on smoking and periodontal therapy in the chapter by
Labriola et al. in this volume, the focussed question
was: ‘‘In patients with chronic periodontitis, what is
the effect of smoking or smoking cessation on the
response to nonsurgical periodontal therapy in terms
of clinical and patient-centred outcomes?’’(14). The
question has set the types of patients (individuals
with chronic periodontitis undergoing nonsurgical
therapy), type of exposure (cigarette smoking) and
types of outcomes (clinical and patient-centred) to be
investigated, each aspect being defined in more detail
within the protocol. As this is a prognostic research
question, where exposure (smoking) cannot be
15
Needleman et al.

Table 2. The potential of systematic reviews

What a high quality systematic review can do
• Find and summarize all available studies.
– A comprehensive search will identify all relevant studies up to the point of the date of the completion of the search.
This should give the reader greater confidence that bias in selecting studies has been minimized.
• Provide an objective assessment of the quality or research and in particular the degree of protection from bias within
the original studies.
– Components of methodological quality that have evidence of affecting bias can be evaluated. It may be much harder
to evaluate the impact of other ÔqualityÕ issues if there is no consensus on how to measure them. An example could be
the quality of the treatment provided.
• Estimate research effects across multiple studies with meta-analysis.
– Meta-analysis is valid only if studies are similar in their research question and design.
– Meta-analysis can estimate uncertainty and precision of the effect.
– Meta-analysis may generate hypotheses for differential effects across subgroups of the population tested.
• If the effect is consistent across multiple studies (with small differences in design), then it may more readily possible to
generalise the results to clinical practice than the results from a single study.
• Overcome limitations of underpowered studies in detecting a true difference if such a true difference really exists.

What a high quality systematic review cannot do

• It cannot be used in isolation to dictate clinical practice.
• It is a synthesis of available research and must be used in context with clinical judgement and patient preference.
• Produce strong conclusions if the research base is weak in quality.
– The value of the review will then be to present a comprehensive objective summary of the strength of the data and to
identify the design of research to answer important gaps.
• Overcome limitations of narrowly designed clinical research.
– If the clinical studies only investigate the effect of an intervention in highly selected individuals, the conclusions
cannot be generalised outside of these conditions. However, the objective communication of any limitations in the
research base will help to set the degree of uncertainty and indicate the priorities for future research.
• Exclude relevant studies.
– Although the majority of hits from the search will be excluded, this is due to the deliberate strategy of achieving high
sensitivity (likelihood of finding all relevant studies) but low precision (likelihood of only finding relevant studies).
Therefore, it is common to find that more than 90% of the search records are totally irrelevant to the question and
must be excluded. The alternative approach of aiming for high precision also carries a high risk of missing relevant
studies, although it will appear as if few studies are being excluded.
• Be a miracle research design.
– All research has strengths and limitations/weaknesses. Systematic reviews are no different from other research
designs in this respect.

randomized, the cohort study is the research design
of choice to incorporate into this investigation.
These components help in the design of the search
strategy that aims to be comprehensive. Usually,
searching of multiple electronic databases is carried
out together with searching other sources. The most
commonly searched databases include MEDLINE
(strong on English-language studies), EMBASE
(strong on other European languages), and CENTRAL
(the Cochrane Collaboration register of trials
records). Searching only electronic databases can
miss important data, as records on the database may
not be appropriately coded. To supplement the
electronic search, other approaches are used.
Typically, this will include checking for publications
in the bibliographies of retrieved studies and review
articles, hand-searching of journals for missed
reports, and contacting researchers, industry and
journals for unpublished data.
The search strategy aims for high sensitivity, i.e. the
greatest chance of finding all relevant studies. The
downside of this approach is low precision, i.e. in
addition to the relevant studies, the search will
identify many irrelevant hits (probably more than
90% of hits from the search will not be relevant). For
example, in a systematic review on systemic anti-
microbials, the search identified 1,300 hits. Screening
of the title and abstracts (if available) indicated that
158 papers might be relevant. Once the full text of the
studies had been reviewed, 25 trials were judged
relevant and could be included (9). At first sight,
rejecting 1,275/1,300 studies would appear to be
wasting potentially useful data. However, the delib-
erately inclusive search identifies a large number of

16

irrelevant papers, including veterinary medicine,
review papers, duplicate reporting of research and
laboratory studies. The systematic review screens the
search findings against prestated criteria. These cri-
teria aim to exclude studies irrelevant to answering
the question, but do not attempt to exclude on the
basis of the quality of the study. Instead, the quality of
relevant studies is critically appraised using objective
criteria that could influence the study outcome.
The dimension of quality can be incorporated into
a systematic review in a number of ways. If the
studies are similar enough to be combined in a meta-
analysis, the impact of quality on the overall result
can be estimated (through sensitivity analyses or
meta-regression). If meta-analysis is not possible, the
quality of studies can be summarized in narrative
tables, in particular for those elements designed to
protect against bias. Whilst this may not be as
powerful as the use of meta-analysis, it will highlight
limitations to be placed upon the conclusions. Fol-
lowing pooling of the data with meta-analysis or
qualitative methods, the conclusions from the
investigation can be drawn and related to the data
derived from the review.
The systematic review will not be appropriate for
some questions. For instance, to address the ques-
tion, ÔWhich indices have been used to measure gin-
givitis?Õ a descriptive survey will be more appropriate.
However, systematic methods should be adopted for
some aspects, in particular to ensure that the search
is both comprehensive and contemporary. This
might form an important initial stage to answering a
question such as ÔWhich gingivitis indices have been
validated?Õ This is a research question answerable by
a systematic review.
The development of evidence-
based periodontology
Evidence-based periodontology is built upon devel-
opments in clinical research design throughout the
18th, 19th and 20th centuries (15, 20, 23, 28).
Evidence-based medicine has only been known for
just over a decade and the term was coined by the
clinical epidemiology group at McMaster University
in Canada (4).
The influence of the McMaster group spread far.
One of the earliest to take up the challenge in peri-
odontology (in fact in oral health research overall)
was Alexia Antczak Bouckoms in Boston, USA. Ant-
czak Bouckoms and colleagues challenged the
methods and quality of periodontal clinical research
Evidence-based periodontology
in the mid 1980s (3) and set up an Oral Health Group
as part of the Cochrane Collaboration in 1994. The
editorial base of the Oral Health group subsequently
moved to Manchester University in 1997 with Bill
Shaw and Helen Worthington as co-ordinating edi-
tors (http://www.cochrane-oral.man.ac.uk/). The
first Cochrane systematic review in periodontology
was published in 2001 and researched the effect of
guided tissue regeneration for infrabony defects (21).
Many individuals have been active in the critical
analysis of the periodontal literature. These include
Jan Egelberg, Loma Linda University, Noel Claffey,
Trinity College Dublin, and Gary Greenstein, Uni-
versity of Medicine and Dentistry of New Jersey.
There have been many notable events in evidence-
based periodontology. The 1996 World Workshop in
Periodontology held by the American Academy of
Periodontology included elements of evidence-based
healthcare, supported by Michael Newman at UCLA
(2). The 2002 European Workshop on Periodontology
became the first international workshop to use rig-
orous systematic reviews to inform the consensus.
The workshop was organized by the European Acad-
emy of Periodontology for the European Federation of
Periodontology, under the chairmanship of Professor
Klaus Lang. Sixteen focussed and rigorous systematic
reviews formed the basis of intense consensus dis-
cussions. A similar approach was used subsequently
by the American Academy of Periodontology for the
Contemporary Science Workshop in 2003.
Many other groups are now using similar methods
in healthcare and research. Most recently, the Inter-
national Center for Evidence-Based Oral Health was
launched in 2003 (http://www.eastman.ucl.ac.uk/
iceboh) to produce high quality evidence-based
research with an emphasis on, but not limited to,
periodontology and implants and to provide generic
training in systematic reviews and research methods.
Study designs and critical appraisal
Different study designs
Different clinical research questions require evalua-
tion through different study designs. A study to
determine the effectiveness of surgical therapy com-
pared with nonsurgical debridement deals with the
effectiveness of a treatment option and would be best
answered by a randomized controlled trial (RCT) or,
ideally, a systematic review of RCTs. However, it must
be noted that although RCTs and systematic reviews
of RCTs may well be the Ôgold standardÕ upon which
17
Needleman et al.
to base decisions on the effectiveness of interven-
tions, they are not necessarily appropriate, or ethical,
to answer all questions. An RCT would obviously not
be helpful in answering the question posed on the
epidemiological evidence of plaque in the etiology of
periodontitis. For such questions regarding prognosis
or etiology, cohort studies would be more appropri-
ate. Table 3 illustrates the types of study designs
most suitable for different types of research questions
arising in periodontology. The most appropriate
source of information will depend upon the type of
study design being sought.
Critical appraisal: Why, what and how?
Why critically appraise?
Evidence-based periodontology, as its name implies,
is periodontology that is based on evidence, but not
just any so-called evidence. Richards wrote a toolbox
article for the journal Evidence-based Dentistry enti-
tled ÔNot all evidence is created equalÕ (24). We have
already seen in this chapter that the quality of evi-
dence may vary according to study design and that
this has led to the concept that there can be a hier-
archy of evidence. One hierarchy is illustrated in
Table 4 and is specific to studies on therapy, pre-
vention, etiology, and harm. Other suggested levels
for different types of research question can be found
at the Center for Evidence-Based Medicine: (http://
www.cebm.net/levels_of_evidence.asp#levels).
The publication of research in a high-ranking
journal may not be an absolute guarantee of quality.
Within the medical literature there are methodolo-
gical studies which have empirically shown that
quality is not merely a hypothetical concept but also
affects study outcomes. As examples of this, the re-
views of Schulz et al. (26), Moher et al. (17) and Juni
et al. (12) showed that in studies in which there was
inadequate concealment of treatment allocation, the
treatment effects were exaggerated by about 40%
compared to trials of higher quality.
Quality assessment of trials in periodontology and
implantology
Two recent studies have investigated the methodo-
logical quality of RCTs in periodontology and
implantology as assessed by their publications. Both
studies targeted RCTs for investigation due to the
importance of the RCT in providing evidence for
the effect of interventions and also because of the
empiric data indicating the effect of key domains of
methodology on bias.
18
Montenegro et al. (18) conducted a systematic
review of the quality of RCTs of periodontal ther-
apy, published in Journal of Periodontology, Journal
of Clinical Periodontology or Journal of Periodontal
research over a 3-year period from 1996 to 1998.
From the electronic search, 283 papers were poss-
ibly relevant and 177 studies met the inclusion
criteria of being an RCT, performed on humans and
for which a full text article was available. Screening
and data abstraction were performed independently
and in duplicate to minimise error and bias. The
evaluation was not performed blind to author
affiliation identity of the RCTs as the evidence
suggests that this has a minimal impact on out-
come (16). In view of the empirical data described
above, the quality components chosen were those
demonstrated to be important for protection from
bias: adequacy of method of generation of the
random sequence, adequacy of method of con-
cealing the allocation sequence from the patient
recruitment, examiner blinding (where it was
judged possible to achieve), and handling of losses
and withdrawals.
The results indicated that 29/177 (17%) of RCTs
employed a clearly adequate method of generating
the random number sequence, and that 12/177 (7%)
of studies described adequate allocation conceal-
ment (Fig. 3). Furthermore, where examiner blinding
was possible, 97/177 (55%) of studies reported an
adequate method. Clear accounting for study sub-
jects was present in 100/177 (55%) of reports. Since
the study was conducted on trial reports, it is not
clear how much of the inadequacy was due to
incomplete reporting rather than inadequate study
methods. If the data do reflect study conduct, then
bias and exaggeration of the effect of the test inter-
ventions could be a problem with some trials in
periodontology.
Similar results were found when investigating
RCTs of oral implants (6). This study searched for
RCTs up to the end of 1999 in multiple databases.
Seventy-four publications were located and 43 RCTs
were quality assessed as many studies were pre-
sented in multiple publications. Although the
methods and criteria were a little different for this
study compared with the quality appraisal of peri-
odontal studies, the results are broadly comparable.
A clearly adequate method of randomization/con-
cealment of allocation was present in 1/43 (2%)
papers. Blinding was described in 12/43 (28%)
studies and the reasons for withdrawals and losses
to follow-up were specified in 33/43 (77%) of
reports.
 Table 3. Study designs and the types of questions they address
Definition of study design
Experimental studies
Randomized-controlled trial: parallel group design – a group of
participants (or other unit of analysis, e.g. teeth) is randomized into
different treatment groups. These groups are followed up for the
outcomes of interest
Randomized-controlled trial: split-mouth design – each patient is
his/her own control. A pair of similar teeth, or groups of teeth
(quadrants), may be selected and randomly allocated to different
treatment groups.
Non-randomized controlled trial – allocation of participants under
the control of the investigator, but the method falls short of
genuine randomization.
Observational studies
Cohort: a longitudinal study, identifying groups of participants
according to their exposure/intervention status. Groups are followed
forward in time to measure the development of different outcomes.
Case-Control: a study that identifies groups of participants
according to their disease/outcome status. Groups are investigated/
questioned to determine their exposure status
Cross-sectional: a study (survey) undertaken on a defined population
at a single point in time (snap-shot). Subjects are observed on just
one occasion and are not followed up.
19
Used for (examples given in italics)
Evaluating the effectiveness of an intervention
Randomized controlled trial comparing the effectiveness of surgical therapy
and nonsurgical debridement.
Controlled trial comparing two methods of treating periodontal intrabony
defects using pairs of sites where the LHS is always group A and the RHS group B.
Measuring the incidence of a disease; looking at the causes of disease;
determining prognosis.
Cohort study looking at the progress of periodontitis over time and relating this to
external factors such as smoking or plaque.
Identifying potential risk factors for a disease; looking at the possible causes of
disease.
Case-control study looking at the prevalence of periodontitis and relating this to
factors such as genetic markers.
Measuring the prevalence of a disease or risk factor in a defined population at a
specific time.
A cross-sectional study to determine the current periodontal treatment needs in
a specific population.
Evidence-based periodontology
Needleman et al.
Table 4. Center for Evidence-Based Medicine hier-
archy of evidence for studies on therapy, prevention,
etiology or harm (http://www.cebm.net/levels_of_
evidence.asp#levels)
Level Type of evidence
Ia Systematic review (with homogeneity*) of
randomized controlled trials (RCT).
1b Individual RCT (with narrow confidence
interval, see notes below).
2a Systematic review (with homogeneity*) of
cohort studies.
2b Individual cohort study (including low quality
RCT; e.g. < 80% follow-up).
2c ÔOutcomesÕ research; Ecological studies.
3a Systematic review (with homogeneity*) of
case-control studies.
3b Individual case-control study.
4 Case-series (and poor quality cohort and
case-control studies§)
5 Expert opinion without explicit critical
appraisal, or based on physiology, bench
research or Ôfirst principles.
Users can add a minus-sign Ô–Õ to denote the level of that
fails to provide a conclusive answer because of:
EITHER a single result with a wide Confidence Interval
(such that, for example, an absolute risk reduction in an
RCT is not statistically significant but whose confidence
intervals fail to exclude clinically important benefit or
harm);
OR a Systematic Review with troublesome (and statisti-
cally significant) heterogeneity.
Such evidence is inconclusive.
*A systematic review that is free of worrisome variations
(heterogeneity) in the directions and degrees of results
between individual studies. Not all systematic reviews
with statistically significant heterogeneity need be wor-
risome, and not all worrisome heterogeneity need be
statistically significant. As noted above, studies display-
ing worrisome heterogeneity should be tagged with a Ô–Õ
at the end of their designated level.
§ Poor quality cohort study: one that failed to clearly de-
fine comparison groups and/or failed to measure expo-
sures and outcomes in the same (preferably blinded),
objective way in both exposed and nonexposed individ-
uals and/or failed to identify or appropriately control
known confounders and/or failed to carry out a suffi-
ciently long and complete follow-up of patients. Poor
quality case-control study: one that failed to clearly define
comparison groups and/or failed to measure exposures
and outcomes in the same (preferably blinded), objective
way in both cases and controls and/or failed to identify or
appropriately control known confounders.
Improving the quality of reporting of
clinical research in periodontology
The adequacy of reporting of clinical research is
crucial if the reader is to evaluate the quality and
20
60%
50%
40%
30%
20%
10%
0%
Randomization Allocation Examiner blinding Accounting for all
method concealment subjects
Fig. 3. Quality of reporting of randomized controlled tri-
als in periodontology (18). Percentage of studies with
adequate method.
possible impact of studies. The importance of sev-
eral of the quality issues that we have described has
not been thoroughly appreciated until relatively
recently. Therefore, it is unfair to judge the past
from the standpoint of current knowledge. In
addition, the pressure on page numbers in paper-
based journals can restrict detail. Hopefully this
aspect will be alleviated by initiatives in electronic
publication.
Guidelines are available to help the publication of
clinical research. These guidelines are well accepted
by high impact biomedical journals and offer guid-
ance not only to authors but also to editors and
reviewers. These guidelines include CONSORT
(Consolidated Standards of Reporting Trials) for
reporting randomized controlled trials and STARD
(Standards for Reporting of Diagnostic Accuracy) for
reporting studies on diagnostic tests (http://consort-
statement.org/). In addition, three guidelines for
reporting systematic reviews are available: QUOROM
(Quality of Reporting of Meta-analyses) (http://
consort-statement.org/), MOOSE (Meta-analysis Of
Observational Studies in Epidemiology) (27), and
QUADAS (Quality Assessment of studies of Diagnos-
tic Accuracy included in Systematic reviews) (29). For
clarification, it should be remembered that system-
atic reviews are termed meta-analyses by some in
North America, whereas the term meta-analysis is
usually reserved only for the statistical combining of
data which may or may not be part of a systematic
review.
The format of these guidelines is similar. Each
presents a checklist of items for incorporation into
the research report. The selection of items is evi-
dence-based as far as possible and otherwise derived
 Evidence-based periodontology

(a)
PAPER Item Description Reported
SECTION on
And topic Page #
TITLE & 1 How participants were allocated to interventions
ABSTRACT (e.g., "random allocation", "randomized", or
"randomly assigned").
INTRODUCTION 2 Scientific background and explanation of rationale.
Background
METHODS 3 Eligibility criteria for participants and the settings
Participants and locations where the data were collected.
Interventions 4 Precise details of the interventions intended for
each group and how and when they were actually
administered.
Objectives 5 Specific objectives and hypotheses.
Outcomes 6 Clearly defined primary and secondary outcome
measures and, when applicable, any methods
used to enhance the quality of measurements
(e.g., multiple observations, training of assessors).
Sample size 7 How sample size was determined and, when
applicable, explanation of any interim analyses
and stopping rules.
Randomization -- 8 Method used to generate the random allocation
sequence sequence, including details of any restrictions
generation (e.g., blocking, stratification).
Randomization -- 9 Method used to implement the random allocation
allocation sequence (e.g., numbered containers or central
concealment telephone), clarifying whether the sequence was
concealed until interventions were assigned.
Randomization -- 10 Who generated the allocation sequence, who
Implementation enrolled participants, and who assigned
participants to their groups.
Blinding 11 Whether or not participants, those administering
(masking) the interventions, and those assessing the
outcomes were blinded to group assignment.
When relevant, how the success of blinding was
evaluated.
Statistical 12 Statistical methods used to compare groups for
methods primary outcome(s). Methods for additional
analyses, such as subgroup analyses and
adjusted analyses.
RESULTS 13 Flow of participants through each stage (a
diagram is strongly recommended). Specifically,
Participant flow for each group report the numbers of participants
randomly assigned, receiving intended treatment,
completing the study protocol, and analyzed for
the primary outcome. Describe protocol deviations
from study as planned, together with reasons.
Recruitment 14 Dates defining the periods of recruitment and
follow-up.
Baseline data 15 Baseline demographic and clinical characteristics
of each group.
Numbers 16 Number of participants (denominator) in each
analyzed group included in each analysis and whether the
analysis was by ‘intention-to-treat’. State the
results in absolute numbers when feasible (e.g.,
10/20, not 50%).
Outcomes and 17 For each primary and secondary outcome, a
estimation summary of results for each group, and the
estimated effect size and its precision (e.g. 95%
confidence interval).
Ancillary 18 Address multiplicity by reporting any other
analyses analyses performed, including subgroup analyses
and adjusted analyses, indicating those pre-
specified and those exploratory.
Adverse events 19 All important adverse events or side effects in
each intervention group.
DISCUSSION 20 Interpretation of the results, taking into account
Interpretation study hypotheses, sources of potential bias or
imprecision and the dangers associated with
multiplicity of analyses and outcomes.
Generalisability 21 Generalisability (external validity) of the trial
findings.
Overall evidence 22 General interpretation of the results in the context
of current evidence.
Fig. 4. a) CONSORT Checklist of items to include when reporting a randomized trial. b) CONSORT Flow chart. Available
from: http://www.consort-statement.org/.

21
Needleman et al.
(b) Assessed for
eligibility (n= ... )
Enrollment
Randomized (n = ... )
Allocated to intervention
Allocation
(n = ... )
Received allocated
intervention (n = ... )
Did not receive allocated
intervention
(give reasons) (n = ... )
Follow up
Lost to follow up (n = ... )
(give reasons)
Discontinued intervention
(n = ... ) (give reasons)
Analysis
Analysed (n = ... )
Excluded from analysis
(give reasons) (n = ... )
Fig. 4. Continued.
by a Delphi approach to consensus. In addition to the
checklist, a chart is used to illustrate the flow of pa-
tients through the study. The checklist and chart for
CONSORT are illustrated in Fig. 4a, b. The checklist
should accompany the manuscript in its journal
submission but not be part of the final paper. The
intention with the chart, however, is that it should be
published as part of the paper. Whilst the checklist
has numerous items, each can be concisely ad-
dressed and is unlikely to be the main cause for
excessive length of a publication.
At the time of writing, oral health journals that have
adopted CONSORT as editorial policy and their dates
of adoption are: British Dental Journal (1999), Journal
of Orthodontics (2000), International Journal of End-
odontics (2003) and Journal of Dental Research (2004).
The British Dental Journal is the only one that has
adopted QUOROM (2002).
What should be appraised?
Given that some evidence is ÔbetterÕ than other evi-
dence, it seems reasonable to place greater emphasis
on ÔgoodÕ than on ÔpoorÕ quality evidence when mak-
ing clinical decisions. The problem arises as to how
exactly we decide what constitutes good quality evi-
dence. This process is critical appraisal. The validity of
published evidence is potentially affected by the
quality of every stage of the experimental process
from aims and objectives, through design, execution,
analysis, interpretation, and finally publication. Al-
22
Excluded (n = ... )
Not meeting
inclusion criteria
(n = ... )
Refused to participate
(n = ... )
Other reasons (n = ... )
Allocated to intervention
(n = ... )
Received allocated
intervention (n = ... )
Did not receive allocated
intervention
(give reasons) (n = ... )
Lost to follow up (n = ... )
(give reasons)
Discontinued intervention
(n = ... ) (give reasons)
Analysed (n = ... )
Excluded from analysis
(give reasons) (n = ... )
though deliberate deception is always a possibility,
the majority of problems that arise are in fact unin-
tentional. Most methodological errors may be classi-
fied as being the result of bias, confounding, or
chance. Therefore, for the purpose of this chapter,
quality will be discussed in relation to these meth-
odological issues. Other aspects of study conduct may
well be critical to the validity of a study but will not be
considered in this chapter as they will be specific for a
particular study. Such factors could include how well
treatment or supportive maintenance was provided.
Bias
Bias is a systematic error. It leads to results which are
consistently wrong in one or another direction. Bias
leads to an incorrect estimate of the effect of a risk
factor or exposure (e.g. smoking) on the development
of a disease or outcome of interest (e.g. response to
periodontal therapy). The observed effect will be
either above or below the true value. Many types of
bias have been identified, however, the main types
relate to:
• how subjects were selected for inclusion in a study
(selection bias);
• provision of care (performance bias);
• assessment of outcomes (detection/measurement
bias);
• occurrence and handling of patient attrition
(attrition bias).
Selection bias occurs when there is a systematic
difference between the characteristics of the subjects
selected for a study and the characteristics of those
who were not. For instance, selection bias will often
occur with volunteers (self-selection bias). People
who volunteer to participate in a study tend to be
different from the general population. Similarly, it is
important to consider whether people might have
selectively withdrawn from the study before its
completion (attrition bias). They may have with-
drawn at random, or because of some factor related
to the study, e.g. the treatment they were receiving
was ineffective or uncomfortable in comparison with
the alternative treatment. It is necessary to decide
whether the results of the investigation were likely to
have been compromised if one group of subjects had,
on average, a shorter follow-up as a result of more
people dropping-out.
The avoidance of selection bias is a major concern
in the design of case-control studies. In this type of
study it is essential to ensure that controls are rep-
resentative of the population from which the cases
originated. Suppose a group of researchers is con-

ducting a case-control study to assess the effect of
cigarette smoking on the development of aggressive
periodontitis. In our hypothetical example, cases are
patients referred to a dental hospital with aggressive
periodontitis and controls are non-dental patients
admitted to a nearby hospital with chronic bronchi-
tis. A standard questionnaire is administered to both
cases and controls that includes questions on lifetime
smoking habits. The researchers may find no evi-
dence from this study of an association between
cigarette smoking and aggressive periodontitis. Can
we accept this conclusion? The problem with this
study is that the choice of controls is biased, as the
prevalence of smoking among patients admitted with
chronic bronchitis is likely to be much higher than
among the general population resident in the catch-
ment area of the hospitals from which the cases and
controls originated. Consequently, the strength of
the association between smoking and aggressive
periodontitis will most likely be under-estimated in
this study.
Randomized controlled trials are less likely to be
affected by selection bias if the randomization is
properly conducted. Randomization is a two-stage
process. The first stage is the generation of a true
random sequence. Typically, this is achieved through
computer software or a random number table. Whilst
a tossed coin is theoretically acceptable, we suggest
using a method that can be audited later on for the
purposes of quality assessment, such as a computer
generated list.
The second stage of randomization is less well
understood or carried out. Once the random
sequence is generated, it must be concealed from
those selecting patients for a study until the indi-
vidual has been recruited into the trial. If not,
despite the sequence being random, the researcher
will be aware of whether the patient will be
entered into test or control group. This knowledge
provides the opportunity for selection bias, whether
intentional or not. This second stage of randomi-
zation is termed allocation concealment (see Fig. 5
for an outline of this process). The key question to
ask is, Was the recruitment of patients into a trial
entirely unpredictable with respect to test or con-
trol group?
Performance bias occurs when different study
groups do not receive therapy in the same fashion or
to the same standard. This may occur if the people
providing the therapy are aware of which groups the
participants have been allocated to. Depending on
the nature of the investigation, it may be either a
relatively simple or a difficult task to ensure that
Evidence-based periodontology
Step 1
Generate a true random sequence
- computer generated is best
- tossed coin is acceptable
Step 2
Allocation concealment
- conceal the sequence for study recruitment
- sequentially numbered truly opaque envelopes/drug
containers, centrally kept randomization accessed by
telephone, e.g. pharmacy
Fig. 5. The two stages of randomization.
therapists remain masked (blinded) to the treatment
allocation. The use of placebo, where appropriate,
greatly facilitates masking; placebo controlled trials
are usually easy to organize in such a way as to leave
the therapist masked to the treatment allocation.
However, if the interventions to be compared are
quite dissimilar in their delivery (e.g. surgical vs.
nonsurgical therapy), then masking becomes con-
siderably more challenging. Under these circum-
stances the best available option might be to ensure
that the therapist remains masked until the last
possible moment to ensure that all therapy prior to
that point has been undertaken as even-handedly as
possible. So, for example, in a split-mouth study
comparing scaling and root planning vs. scaling and
root planning plus an adjunctive locally delivered
antimicrobial, it might be possible to complete the
mechanical therapy at all appropriate sites prior to
the therapist finding out which particular sites are to
receive the adjunctive therapy. However, the risk of
carry-over effects of the local antibiotic affecting the
scaling and root planning-only sites should not be
ignored.
Measurement (information) bias occurs when the
measurements of exposure and/or outcome are not
valid (i.e. they do not measure correctly what they
are supposed to measure). Errors in measurement
may be introduced by the observer (observer bias),
by the study individual (responder bias), or by the
instruments (instrument bias) used to make the
measurements (e.g. a badly designed questionnaire).
As a result of measurement errors, study partici-
pants will be misclassified in relation to their
exposure and/or outcome status. This misclassifi-
cation has particularly serious implications if the
errors in exposure measurement are related to the
participant’s outcome status. Ideally the person
undertaking the examination should be blinded
23
Needleman et al.
(masked). This is more important for measures in
which there is the potential for subjectivity (e.g.
pocket depth, colour change) than for objective
measures (e.g. tooth loss).
Bias is a consequence of defects in the design or
execution of a study. Bias cannot be controlled dur-
ing the statistical analysis of the data and cannot be
eliminated by increasing the size of the study.
Publication bias
Publication bias refers to the greater likelihood of
publication of studies with positive results than those
with neutral or negative results (5). The risk with this
type of bias is that interventions appear to perform
better than they will in clinical practice. For instance,
publication bias might mean that although several
studies were published and the data available to be
included in a meta- analysis, a larger number of
studies were actually conducted but not published.
Of these ÔmissingÕ studies, some may show no dif-
ference between the intervention group and the
control group, or even the control group performing
better. If these additional studies had been published,
the results of the meta-analysis could have been
different. Therefore, the sample of published studies
is a biased sample and does not represent the com-
plete population of all research on this question.
Graphic and formal statistical tests are available to
investigate publication bias but need approximately
10 or more studies to have adequate power. Figure 6
illustrates this situation using hypothetical data.
Confounding
Confounding is a term that describes the situation
where an estimate of the association between an
exposure and the disease is mixed up with the real
effect of another exposure on the same disease, the
two exposures being correlated. It is a difficult con-
cept that may be illustrated with the help of the fol-
lowing example. Suppose we find that coffee drinkers
have a poorer response to periodontal therapy than
noncoffee drinkers. Does it mean that coffee drinking
affects the response to therapy? The problem here is
that there is an alternative explanation. Smoking may
be an independent risk factor for poor treatment
response and it is possible that people who drink
coffee are more likely to smoke than those who do
not. Perhaps the observed association is actually due
to smoking habits, not coffee drinking (Fig. 7).
Age and sex are the most common confounding
variables in health-related studies because these two
variables are not only associated with most exposures
24
we are interested in, such as diet, smoking habits,
health beliefs, etc., but are also independent risk
factors for many diseases.
Confounding can be dealt with at the design stage
of an investigation by:
• Randomization – By randomly allocating subjects
to study groups it is hoped that confounders are
distributed equally between the groups. This is
usually the most effective way of minimizing the
problem of confounding. If randomization is
done properly, it has the advantage that it con-
trols for both known and unknown confounders
provided the sample size is sufficiently large.
• Restriction – This limits participation in a study to
specific groups which are similar to each other
with respect to the confounder (e.g. if smoking is
likely to be a confounder then only nonsmokers
will be included in the study).
• Matching – This selects comparison groups with
similar backgrounds (e.g. nonsmokers are matched
with other nonsmokers, while smokers are mat-
ched with other smokers).
Confounding can also be controlled for in the ana-
lysis by:
• Stratification – Here the strength of the association
is measured separately in each well-defined sub-
group (e.g. in the smokers and the nonsmokers
separately). The results are then pooled together
using basic statistical techniques to obtain an
overall summary measure of the association Ôadjus-
tedÕ or ÔcontrolledÕ for the effects of the confounder.
• Statistical modelling – These are more sophisti-
cated mathematical techniques that can simulta-
neously take into consideration the effects of
several possible confounders that have been
recorded by the investigators.
It is only possible to control for confounders in the
analysis if data on them were collected during the
study. Obviously, the extent to which confounding
can be controlled for will depend on the accuracy of
these data. However, in some situations it may be
virtually impossible to gain complete and accurate
information on confounders. Some confounders
may be so difficult to assess that even attempting to
adjust for them in a statistical model will not com-
pletely control for their effect. For example, Hujoel
et al. have argued that the confounding effect of
smoking is virtually impossible to measure with
sufficient precision in studies that attempt to look at
the association between periodontal diseases and
systemic health and that such studies may only
provide valid results if they are restricted to non-
smokers (10).
 Evidence-based periodontology

(a) (b)
50
study 1
study 2 45
study 3
study 4 40
study 5
35
study 6

1/standard error
study 7 30
study 8
study 9 25
study 10
20
study 11
study 12 15
study 13
study 14 10
study 15
5

Combined 0
–2 –1 0 1 –.4 –.2 0 .2 .4
mean mean difference
←Favours control
Favours intervention→

(c) (d)
study 1
50
study 2
study 3 45
study 1
study 2 40
study 4
study 3 35
study 5
1/standard error
study 6 30
study 4
study 7 25
study 8
study 9
study 5 20
study 10
study 11 15
study 12
study 13 10
study 14
study 15 5

Combined 0
–2 –1 0 1 –.4 –.2 0 .2 .4
mean mean difference
←Favours control Favours
intervention→

Fig. 6. Illustration of publication bias. a) Forest plot
showing the results of a meta-analysis of 15 studies. A very
small improvement is indicated in favour of the test
intervention since the diamond shape (representing the
95% confidence interval for the pooled result) does not
cross the zero (‘no-effect’) line. b) Funnel plot for these
studies. In the absence of publication bias, it is anticipated
that the plot would form a funnel shape. As no funnel
shape was produced, this indicates the possibility of
missing studies. These studies would be expected to pro-
duce data points that lie somewhere within the shaded
area. c) Another Forest plot, including the data from these
extra, previously ÔmissingÕ studies. The 95% confidence
interval for the pooled result now crosses the line of no
effect, indicating no evidence that the intervention is any
more effective than the control.

spurious association samples drawn from that population. The extent to

Coffee drinking Poor treatment response
association
Smoking habits
(confounder)
Fig. 7. An example of confounding.
Chance
Chance (sampling error) plays a role in most studies
of humans since it is rarely if ever possible to include
an entire population in an investigation. We therefore
attempt to infer information about the population on
the basis of information obtained from representative
which the sample results reflect the likely result in the
population is assessed by performing statistical sig-
risk factor
nificance tests and, more importantly, by calculating
confidence intervals. A proper discussion of these
methods is beyond the scope of this chapter, but in
general, studies with small sample sizes will be more
prone to sampling error and will provide less robust
estimates than studies with larger samples.
Interpretation
It is worth noting that authors may also fail to
interpret their experimental results correctly. So,

25
Needleman et al.

even if the study has been well conducted and
appropriately analyzed, there is still the potential to
draw incorrect conclusions from the results.
How to critically appraise?
When appraising quality it is necessary to consider
those factors that may affect the outcome of a study.
These will inevitably vary according to both the topic
of the original research and the study designs
employed, so it is not possible to devise a single
system that will be appropriate for every occasion. As
a general rule, the aforementioned domains of bias,
confounding and chance will all have to be ap-
praised.
Some reviewers have attempted to devise com-
posite scales that give scores for the various quality
domains (11). These scores are then summed to an
overall summary measure for the study as a whole.
There are problems with this approach. Many
quality items may not be based on empirical evi-
dence and the scores attached to each item will
inevitably be subjective. It is also doubtful whether a
single summary score is likely to provide an
adequate overall assessment of the quality of a
particular study. When different composite scales
are applied to the same studies, differing scores and
rankings may occur. For these reasons, composite
scales have largely gone out of favour. An alternative
approach is to appraise each quality component
separately (12).

Table 5. Quality assessment checklist for randomized controlled trials in periodontology used by Montenegro
et al. (18)
Item
Randomization
Allocation concealment
Blinding of patient, caregiver,
and examiner were considered
separately
Withdrawals and drop outs
Classification
Adequate
Unclear
Inadequate
Adequate
Unclear
Inadequate
Recorded as adequate, inadequate,
unclear, or for examiner blinding,
not applicable if the study
design precluded the
possibility of blinding.
Were all patients who entered the
trial properly accounted for at the end?
Where dropouts occurred, the use of
analyses to allow for losses (such as
intention to treat) was noted.
Definition
If generated by random number
table (computer generated or not);
tossed coin; and shuffled cards.
Study refers to randomization but
either does not adequately explain
the method or no method was reported.
Methods include alternate assignment,
hospital number, and odd/even birth date.
Methods included central randomization
(e.g. by telephone to a pharmacy or trial
office), pharmacy sequentially numbered/
coded containers, and sequentially
numbered opaque envelopes.
If the study referred to allocation
concealment but either did not
adequately explain the method or
no method was reported.
Involved methods where randomization
could not be concealed, such as alternate
assignment, hospital number,
and odd/even birth date.

26

For rigorous systematic reviews, independent
reviewers usually undertake quality appraisal in
duplicate and checklists are frequently employed for
this purpose. Two such checklists that have been
used previously are reproduced here as examples
(Tables 5 and 6) (7, 18). These checklists are based on
a combination of factors that have been shown
empirically to affect quality (such as allocation
Table 6. Quality assessment checklist for systematic
reviews in dentistry used by Glenny et al. (7)
Question (possible categories)
A. Did review address a focused question?
(yes, no, can’t tell)
B. Did authors look for appropriate papers?
(yes, no, can’t tell)
C. Do you think authors attempted to identify all
relevant studies?
(yes, no, can’t tell)
D. Search for published and unpublished literature
(yes, no, can’t tell)
E. Were all languages considered?
(yes, no, can’t tell)
F. Was any hand searching carried out?
(yes, no, can’t tell)
G. Was it stated that the inclusion criteria were
carried out by at least two reviewers?
(yes, no, can’t tell)
H. Did reviewers attempt to assess the quality of the
included studies?
(yes, no)
I. If so did they include this in the analysis?
(yes, no, can’t tell, not applicable)
J. Was it stated that the quality assessment was
carried out by at least two reviewers?
(yes, no, not applicable)
K. Are the results given in a narrative or pooled
statistical analysis?
(narrative, pooled, not applicable)
L. If the results have been combined was it
reasonable to do so?
(yes, no, can’t tell, not applicable)
M. Are the results clearly displayed?
(yes, no, not applicable)
N. Was an assessment of heterogeneity made and
reasons for variation discussed?
(yes, no, not applicable)
O. Were results of review interpreted appropriately?
(yes, no, can’t tell, not applicable)
Evidence-based periodontology
concealment) and also topic specific factors deemed
important by the reviewers. Other checklists cover a
broad range of types of research and can be
found on the excellent Critical Appraisal Skills
Programme website (http://www.phru.nhs.uk/casp/
appraisa.htm).
The use of checklists with objective criteria helps
to safeguard the quality of the quality appraisal
process itself. The process of devising the check-
list helps to ensure that all relevant quality issues
are included in the assessment. Written, piloted
checklists reduce, but can never completely elim-
inate individual subjectivity in decisions. Having a
written list means that it is more likely that the
quality assessors will be both consistent and
repeatable.
The results of the quality appraisal are used to
assess the value of the evidence and to aid clinicians
and reviewers in their efforts to place the evidence
into context. This might be a part of the formal pro-
cess of undertaking a systematic review or the
informal act of reading and assessing recently pub-
lished literature as part of everyday periodontal
practice.
Conclusions
The principles of evidence-based healthcare provide
structure and guidance to facilitate the highest levels
of patient care. There are numerous components to
evidence-based periodontology including the pro-
duction of best available evidence, the critical
appraisal and interpretation of the evidence, the
communication and discussion of the evidence to
individuals seeking care and the integration of the
evidence with clinical skills and patient values. This
volume of Periodontology 2000 is mainly concerned
with the first component, i.e. the generation of best
evidence and, alone, is not enough to practise evi-
dence-based healthcare. However, an understanding
of the principles should help to underpin the latter
aspects. Evidence-based healthcare is not an easier
approach to patient management, but should provide
both clinicians and patients with greater confidence
and trust in their mutual relationship.
References
1. Alderson P, Green S, Higgins JPT, eds. Cochrane Reviewers’
Handbook 4.2.2 [updated. ]. In: The Cochrane Library, Issue 1.
Chichester: John Wiley & Sons, 2004.
27
Needleman et al.
2. American Academy of Periodontology. Proceedings of the
1996 World Workshop in Periodontics. Lansdowne, Vir-
ginia, July 13-17, 1996. Ann Periodontol 1996: 1: 1–947.
3. Antczak-Bouckoms A, Tang J, Chalmers TC. Quality
assessment of randomized controlled trials in dental
research. J Periodontal Res 1986: 21: 305–314.
4. EBM Working Group. Evidence-based medicine. JAMA
1992: 268: 2420–2425.
5. Egger M, Davey-Smith G, Altman DG. Systematic Reviews in
Health Care, 2nd edn, London: BMJ Books, 2001.
6. Esposito M, Coulthard P, Worthington H, Jokstad A. Quality
assessment of randomized controlled trials of oral
implants. Int J Oral Maxillofac Implants 2001: 16: 783–792.
7. Glenny AM, Worthington H, Esposito M, Coulthard P. The
assessment of systematic reviews in dentistry. Eur J Oral
Sci 2003: 111: 85–92.
8. Gray JAM. Evidence-based Healthcare. Edinburgh: Churchill
Livingstone, 1997.
9. Herrera D, Sanz M, Jepsen SJ, Needleman IG, Roldan S. A
systematic review on the effect of systemic antimicrobials
as an adjunct to scaling and root planning in perio-
dontitis patients. J Clin Periodontol 2002: 29 (Suppl 3):
136–159.
10. Hujoel PP, Drangsholt M, DeRouen TA. Periodontitis –
systemic disease associations in the presence of smo-
king – causal or coincidental? Periodontol 2000 2002: 30:
51–60.
11. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ,
Gavaghan DJ, et al. Assessing the quality of reports of
randomized clinical trials: is blinding necessary? Control
Clin Trials 1996: 17: 1–12.
12. Juni P, Altman DG, Egger M. Assessing the quality of con-
trolled clinical trials. Br Med J 2001: 323: 42–46.
13. Khan KS, ter Riet G, Glanville J, Sowden AJ, Kleijnen J.
Undertaking systematic reviews of research on effective-
ness. Http://www.York.Ac.Uk/Institute/crd/report4.htm [2nd],
2001.
14. Labriola A, Needleman I, Moles DR. Systematic review of
the effect of smoking on nonsurgical periodontal therapy.
Periodontol 2000 2005: 37: 124–137.
15. Mathews JR. Quantification and the Quest for Medical
Certainty. Princeton: Princeton University Press, 1995.
16. Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A,
et al. Assessing the quality of reports of randomized trials:
implications for the conduct of meta-analyses. Health
Technol Assess 1999: 3: 1–98.
28
17. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M,
et al. Does quality of reports of randomized trials affect
estimates of intervention efficacy reported in meta-analy-
ses? Lancet 1998: 352: 609–613.
18. Montenegro R, Needleman I, Moles D, Tonetti M. Quality
of RCTs in periodontology – a systematic review. J Dent Res
2002: 81: 866–870.
19. Needleman IG. A guide to systematic reviews. J Clin
Periodontol 2002: 29 (Suppl. 3): 6–9.
20. Needleman IG. Introduction to evidence based dentistry.
In: Clarkson, J, Harrison, JE, Ismail, AI, Needleman, IG,
Worthington, H, eds. Evidence Based Dentistry for Effective
Practice. London: Martin Dunitz, 2003: 1–17.
21. Needleman IG, Giedrys-Leeper E, Tucker RJ, Worthington
HV. Guided tissue regeneration for periodontal infra-bony
defects (Cochrane Review). The Cochrane Library. Oxford:
Update Software. http://www.update-software.com/clibhome/
clib.htm, 2001.
22. Needleman IG, Giedrys-Leeper E, Tucker RJ, Worthington
HV. Guided tissue regeneration for infrabony defects –
A systematic review. J Periodontal Res 2002: 37: 380–
388.
23. Rangachari PK. Evidence-based medicine: old French wine
with a new Canadian label? J R Soc Med 1997: 90: 280–284.
24. Richards D. Not all evidence is created equal – so what is
good evidence? Evid Based Dent 2003: 4: 17–18.
25. Roccuzzo M, Bunio M, Needleman I, Sanz M. Periodontal
plastic surgery for treatment of localized gingival reces-
sions – A systematic review. J Clin Periodontol 2002: 29
(Suppl 3): 178–194.
26. Schultz KF, Chalmers I, Hayes RJ, Altman D. Empirical
evidence of bias: Dimensions of methodological quality
associated with estimates of treatment effects in controlled
trials. JAMA 1995: 273: 408–412.
27. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD,
Rennie D, et al. Meta-analysis of observational studies in
epidemiology: a proposal for reporting. Meta-analysis of
Observational Studies in Epidemiology (MOOSE) group.
JAMA 2000: 283: 2008–2012.
28. Swales J. The troublesome search for evidence: three
cultures in need of integration. J R Soc Med 2000: 93: 402–
407.
29. Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J.
The development of QUADAS: a tool for the quality
assessment of studies of diagnostic accuracy included in
systematic reviews. BMC Med Res Methodol 2003: 3: 25.