Journal of Clinical Epidemiology 127 (2020) 59e68

ORIGINAL ARTICLE

Quality assessment of prevalence studies: a systematic review

Celina Borges Migliavacaa,b,*, Cinara Steinb, Ver^
onica Colpanib, Zachary Munnc,
Maicon Falavignaa,b, Prevalence Estimates Reviews e Systematic Review Methodology Group
(PERSyst)
a
Programa de P os-Graduaç~ao em Epidemiologia, Universidade Federal do Rio Grande do Sul, Rua Ramiro Barcelos, 2400, CEP 90035-003, Santa Cecılia,
Porto Alegre, Rio Grande do Sul, Brazil
b
Hospital Moinhos de Vento, Porto Alegre, Rua Ramiro Barcelos, 910, CEP 90035-001, Floresta, Porto Alegre, Rio Grande do Sul, Brazil
c
Joanna Briggs Institute, Faculty of Health Sciences, University of Adelaide, Adelaide, Australia
Accepted 30 June 2020; Published online 15 July 2020

Abstract
Objectives: The objective of the study is to identify items and domains applicable for the quality assessment of prevalence studies.
Study Design and Setting: We searched databases and the gray literature to identify tools or guides about the quality assessment of
prevalence studies. After study selection, we abstracted questions applicable for prevalence studies and classified into at least one of the
following domains: ‘population and setting’, ‘condition measurement’, ‘statistics’, and ‘other’. PROSPERO registration:CRD42018088437.
Results: We included 30 tools: eight (26.7%) specifically designed to appraise prevalence studies and 22 (73.3%) adaptable for this
purpose. We identified 12 unique items in the domain ‘‘population and setting’’, 16 in the domain ‘‘condition measurement’’, and 14 in
the domain ‘‘statistics’’. Of those, 25 (59.5%) were identified in the eight specific tools. Regarding the domain ‘‘other’’, we identified
77 unique items, mainly related to manuscript writing and reporting (n 5 48, 62.3%); of those, 24 (31.2%) were identified in the eight
specific tools and 53 (68.8%) in the additional 22 nonspecific tools.
Conclusion: We provide a comprehensive set of items classified by domains that can guide the appraisal of prevalence studies, con-
duction of primary prevalence studies, and update or development of tools to evaluate prevalence studies. Ó 2020 The Authors. Published
by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Keywords: Prevalence; Cross-sectional studies; Bias; Methodological quality; Quality assessment

1. Background are also useful to evaluate the impact of health interventions

Prevalence is an epidemiological measurement that repre-
sents the proportion of the population affected by certain
condition [1]. Because they reflect the importance of
different diseases for the society, prevalence estimates are
of great importance for health-related decision-making. For
instance, these estimates are used to assess the burden of
different conditions, helping in the definition of priorities
for interventions, guideline development, and research. They
Competing interests: Z.M. is director of the Transfer Science program
of the Joanna Briggs Institute (JBI). The authors have no other competing
interests to declare.
Declaration of interests: The authors declare the following financial in-
terests/personal relationships which may be considered as potential
competing interests: Z.M. is director of the Transfer Science program of
the Joanna Briggs Institute (JBI). The authors have no other competing in-
terests to declare.
* Corresponding author. Tel.: þ55 51 997107769; fax: þ55 51
35378347.
E-mail address: celinabm7@gmail.com (C.B. Migliavaca).
because they show changes and trends over time in condi-
tions of interest. For health technology assessments, preva-
lence data are also used in the estimation of costs, being an
essential parameter in economic models [2e5]. The number
of systematic reviews of prevalence indexed in Medline has
increased more than ten-fold in the last decade [6].
Despite the importance of prevalence studies, the risk of
bias assessment of this type of study is heterogeneous, usu-
ally inappropriate and often neglected. A systematic review
conducted in 2010 identified five tools specifically developed
to appraise prevalence studies, and the authors of that review
concluded that included tools presented several limitations
specially regarding applicability and lack of consensus about
which domains should be assessed [7]. In comparison, there
are standard, recommended, and widely used tools for other
study designs, such as RoB 2.0 for randomized clinical trials,
ROBINS-I for observational studies, and QUADAS-2 for
diagnostic studies [8e10].

https://doi.org/10.1016/j.jclinepi.2020.06.039
0895-4356/Ó 2020 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/
licenses/by-nc-nd/4.0/).
60 C.B. Migliavaca et al. / Journal of Clinical Epidemiology 127 (2020) 59e68
what is new section
Key findings
 We systematically reviewed tools used to assess
risk of bias of prevalence studies.
 We identified 30 tools; eight of them were specif-
ically designed for prevalence studies
What this adds to what was known?
 There was a great variability among items assessed
in each tool.
 Not all tools assessed all domains, and there was
overlap among items in some tools.
What is the implication and what should change
now?
 We provide a comprehensive set of items useful to
appraise prevalence studies.
In light of the above, the objective of this study is to sys-
tematically review, evaluate, and compare available tools
designed to assess the risk of bias of prevalence studies
in order to identify the domains and items used to evaluate
this type of study, providing information that could be used
in the development and update of tools, critical appraisal of
this type of study, and conduction of primary studies of
prevalence.
2. Methods
2.1. Study design, protocol, and registration
The present study is a systematic review. The study pro-
tocol was registered on PROSPERO, under the registration
number CRD42018088437.
2.2. Search strategy and data sources
We searched Medline (via PubMed), Embase, and Web of
Science up to August 2019 using terms such as ‘‘prevalence’’,
‘‘cross-sectional studies’’, and ‘‘critical appraisal’’. The
complete search strategy is presented in Additional file 1.
The search was not limited by date or language of
publication.
To identify studies not indexed by these databases, we
also screened the first 200 results on Google Scholar. We
also manually searched the reference list of relevant studies
and searched for instruments on websites of institutions
related to the topic. Moreover, we conducted a systematic
search for systematic reviews of prevalence of clinical con-
ditions published between February 2017 and February
2018 and indexed in Medline to identify further instruments
used to assess the quality of individual prevalence studies
[6].
For each instrument found, we conducted an internet
search for complementary material, including handbooks
or manuals for the instrument in question.
2.3. Study selection
We included methodological studies, manuals, or hand-
books with general guidance or specific tools applicable for
the critical appraise of prevalence studies. First, we re-
viewed the title and abstracts of all records identified in
our search to select all potentially relevant studies. Then,
we assessed the full text of selected studies and included
studies meeting the eligibility criteria. Study selection
was conducted by two reviewers independently (C.B.M.
and C.S.). Disagreements were solved by consensus or arbi-
trated by a third reviewer (V.C. or M.F.).
A tool was eligible if (1) it was developed to critically
appraise prevalence studies or (2) the authors stated it could
be applied to appraise prevalence studies or (3) it was used
by systematic review authors to appraise the quality of in-
dividual prevalence studies.
2.4. Data extraction
We extracted relevant information for each tool using
predesigned and piloted tables. Data extracted included:
process of development, applicability, structure, and con-
tent of the tool. Data extraction was conducted by two in-
dependent reviewers (C.B.M. and C.S.). Disagreements
were solved by consensus or arbitrated by a third reviewer
(V.C. or M.F.).
2.5. Data analysis
We classified each question or statement of the instru-
ments into items, which represented the objective of assess-
ment. The items were not prespecified. Each item is unique
and based on the fact that it addresses a different aspect of
quality of the study under appraisal. A new item was
created whenever the question/statement from the included
instrument would represent a different aspect of risk of
bias, trying to be as sensitive as possible. Questions or
statements from different instruments (sometimes even
from the same instrument) that assessed the same risk of
bias aspect, but with different wording, were merged under
the same item. The judgments regarding the classification
of questions and statements into items and domains are
available in Additional file 5. Afterward, we classified the
items into three key domains: ‘‘population and setting’’,
‘‘condition measurement’’, and ‘‘statistics’’. The domains
were defined a priori based on the main components of a
prevalence research question (population and condition)
and considering the importance of appropriate statistical
data analysis. If a question was applicable to appraise prev-
alence studies but covered a different domain (such as
 C.B. Migliavaca et al. / Journal of Clinical Epidemiology 127 (2020) 59e68 61

Is the tool specific for prevalence

studies?

Yes No

Classify all questions / statements Does the study provide guidance

into items and domains. about which questions / statements
are applicable for prevalence
studies? *

Yes No

Follow the guidance and only Classify questions / statements in

classify applicable questions / applicable or not, and then only
statements into items and domains. classify applicable questions into
items and domains.

Fig. 1. Flowchart of classification of questions/statements. *In some instruments, the guidance about which questions/statements were applicable
for prevalence studies was based on study aspects such as intervention or the comparison group.

reporting or study methods), it was included under the clas-
sification ‘‘other’’.
As described, we included not only tools specifically de-
signed to assess prevalence studies but also tools that could
be adapted for this purpose. Thus, not all questions from
nonspecific tools were applicable for prevalence studies (such
as questions assessing the comparability among groups, or the
description of intervention), and we only categorized the
applicable ones. If the instrument provided guidance about
which questions should be used to assess prevalence studies,
we followed these instructions. If not, before classyifing the
questions into items and domains, we evaluated if they were
applicable for prevalence studies or not. If classified as appli-
cable, the question was categorized into items and domains as
previously described. Questions classified as not applicable
were not further evaluated (Fig. 1).
The process of selection and classification of questions/
statements into items and domains was conducted by two
reviewers independently (C.B.M. and C.S.). Discrepancies
were solved by consensus or arbitrated by a third reviewer
(V.C. or M.F.).
3. Results
3.1. Study selection
Our search resulted in 1,690 unique references. After se-
lection of titles and abstracts, we assessed 105 full texts for
eligibility. Finally, we included in the review 30 tools
[11e41]. Fig. 2 presents the flowchart of study selection.
The list of full texts excluded with reason is available in
Additional file 2.
3.2. Tools specific for prevalence studies
Out of the 30 tools, eight (26.7%) were specifically de-
signed to appraise prevalence studies [11e19]. Table 1 sum-
marizes the main characteristics of these tools. Among these

Records identified through Additional records identified

database searching through other sources
(n = 2.858) (n = 66)
PubMed: 870 Google Scholar and websites: 2
Embase: 1.129 Reference lists: 54
Web of Science: 859 Systematic reviews: 10

Duplicates removed
(n = 1.235)
Records screened
(n = 1.689)
Records excluded
(n = 1.590)
Full-text assessed for elegilibity
(n = 99)
Full text articles excluded
(n = 67)
Does not describe new tool (n = 36)
Full text included in qualitative Tool not applicable for prevalence studies (n = 28)
Other (n = 3)
synthesis
(n = 32)
Number of tools = 30

Fig. 2. Flowchart of study selection.

62
Table 1. Main characteristics of tools specifically designed to appraise prevalence studies
Instrument (name of the
instrument or author,
year)
Al-Jader et al., 2002
[11]
Boyle, 1998 [12]
Giannakapoulos et al.,
2012 [13]
Hoy et al., 2012 [14]
Loney et al., 1998 [15]
MORE, 2010 [[16]]
Silva et al., 2001 [17]
C.B. Migliavaca et al. / Journal of Clinical Epidemiology 127 (2020) 59e68
Context of development
(clinical condition)
Genetic disorders.
Psychiatric disorders on
general population
settings.
Any clinical condition.
Any clinical condition.
Any clinical condition.
Chronic conditions.
Risk factors of chronic
diseases.
Process of development
First version of the tool;
pilot test with
multidisciplinary
assessors to evaluate
reproducibility and
feasibility; final
version of the tool and
test for inter-rater
agreement.
NR
Search for criteria to
define a high-quality
study of prevalence;
development of the
first version; pilot
tests to determine
inter-rater agreement
and reliability; the
final version of the
tool.
Search for instruments;
definition of
important criteria to
be assessed and
creation of the draft
tool; pilot tests with
professionals;
assessment of inter-
rater agreement, ease
of use, timeliness; the
final version of the
tool.
Review of important
criteria; development
of the tool; pilot test
in prevalence studies
of dementia.
Systematic search for
instruments to assess
prevalence and
incidence studies;
selection of important
criteria; development
of the first version;
pilot test with experts
to assess face
validity, inter-rater
agreement, and
reliability; the final
version of the tool.
NR
Summary and reporting of
Structure results
Seven questions, with Maximum score: 100
different answer points.
options; each answer No cutoff point defined.
option with an
associated score.
Ten questions, split in No overall summary.
three sections. No Descriptive reporting of
predefined answer results.
options.
Eleven questions, split Maximum score: 19
in three sections, plus points. Studies are
a question about classified in
ethics. Each question accordance with their
with two or three total score as poor (0
answer options; each e4), moderate (5e9),
answer option with an good (10e14), or
associated score. outstanding (15e19).
Ten questions with two Question for overall
standard answer appraisal with three
options (high risk of answer options (low
bias/low risk of bias). risk of bias/moderate
risk of bias/high risk
of bias), based on
rater’s judgment.
Eight statements, one Maximum score: eight
point for each points.
criterion achieved. No cutoff point defined.
Thirty-two questions, No overall summary
with different answer Descriptive reporting of
options; each answer results.
option is classified as
‘minor flaw’, ‘major
flaw’, or ‘poor
reporting’.
Nineteen questions, Maximum score: 100
split in three sections. points.
Two or three answer No cutoff point defined.
options, with an
associated score.
(Continued )
 C.B. Migliavaca et al. / Journal of Clinical Epidemiology 127 (2020) 59e68 63

Table 1. Continued
Instrument (name of the
instrument or author,
year)
The Joanna Briggs Any clinical condition.
Institute Prevalence
Critical Appraisal
Tool, 2014
[[18],[19]]
Context of development
(clinical condition)
Systematic search for
Summary and reporting of
Process of development Structure results
Nine questions with four Question for overall
instruments to assess standard answer appraisal with three
prevalence studies; options ( yes/no/ answer options
review and selection unclear/not (include/exclude/seek
of applicable criteria; applicable).a further info), based
development of the on rater’s judgment.
draft tool; pilot tests
with professionals to
assess face validity,
applicability,
acceptability,
timeliness, and ease
of use; the final
version of the tool.

Abbreviation: NR, not reported.

a
Ten questions in previous versions.

tools, seven (87.5%) were new tools [11e13,15e19], and
one (12.5%) was an adaptation of an existing instrument
[14]. Four tools (50.0%) were developed to assess studies
of prevalence of any clinical condition [13e15,18,19],
whereas the other four tools (50.0%) were developed to
appraise prevalence studies of specific medical fields
[11,12,16,17]; however, with some adaptations, they could
all be applied to any clinical condition. The process of devel-
opment of all instruments included search and review of rele-
vant criteria, piloting test(s), and adjustments for the final
version. The median number of questions in the tools was
10, ranging from seven to 32. Four tools (50.0%) were scales,
with numeric results [11,13,15,17], and four (50.0%) were
descriptive checklists [12,14,16,18,19]. Among the scales,
only one suggested cutoff values to define the overall quality
of the study [13]; among the checklists, two had an overall
appraisal question, but they were answered based on rater’s
judgment, without guidance of how to consider the previous
questions to define a summary assessment [14,18,19].
Regarding the domains assessed, seven tools (87.5%)
covered all key domains [12e19]; however, there was vari-
ability regarding the items assessed by each tool. Table 2
describes the items of each tool, classified by domains. In
the domain ‘‘population and setting’’, the main items as-
sessed were ‘‘appropriate sampling’’ (seven tools, 87.5%),
‘‘appropriate response rate’’ (five tools, 62.5%), and
‘‘representative sample’’ (four tools, 50.0%). In the domain
‘‘condition measurement’’, the main items assessed were
‘‘valid measurement of condition’’ (six tools, 75.0%),
‘‘standard measurement of condition’’ (six tools, 75.0%),
and ‘‘reliable measurement of condition’’ (five tools,
62.5%). In the domain ‘‘statistics’’, the main items assessed
were ‘‘precision of estimate’’ (six tools, 75.0%), ‘‘data
analysis considering sampling’’ (three tools, 37.5%),
‘‘appropriate sample size’’ (two tools, 25.0%), and ‘‘sub-
group analysis’’ (two tools, 25.0%). Overall, these tools
provided 25 unique items related to the assessment of
quality of prevalence studies, six related to ‘‘population
and setting’’, nine related to ‘‘condition measurement’’,
and 10 related to ‘‘statistics’’. In addition, 24 items were
classified as ‘‘other’’, mainly related to reporting (Table 2).
3.3. Tools adapted for prevalence studies
Among the 30 included tools, 22 (73.3%) were not spe-
cific for prevalence studies. The main characteristics of
these tools and the items and domains assessed by them
are presented in Additional file 3 and 4, respectively. These
tools provided six unique additional items for the domain
‘‘population and setting’’, seven items related to ‘‘condition
measurement’’, and four items related to ‘‘statistics’’.
Moreover, these tools provided 53 items classified under
the domain ‘‘other’’ (Additional file 4).
3.4. Items
We identified 710 questions/statements from the
included tools that were compiled into 119 different items.
We identified 42 unique items classified under the do-
mains ‘‘population and setting’’ (12 items), ‘‘condition
measurement’’ (16 items), and ‘‘statistics’’ (14 items); of
those, 25 (59.5%) were identified in the eight specific tools
and 17 (40.5%) were identified in the additional 22 nonspe-
cific tools. Table 3 summarized the items assessed in each
domain among all tools.
In the domain ‘‘other’’, we identified 77 unique items; of
those, 24 (31.2%) were identified in the eight specific tools
and 53 (68.8%) were identified in the additional 22 nonspe-
cific tools. We classified the items in the domain ‘‘other’’
into three categories: ‘‘manuscript writing and reporting’’,
‘‘study protocol and methods’’, and ‘‘nonclassified’’. These
categories were defined after data extraction, based on our
findings. Table 4 presents the items classified as ‘‘other’’
stratified by these categories.
64 C.B. Migliavaca et al. / Journal of Clinical Epidemiology 127 (2020) 59e68

Table 2. Items assessed by tools specifically designed to appraise prevalence studies, classified by domainsa

Instrument (name of the Domain

instrument or author, year)
Al-Jader et al., 2002 [11]
Boyle, 1998 [12]
Giannakapoulos et al.,
2012 [13]
Hoy et al., 2012 [14]
Loney et al., 1998 [15]
MORE, 2010 [[16]]
Silva et al., 2001 [17]
The Joanna Briggs Institute Representative sample,
Prevalence Critical
Appraisal Tool, 2014
[18,19]
a
The full set of question for every tool is presented in Additional file 5.
b
Tool specific for studies of prevalence of genetic conditions.
Population and setting Condition measurement Statistics Other
Representative sample, e Precision of estimates Description of condition of
ethnic characteristics of interest, reporting of year
population source, of conduction of studies,
appropriate size of reporting of size of
population sourceb population source
Representative sample, Reliable and valid Precision of estimates, data Description of target
appropriate sampling measurement of analysis considering population, standard data
condition sampling collection
Appropriate sampling and Reliable, standard, and Precision of estimates, data Description of target
response rate valid measurement of analysis considering population, ethics
condition response rate and special
features
Representative sample, Appropriate definition of Appropriate numerator and Appropriate data collection
appropriate sampling, condition, reliable, denominator parameters
and appropriate response standard, and valid
rate measurement of
condition, and
appropriate length of
prevalence period
Appropriate sampling and Appropriate, standard, and Appropriate sample size, Appropriate study design,
appropriate response rate unbiased measurement precision of estimates description of
of condition participants, setting, and
nonresponders
Appropriate sampling and Appropriate, reliable, Precision of estimates, Reporting of study design,
appropriate response rate standard, and valid appropriate exclusion description of study
measurement of from analysis, data objectives, reporting of
condition, assessment of analysis considering inclusion flowchart,
disease severity and sampling, subgroup description, and role
frequency of symptoms, analysis, and adjustment funding, and reporting of
type of prevalence of estimates conflict of interest, ethics
estimate (point or
period), and appropriate
length of prevalence
period
Appropriate sample source Appropriate definition of Precision of estimates, Description of study
and appropriate sampling condition, standard and subgroup analysis, data objectives and sampling
valid measurement of analysis considering frame, quality control of
condition sampling data, applicability and
generalizability of results
Reliable, standard, and Appropriate sample size, Description of participants
appropriate sampling, valid measurement of appropriate statistical and setting, objective
and appropriate response condition analysis, data analysis criteria for subgroup
rate considering response rate definitions

4. Discussion Not all domains were covered by all tools, and even
when they were covered, they were not always properly as-
In this systematic review, we identified, summarized, sessed. Some tools did not consider important aspects in-
and compared 30 instruments used for the quality assess-
side each domain, such as representativeness of sample,
ment of prevalence studies. Our results, similar to what
estimation of sample size, and appropriate measurement
was found in other reviews, show that there is great vari-
of condition, and there was an overlap among questions
ability among tools and there is no consensus about which
in the same instrument, which may lead to penalization
domains should be assessed in prevalence studies [42,43].
of the same study for the same reason more than once.
We classified all questions or statements into items and do-
Moreover, many instruments assessed not only risk of bias
mains, creating a comprehensive set of 119 items useful for
but also reporting and manuscript writing. It is important to
the assessment of prevalence studies.
 C.B. Migliavaca et al. / Journal of Clinical Epidemiology 127 (2020) 59e68 65

Table 3. Unique items identified among all included tools and classified into a key domain
Population and setting (n [ 12)
 Appropriate samplea (seven tools)
 Unbiased samplea (one tool)
 Representative sample (14 tools)
 Appropriate sample source (two tools)
 Appropriate size of population source
(one tool)
 Ethnic characteristics of population
source (one tool)
 Appropriate sampling (15 tools)
 Random samplinga (one tool)
 Standard selection of participantsa (two
tools)
 Participation rate of eligible personsa
(one tool)
 Appropriate response rate (19 tools)
 Assessment of nonrespondersa (one
tool)
Condition measurement (n [ 16)
 Type of prevalence estimate (point or
period) (one tool)
 Appropriate length of prevalence period
(two tools)
 Appropriate definition of condition (two
tools)
 Appropriate measurement of condition
(11 tools)
 Accurate measurement of conditiona
(one tool)
 Precise measurement of conditiona
(one tool)
 Quality control of measurement meth-
odsa (two tools)
 Valid measurement of condition (21
tools)
 Reliable measurement of condition (15
tools)
 Standard measurement of condition
(10 tools)
 Unbiased measurement of condition
(three tools)
 Reproducible measurement of condi-
tiona (two tools)
 Assessment of disease severity and
frequency of symptoms (one tool)
 Data collection performed by investi-
gators unrelated to patientsa (one tool)
 Face validitya (one tool)
 Selective outcome reportinga (one tool)
Statistics (n [ 14)
 Sample size estimationa (10 tools)
 Appropriate sample size (seven tools)
 Appropriate statistical analysis (13
tools)
 Appropriate numerator and denomina-
tor parameters (one tool)
 Appropriate exclusion from analysis
(one tool)
 Adjustment of estimates (one tool)
 Data analysis considering sampling
(three tools)
 Data analysis considering the response
rate (seven tools)
 Data analysis considering special fea-
tures (one tool)
 Missing data handlinga (one tool)
 Random errora (three tools)
 Precision of estimate (11 tools)
 Subgroup analysis (five tools)
 Data fishinga (two tools)

a
Items from nonspecific tools only.

distinguish between these two concepts as poor reporting is
not a reflection on the quality of a study or whether the re-
sults from a study are at risk of bias.
We conducted a broad search, using important databases
and including alternative data sources. Our search was very
sensitive to identify tools specifically designed for preva-
lence studies, but we probably have not included all instru-
ments that could be adapted for this purpose. This could be
a limitation of our study; however, we believe our results
are representative of the items and domains used to
appraise prevalence studies because we probably achieved
a saturation of items, and this work is the most comprehen-
sive overview of tools to assess prevalence studies to date.
Another possible limitation of our review is that data
abstraction and classification of questions into items and
domains required judgment, which can lead to different de-
cisions by different assessors. We tried to overcome this by
conducting the classification independently by two re-
viewers with the assistance of third reviewers in case of dis-
crepancies. In addition, our decision-making was informed
by a protocol which reduced the chance of making ad hoc,
subjective decisions during the conduct of our review.
Moreover, to enhance transparency of the process, our
judgments regarding the classification of questions and
statements into items and domains are available in
Additional file 5.
The main objective of this study was to map the literature
to identify items used to assess the risk of bias of prevalence
studies, generating a comprehensive bank of items. Some of
the items identified are similar and there may be overlap
among them in terms of their broad definition. However,
we do believe that subtle differences in terminology and
nomenclature are important in the contextualization of the
tools in which they are derived. This granular approach is
justified in this case as this will facilitate the future develop-
ment of a new risk of bias assessment tool. As an example of
the importance of identifying subtle differences between
similar items in tools, we attempt in the following to clarify
the differences between valid, reliable, reproducible, and un-
biased measurement of the condition (which some readers
may have considered synonymous terms previously):
 Valid measurement of the condition: the measurement
of the condition is performed with methods that actu-
ally measures or detects what it is supposed to
measure.
 Reliable measurement of the condition: this is related
to the consistency of a measure. A highly reliable
measure produces similar results under similar
66 C.B. Migliavaca et al. / Journal of Clinical Epidemiology 127 (2020) 59e68

Table 4. Items classified as ‘other’, categorized into three subgroups

Manuscript writing and reporting (n [ 48) Study protocol and methods (n [ 12) Nonclassified (n [ 17)
 Clear reporting of authors and affilia-  Specific objectivesa (two tools)  Importance of studya (three tools)
tionsa (one tool)  Study protocola (one tool)  Quality control of data (one tool)
 Appropriate title a(one tool)  A priori statistical analysis plana (one  Relevance of the research questiona
 Appropriate abstracta (one tool) tool) (one tool)
 Study justified by literature reviewa  Appropriate study design (11 tools)  Relevance of outcomesa (one tool)
(one tool)  Appropriate review of the existing liter-  Identification of biasa (one tool)
 Description of the problema (one tool) aturea (two tools)  Consistent resultsa (three tools)
 Theoretical frameworka (one tool)  Appropriate methodsa (three tools)  Believable resultsa (one tool)
 Clear hypothesisa (one tool)  Appropriate data collection (two tools)  Conclusion plausiblea (two tools)
 Clear study questionsa (eight tools)  Standard data collection (one tool)  Possible alternative conclusionsa (one
 Description of study objectives (13  Consideration of important variablesa tool)
tools) (one tool)  Relevance of conclusionsa (one tool)
 Description of condition of interest (10  Consideration of privacy and sensitivity  Applicability of results (one tool)
tools) of conditiona (one tool)  Generalizability of results (six tools)
 Description of target population (seven  Objective criteria for subgroup defini-  Ethics (six tools)
tools) tions (one tool)  Effect of conflict of interesta (two tools)
 Description of the setting (six tools)  Role of funding (one tool)
 Reporting of the study design (five  Bias due to fundinga (one tool)
tools)  Reader’s interpretation of studya (three
 Description of methodsa (three tools) tools)
 Reporting of the size of the population  Othera (one tool)
source (one tool)
 Description of the sampling frame (five
tools)
 Description of eligibility criteriaa (seven
tools)
 Reporting of the year of conduction of
study (one tool)
 Description of statistical analysis
(seven tools)
 Appropriate data reportinga (three
tools)
 Appropriate reporting of resultsa (10
tools)
 Reporting of inclusion flowchart (three
tools)
 Reporting of sample size (one tool)
 Reporting of the response ratea (three
tools)
 Description of nonresponders (four
tools)
 Description of participants (10 tools)
 Reporting of data collection procedur-
esa (two tools)
 Clear description of data sourcesa (one
tool)
 Description of missing dataa (two tools)
 Reporting of adjusted estimatesa (one
tool)
 Reporting of statistical significancea
(two tools)
 Reporting of clinical significancea (two
tools)
 Reporting of discussiona (one tool)
 Appropriate discussiona (three tools)
 Discussion based on resultsa (one tool)
 Reporting of all possible interpretation
of resultsa (one tool)
 Discussion of biasa (two tools)
 Discussion of limitationsa (five tools)
 Discussion of strengthsa (one tool)

(Continued )
 C.B. Migliavaca et al. / Journal of Clinical Epidemiology 127 (2020) 59e68
Table 4. Continued
Manuscript writing and reporting (n [ 48) Study protocol and methods (n [ 12)
 Comparison of results with the existing
literaturea (three tools)
 Description of study conclusionsa (one
tool)
 Appropriate conclusionsa (two tools)
 Conclusion based on resultsa (10 tools)
 Reporting of an additional information
sourcea (one tool)
 Description of funding (two tools)
 Reporting of conflict of interest (four
tools)
 Clear referencesa (one tool)
 Recommendations for future researcha
(two tools)
a
Items from nonspecific tools only.
conditions, so all things being equal, repeated testing
should produce similar results.
 Reproducible measurement of the condition: the mea-
surement of the condition is performed using methods
capable of being reproduced at a different time or
place and by different people
 Unbiased measurement of the condition: measure-
ment of the condition free of systematic errors that
could deviate the results from the truth.
Therefore, even though the differences among items are
subtle, we do believe they are important to inform the
development of a new tool.
It is not possible to strongly recommend a tool because
there is great variability in their content. A new tool, domain
based and with broader coverage and applicability is needed.
However, among the currently available tools specific for
prevalence studies, the Joanna Briggs Institute Prevalence
Critical Appraisal Tool has a higher methodologic rigor
and addresses what we consider the most important items
related to the methodological quality of prevalence studies
and may be considered the most appropriate tool [18,19].
5. Conclusions
We have now identified a comprehensive set of items
and domains that is broader than any of the individual tools
included in this review. This data set can be now be used by
those interested in the critical appraisal of prevalence
studies, by authors of prevalence studies, and to inform
the development or update of future tools to critically
appraise prevalence studies.
CRediT authorship contribution statement
Celina Borges Migliavaca: Formal analysis, Investiga-
tion, Visualization, Data curation, Writing - original draft.
67
Nonclassified (n [ 17)
Cinara Stein: Formal analysis, Investigation, Writing - re-
view & editing. Ver^onica Colpani: Validation, Project
administration, Writing - review & editing. Zachary
Munn: Methodology, Writing - review & editing. Maicon
Falavigna: Conceptualization, Validation, Writing - review
& editing.
Supplementary data
Supplementary data to this article can be found online at
https://doi.org/10.1016/j.jclinepi.2020.06.039.
References
[1] Fletcher R, Fletcher S, Fletcher GS. Clinical epidemiology: the es-
sentials. 5th ed. Philadelphia: Lippincott Williams & Wilkins; 2013.
[2] Harder T. Some notes on critical appraisal of prevalence studies:
comment on: "The development of a critical appraisal tool for use
in systematic reviews addressing questions of prevalence. Int J Health
Pol Manag 2014;3:289e90.
[3] Wagner MB. Medindo a corr^eencia da doença: preval^encia ou in-
cid^encia? J Pediatr (Rio J 1998;74:157e62.
[4] Oxman AD, Schunemann HJ, Fretheim A. Improving the use of
research evidence in guideline development: 2. Priority setting.
Health Res Pol Syst 2006;4:14.
[5] Rotily M, Roze S. What is the impact of disease prevalence upon
health technology assessment? Best Pract Res Clin Gastroenterol
2013;27:853e65.
[6] Borges Migliavaca C, Stein C, Colpani V, et al. How are systematic
reviews of prevalence conducted? A methodological study. BMC
Med Res Methodol 2020;20:96. https://doi.org/10.1186/s12874-020-
00975-3.
[7] Shamliyan T, Kane RL, Dickinson S. A systematic review of tools
used to assess the quality of observational studies that examine inci-
dence or prevalence and risk factors for diseases. J Clin Epidemiol
2010;63:1061e70.
[8] Sterne JA, Hernan MA, Reeves BC, Savovic J, Berkman ND,
Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias
in non-randomised studies of interventions. BMJ 2016;355:i4919.
[9] Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ,
Reitsma JB, et al. QUADAS-2: a revised tool for the quality
68 C.B. Migliavaca et al. / Journal of Clinical Epidemiology 127 (2020) 59e68
assessment of diagnostic accuracy studies. Ann Intern Med 2011;155:
529e36.
[10] Sterne JAC, Page MJ, Elbers RG, Blencowe NS, Boutron I, Cates CJ,
et al. RoB 2: a revised tool for assessing risk of bias in randomised
trials. BMJ 2019;366:l4898.
[11] Al-Jader LN, Newcombe RG, Hayes S, Murray A, Layzell J,
Harper PS. Developing a quality scoring system for epidemiological
surveys of genetic disorders. Clin Genet 2002;62:230e4.
[12] Boyle MH. Guidelines for evaluating prevalence studies. Evid Based
Ment Health 1998;1:37.
[13] Giannakopoulos NN, Rammelsberg P, Eberhard L, Schmitter M. A
new instrument for assessing the quality of studies on prevalence.
Clin Oral Invest 2012;16:781e8.
[14] Hoy D, Brooks P, Woolf A, Blyth F, March L, Bain C, et al. Assess-
ing risk of bias in prevalence studies: modification of an existing tool
and evidence of interrater agreement. J Clin Epidemiol 2012;65:
934e9.
[15] Loney PL, Chambers LW, Bennett KJ, Roberts JG, Stratford PW.
Critical appraisal of the health research literature: prevalence or inci-
dence of a health problem. Chronic Dis Can 1998;19:170e6.
[16] Shamliyan TA, Kane RL, Ansari MT, Raman G, Berkman ND,
Grant M, et al. Development quality criteria to evaluate nonthera-
peutic studies of incidence, prevalence, or risk factors of chronic dis-
eases: pilot study of new checklists. J Clin Epidemiol 2011;64:
637e57.
[17] Silva LC, Ordunez P, Paz Rodriguez M, Robles S. A tool for assess-
ing the usefulness of prevalence studies done for surveillance pur-
poses: the example of hypertension. Rev Panam Salud Publica
2001;10:152e60.
[18] Munn Z, Moola S, Riitano D, Lisy K. The development of a critical
appraisal tool for use in systematic reviews addressing questions of
prevalence. Int J Health Pol Manag 2014;3:123e8.
[19] Munn Z, Moola S, Lisy K, Riitano D, Tufanaru C. Methodological
guidance for systematic reviews of observational epidemiological
studies reporting prevalence and cumulative incidence data. Int J
Evid Based Healthc 2015;13:147e53.
[20] Academy of Nutrition and Dietetics. Evidence analysis manual: steps
in the academy evidence analysis process. Chicago: Academy of
Nutrition and Dietetics; 2016:106.
[21] Avis M. Reading research critically. II. An introduction to appraisal:
assessing the evidence. J Clin Nurs 1994;3:271e7.
[22] Downes MJ, Brennan ML, Williams HC, Dean RS. Development of a
critical appraisal tool to assess the quality of cross-sectional studies
(AXIS). BMJ Open 2016;6:7.
[23] Berra S, Elorza-Ricart JM, Estrada MD, Sanchez E. [A tool (cor-
rected) for the critical appraisal of epidemiological cross-sectional
studies]. Gac Sanit 2008;22:492e7.
[24] The University of Manchester. Centre for occupational and environ-
mental health (COEH) - critical appraisal.http://research.bmh.
manchester.ac.uk/epidemiology/COEH/undergraduate/
specialstudymodules/criticalappraisal/Accessed 1 August 2020
[25] Downs SH, Black N. The feasibility of creating a checklist for the
assessment of the methodological quality both of randomised and
non-randomised studies of health care interventions. J Epidemiol
Community Health 1998;52:377e84.
[26] DuRant RH. Checklist for the evaluation of research articles. J Ado-
lesc Health 1994;15:4e8.
[27] Fowkes FG, Fulton PM. Critical appraisal of published research:
introductory guidelines. BMJ 1991;302:1136e40.
[28] Gardner MJ, Machin D, Campbell MJ. Use of check lists in assessing
the statistical content of medical studies. Br Med J (Clin Res Ed)
1986;292:810e2.
[29] Glynn L. A critical appraisal tool for library and information
research. Libr Hi Tech 2006;24:387e99.
[30] Kmet LS, Lee RC. Standard quality assessment criteria for evaluating
primary research papers from a variety of fieldsAHFMRHTA initia-
tive20040213. HTA Initiative 2004:2.
[31] Law M, Stewart D, Pollock N, Letts L, Bosch J, Westmorland MIn:
Critical review formdquantitative studies, 20 1998:13.
[32] Margetts B, Vorster H, Venter C. Evidence-based nutrition-review of
nutritional epidemiological studies. South Afr J Clin Nutr 2002;15:
68e73.
[33] Slim K, Nini E, Forestier D, Kwiatkowski F, Panis Y, Chipponi J.
Methodological index for non-randomized studies (minors): develop-
ment and validation of a new instrument. ANZ J Surg 2003;73:
712e6.
[34] Hong QN, Pluye P, Fabregues S, Bartlett G, Boardman F, Cargo M,
et al. Mixed methods appraisal tool (MMAT), version 2018. Canada:
IC Canadian Intellectual Property Office, Industry Canada; 2018.
[35] Wells G, Shea B, O’Connell D, Peterson J, Welch V, Losos M, et al.
Newcastle-Ottawa quality assessment scale cohort studies 2014.
Available at http://www.ohri.ca/programs/clinical_epidemiology/
oxford.asp. Accessed August 1, 2020.
[36] NIH. Quality Assessment tool for observational cohort and cross-
sectional studies 2018. Available at https://www.nhlbi.nih.gov/health-
topics/study-quality-assessment-tools. Accessed August 1, 2020.
[37] Public Health Wales Observatory. Critical appraisal checklist: cross
sectional study 2004. Available at http://www2.nphs.wales.nhs.uk:
8080/PubHObservatoryProjDocs.nsf/($All)/
E7B0C80995DC1BA380257DB80037C699/$File/Cross%
20sectional%20study%20checklist.docx?OpenElement. Accessed
August 1, 2020.
[38] Wong WCW, Cheung CSK, Hart GJ. Development of a quality
assessment tool for systematic reviews of observational studies (QAT-
SO) of HIV prevalence in men having sex with men and associated
risk behaviours. Emerging Themes Epidemiol 2008;5:23.
[39] Viswanathan M, Berkman ND. Development of the RTI item bank on
risk of bias and precision of observational studies. J Clin Epidemiol
2012;65:163e78.
[40] Viswanathan M, Berkman ND, Dryden DM, Hartling L. ahrq
methods for effective health care. assessing risk of bias and con-
founding in observational studies of interventions or exposures:
further development of the RTI item bank. Rockville, MD: Agency
for Healthcare Research and Quality (US); 2013.
[41] Specialist Unit for Review Evidence (SURE). Questions to assist with
the critical appraisal of cross-sectional studies 2018. Available at
https://www.cardiff.ac.uk/__data/assets/pdf_file/0010/1142974/SURE-
CA-form-for-Cross-sectional_2018.pdf. Accessed August 1, 2020.
[42] Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and suscep-
tibility to bias in observational studies in epidemiology: a systematic re-
view and annotated bibliography. Int J Epidemiol 2007;36:666e76.
[43] Jarde A, Losilla JM, Vives J. Methodological quality assessment
tools of non-experimental studies: a systematic review. An Psicol
2012;28:617e28.