Professional Documents
Culture Documents
ABSTRAK
Critical appraisal merupakan bagian dari kedokteran berbasis bukti (evidence-based medicine) diartikan
sebagai suatu proses evaluasi secara cermat dan sistematis suatu artikel penelitian untuk menentukan reabilitas,
validitas, dan kegunaannya dalam praktik klinis. Komponen utama yang dinilai dalam critical appraisal adalah
validity, importancy, dan applicability. Tingkat kepercayaan hasil suatu penelitian sangat bergantung dari
disain penelitian dimana uji klinis menempati urutan tertinggi. Telaah kritis meliputi semua komponen dari
suatu penelitian dimulai dari komponen pendahuluan, metodologi, hasil dan diskusi. Masing-masing komponen
memiliki kepentingan yang sama besarnya dalam menentukan apakah hasil penelitian tersebut layak atau tidak
digunakan sebagai referensi. Kemaknaan secara statistik yang didapat hendaknya juga dibandingkan dengan
kemaknaan secara klinis.
Kata kunci: critical appraisal, telaah kritis, evidence-based medicine, uji klinis, clinical trial.
ABSTRACT
Critical appraisal is an element of evidence-based medicine, which is defined as the process of carefully and
systematically examining a research article to determine its reliability, validity and value in clinical practice.
The major components evaluated in critical appraisal are validity, importancy, and applicability. The level of
reliability of study results depend on the design of study, in which clinical trial has the highest rank. Critical
appraisal includes all components of the research starting from the introduction, method, results and discussion.
Each component has similar value to establish whether the results can realistically be applied as a referrence.
The results of statistical significance should also been compared with its clinical significance.
and [5] expert opinions.1 to take from the findings including applying the
Method of appraising information should be eveidences for clinical practice; and 5) evaluate
done sistematically to provide a good conclusion, the practice performed according to the evidence-
which is the best kind of information. It includes based medicine (Figure 1).2,3
appraisal of various sources of information as Formulating question by using PICO concept
well as appraising the conclusion by providing can also be the element of critical appraisal itself.
supporting evidences. One of the methods The following is the example of PICO concept:
is critical appraisal. By performing critical “In a 50-year-old female patient who is diagnosed
appraisal, we are expected to be able to evaluate with type 2 diabetes mellitus (Patient), how
results, validity and value of scientific article great is the role of three-month diet and exercise
publications systematically. This manuscript program (Intervention) to help normalize the
is going to discuss about critical appraisal and fasting blood glucose (Outcome) compared to
focus on the appraisal of clinical trial articles in diet restriction alone (Comparator)?”
scientific journals.
Starting Critical Appraisal of Clinical Trial
Before we are performing critical appraisal
CRITICAL APPRAISAL of a research article in a scientific journal,
we should first comprehend the article. A
Definition good comprehension of an article will make
By definition, critical appraisal is regarded as our evaluation better. In general, the basic
a process of evaluating a research article carefully components of a research article are introduction,
and systematically to determine the reliability, methods, results and discussion (Figure 2).
validity and application in clinical practice.2 In Usually, when we take a look at a research
other words, through critical appraisal, we decide article, the abstract is the first part that we
a research article is reliable or not. The ability to read. Most of us believe that the answer to our
perform critical appraisal should be one of basic questions is there in a concise form, while it
competence of a clinician in order to recognize would be too risky to consider an abstract as a
and use reliable study data efficiently. true summary of a research. Use it only to decide
The next question is how do we know that the about the topic that we are interested in and to
obtained data has good quality and reliable? How avoid further reading if by reading it we can
could we decide which study is more reliable already see a poor method of study.
when we have data of two different conclusion In fact, we do not have to read the article
on the same topic? For this reason, therefore, we word by word from beginning to end. In
should performm critical appraisal. critical appraisal, we would evaluate the article
Critical appraisal is an important element systematically. The list of questions that we
of EBM. As we have known that there are five should find the answer when reading a research
steps in EBM when evaluating a clinical case, article are shown in Table 1.
i.e.: 1) formulating questions with the concept Overall, components of critical appraisal
of PICO (Population, Intervention, Comparation, for a research article are validity, importance
Outcome); 2) search for the evidence or reference; and applicability, which is more familiar as the
3) appraise the evidence on the component of abbreviation of VIA as shown in Figure 3 as the
validity and importancy; 4) decide what action following.
Asking
Clinical problems questions Search Appraise Decide Evaluate
338
Vol 44 • Number 4 • October 2012 Critical appraisal on journal of clinical trials
Comprehend Evaluate
Outline Abstract
Figure 2. Components of a research article that should be comprehended before doing critical appraisal
339
Murdani Abdullah Acta Med Indones-Indones J Intern Med
validity of selection
Non-causal validity of information
internal validity validity of control for confounding
validity of analysis
Temporality
Degree of association
Validity Causal internal Dose response
validity Consistency
Coherency
Specificity
Biological plaussability
The component of validity is aimed to proof Table 2. List of questions for the introduction of clinical
whether the result is a fact or not; while the journal article4
340
Vol 44 • Number 4 • October 2012 Critical appraisal on journal of clinical trials
Table 3. List of questions for the methods of clinical journal randomization table.4,6
article4 Analysis plan usually will provide
Components Explanation information about what kind of statistical tests
Population and -- Who are the target and accessible that will be used to analyse data and how the level
subjects population in the study?(target of significance is used. Usually, the statistical
population refers to the population
that has become the objective analysis being used is highly associated with
of generalizing the study, which hypothesis of the study; for example, Chi-square
has clinical and demographical test is used when the study aims for comparing
components; while accessible
population is part of target the proportion of unpaired group. Moreover, we
population wh ich has clinical, should also find out about what kind of analysis
demographical, temporal and
spatial components)
method that has been used in the study, either it
is an analysis per protocol or intention to treat
-- What are the inclusion and
exclusion criteria? (ITT) analysis. When the analysed subjects were
Study design -- What are the methods used for those who had participated and completed the
randomization? (for example, study, then the analysis per protocol would be
simple, block, or stratified
randimisation etc).
appropriate. In contrast, ITT analysis is used for
those who had been randomized although they
-- What tools that has been used
for randomization? (table or had not complete the study.4,6,8 Another analysis
computer)? is the interim analysis, i.e. any assessment of
-- Who perform the randomization? data performed before the number of all study
Is there any concealment? What
kind of concealment that has been
subjects has been completed. The analysis is
performed? usually used when there is a suspicion that one
-- Is there any blinding and how the of treatment is more superior than others and if
blinding was performed? the analysis has proven the suspicion, the study
Sample size -- What is the formula of sample size should be stopped.4
that (should be) used? (Is there
any comparison between unpaired The Results
and paired proportion, or between
unpaired and paired mean value?).
The Result section of a clinical trial research
article usually consists of the flow of the study
-- What are the rationales for
determining sample size? (for and the study data including primary and
example, does the study determine secondary outcomes. Table 4 shows a list of
the degree of type I and II error and
the expected power?)
overall questions relevant to the results section.4,7
Analysis plan -- What kind of statistical tests
The flow of the study could be occassionally
that have been used? Are they found in the methods section. It is common to
consistent with the hypothesis and present the flow of the study in diagram or flow
the aims of the study?
chart to provide easy reading about the process
-- What are the methods used for
analysis? (analysis per protocol or
involved in the study. Through the flow, we
intention to treat?) could obtain some information about how many
-- What computer program used for subjects have been recruited, how many subjects
statistical analysis? excluded from the study and how many eligible
-- Is there any interim analysis that subjects. Other information that could be found
has been planned for the study?
is about how large is the percentage of eligible
subjects and what is the percentage of subjects
We should recognize some basic terms that were lost-to follow up.
that are strongly associated with the methods Baseline data is a series of initial data prior
section, which are randomization, blinding and to the intervention. Our task here is to evaluate
concealment. Randomization refers to random whether the groups are comparable in regards of
allocation, i.e. a process of randomization to demographical, socio-economic, and history of
determine to which group a subject will be illness. If the baseline data are not comparable,
allocated. Randomization is useful to eliminate there are potentially confounding variables.
counfounding factors. Blinding is a procedure Therefore, researchers usually use stratification
to mask information about the treatment given; or multivariate methods to control confounding
while concealment is about how to conceal the variables in the analysis.
341
Murdani Abdullah Acta Med Indones-Indones J Intern Med
Table 4. List of questions for the results of clinical journal that statistical significance, which is usually
article4 demonstrated as p value <0.05, should always be
Components Explanation considered with its clinical significance.
Flow and Baseline -- How many subjects are In clinical trials, cohort and case-control
data randomized for each group? studies, there are usually two comparable groups
-- How many and what is the and the results are presented as a relative risk, i.e.
percentage of subjects that
completed the study?
a comparison of outcomes in the intervention and
-- How many and what is the
control groups. If the outcomes are measured as
percentage of subjects that did comparison of the occuring event (for example
not complete the study? the number of recovered subjects compared to
-- What are the reasons for those who does not in a group), then such relative
subjects being excluded from
the study?
risk is known as odd ratio (OR). If the outcomes
-- Are the baseline data
are calculated as a frequency, i.e. by comparing
comparable? the study subjects who experienced the event with
-- What had been done by the the total number of subjects in a group, then the
researcher when the baseline relative risk is defined as risk ratio (RR). When
data are not comparable?
there is no difference found between groups, the
Primary outcomes -- What are the primary
outcomes?
value of OR and RR is 1. Relative risk (either
OR or RR) greater than 1 means that the number
-- Are they statistically or clinicaly
significant? of events are greater in the intervention group
Secondary -- What are the secondary compared to the control group. In contrast, when
outcomes outcomes? (for example, side the relative risk is lesser than 1, the events in the
effect)
intervention group is lesser.
Results of the study would be more useful
when being presented as the risk differences.
Primary outcomes of the study provide
In this setting, we devide the proportion of
answers to primary research question; while
events in the control group with those in the
secondary outcomes answer the additional
intervention group. Risk difference is also
research question. Principally, when reading
usually demonstrated as the number needed
the results of each study, we should perform
to treat (NNT) expressing how many subjects
clinical and statistical interpretation. The
that should be treated during a period of time
statistical interpretation could have been done
to achieve an outcome (treatment) or to prevent
by evaluating the p value and confidence interval
an event (prophylaxis/protection from disease).
of each hypothesis test that has been performed
Epidemiologically, NNT characterizes of how
(usually if p <0.05, the relative risk does not lied
many patients that would need health care in
in the range of 1; proportional or mean value
the form of treatment to reduce the number of
does not located in the range of 0). Moreover,
cases, which is presented as a final endpoint. The
clinical interpretation refers to comparison
following Table 5 illustrates those parameters.
between the study outcome and effect size.
Outcomes may appear as the value of relative The Discussion
risk and proportional difference (if the outcomes In the discussion section, we will find
are categorical scale); mean difference (when information about the researcher’s steatement
the outcomes are numerical scale); it may also regarding validity, clinical significance and
appear as hazard-ratio and incidence difference applicability that related to the study results.
(for outcomes of survival). 4 Please notice However, the researchers are usually only discuss
Table 5. How to calculate odd-ratio, risk ratio, and number needed to treat
Number
Number Risk of Number
of events Odds
of Odd ratio cure Risk ratio Risk difference needed to
(Ex. of cure
patients (frequency) treat
recovery)
Intervention 1,000 150 150/850 150/850 150/1000 150/1000 150 _ 100 = 0.05 1 / 0.05 =
= 1.59 = 1.5 1000 1000
Control 1,000 100 100/900 100/900 100/1000 100/1000 (=0.05%) 20
342
Vol 44 • Number 4 • October 2012 Critical appraisal on journal of clinical trials
CONCLUSION
Critical appraisal is extremely needed as an
assessment of information found in scientific
journal articles before we are convinced to apply
the information as guidelines in clinical practice.
The level of reliability of a study tremendously
depends on the study design, which places
clinical trial as the highest in rank. Critical
appraisal evaluates all components in a research
article including the introduction, method, results
and discussion. A good knowledge about how
to evaluate each component is very necessary
to perform a good critical appraisal. However, it
should be considered that statistical significance
should be compared with clinical significance.
343