Critical Assessment of Surgical

Outcomes and Health Services

Research
Critical appraisal
• is defined as process of carefully and systematically examining
research evidence to judge its
1. Validity
2. value, and
3. relevance to a particular context
• Decisions related to surgical care delivery should be made following a
careful as assessment of
• individual patient preferences
• clinical experience, and
• critical appraisal of the evidence in the medical literature
EVIDENCE-BASED APPROACH TO
SURGERY
• Surgeons must be able to understand the
• evidence-based medicine process in order to identify, access, apply, and
• integrate new knowledge into their clinical practice and
• provide high-quality care for their patients
• evidence-based medicine involves the following three fundamental
principles.
• ability to make optimal clinical decisions requires awareness of the best
available research evidence.
• there must be standards by which to judge whether evidence can be trusted.
• quality of evidence for benefit must be weighed against the risks, burdens,
and costs associated with alternative management strategies, while
simultaneously considering individual patients’ predicaments, values, and
preferences
• evidence-based approach to clinical practice can also be remembered
by the five A’s—
1. Ask
2. Acquire
3. Appraise
4. apply, and
5. assess These five steps provide a model for practicing evidence-
based medicine that all clinicians are encouraged to use when
questions arise during the routine care of patients
evidence-based approach to clinical practice
Evaluating Research Study Questions
• There are several strategies that surgeons can use to review the
literature, evaluate research study questions, and determine whether
scientific evidence should be applied to their clinical practice.
• One strategy for evaluating the implications of a research question is
outlined by the mnemonic FINER—feasible, interesting, novel, ethical,
and relevant.
• FINER provides a framework that prompts the reader to ask several
important questions when considering a research study
FINER
• Is the study feasible and adequately powered to answer the specific
research question?
• Does the study address a topic that is interesting to the surgical
community?
• Is the research novel or innovative?
• does the study meet all ethical standards of research conduct?
• Finally, do the results from the study change surgical practice or
policy, and do they merit further scientific research or evidence
PICOT
Another widely accepted approach to developing and accessing a
research question
structured way to summarize the main components of a research
question
also be used to appraise a study or interpret its results
• PICOT framework—population, intervention, comparison, outcome, and time
PICOT
• (P) Is the study population or clinical problem comparable with
surgical patients or issues that I deal with in routine clinical practice;
(I) Is the intervention truly novel, and does it represent an
improvement over current standards of care;
• (C) Are study comparators valid and representative of usual care;
• (O) Are outcome measures valid and reliable; and
• (T) Does the duration of treatment and follow-up correspond to the
duration important to patients?
PICOT
Study Design
Study Design
• There are many study designs used to undertake surgical research,
1. experimental and
2. observational designs
• study design used to generate evidence often influenced
by practical considerations, such as
resource availability and feasibility.
study design selected to evaluate surgical interventions will
largely determine our levels of confidence about cause and effect
Cont.…..
• Several criteria have been proposed to establish epidemiological
evidence of causal relationships, including the set of nine criteria
originally published by Sir Bradford Hill in 1965.
• Distilled down to the most simplistic terms, causal relationships exist
when
• (1) the cause precedes the effect
• (2) the cause was related to the effect, and
• (3) no other plausible explanation for the effect other than the proposed
cause
• Experimental studies that are prospective and randomized usually
provide the greatest insurance.
Randomized Controlled Trials
• When appropriately designed and conducted, randomized controlled
trials (RCTs) are considered the gold standard for evaluating most
types of health care interventions
• RCTs can be used to compare nearly all types of interventions used in
surgical patients. This includes surgical procedures, medications used
before or after surgery, screening programs, or diagnostic modalities,
to name a few
RCT-weaknesses
they are very expensive to complete
it can take many years to obtain results when large sample sizes are
needed.
results may not be generalizable to patients in everyday practice due to
narrow enrolment criteria.
many surgical interventions are simply not amenable to randomization.
can still yield biased results if the study is not well designed or executed
properly and
lacks methodological rigor.
Cross-Sectional Study
• CSS involves looking at research subjects who differ on one key
characteristic at a specific point in time.
• Exposure and outcome data are collected at the same time point
among subjects who are similar in most other characteristics but
different in a key factor of interest.
• For example, this could involve patients undergoing a specific type of
surgery who come from different age groups, income levels, or
geographic locations.
CSS
• Cross-sectional studies are commonly used to describe disease prevalence or
characteristics that exist in a community or hospital setting.
• inexpensive
• often used to make inferences about possible relationships
• to gather preliminary data to support further research and experimentation (i.e.,
hypothesis generating).
Limitations
• bias by survival
• cannot analyse associations over a longitudinal period of time
• Cannot be used to determine cause-and-effect relationships between exposure and outcome
variables.
• sampling bias if the timing of the cross-sectional study leads to a sample that is not representative of
patients in the general population.
Case-Control Study
• The case-control design uses a different sampling strategy
• investigators identify a group of individuals who exhibit a specific
outcome (i.e., the cases)
• compare this group to a set of individuals who do not exhibit the
outcome of interest (i.e., the controls).
• The cases and controls are then compared with respect to the
frequency of one or more past exposures.
• If the cases have substantially higher odds of exposure to a particular
factor compared to the control subjects, it suggests an association but
does not necessarily provide evidence of causal inference.
CCS
• strengths of case-control studies
• allows investigators to examine risk factors associated with outcomes
that are uncommon
• to examine the association of multiple risk factors with outcomes
simultaneously, and
• to examine outcomes that occur a long period of time after the
exposure occurs.
• They are also relatively inexpensive to complete and can help
generate hypotheses.
CCS
• limited by
• their retrospectively design that allows the potential for recall and
selection bias
• identifying valid control groups to compare to cases can be difficult
• estimating the frequency of exposure in the population at-large is
often inaccurate
• allows evaluation of only one outcome at a time
Cohort Study
• A cohort study is an observational study design
• groups of subjects are identified based on their exposure to a
particular risk factor and then compared to a group that have not
been exposed to that same factor.
• This study design can be conducted from either a forward-looking
(i.e., prospective) or backward-looking (i.e., retrospective) viewpoint.
CS
Prospective cohort studies
• planned in advance carried out over a period of time
• assess outcome incidence among exposure groups.
retrospective cohort studies
• look at data that already exist and
• Attempt to identify risk factors for outcomes that have already
occurred.
CS
• For both types of cohort studies, a higher incidence of outcomes in the exposed
group suggests an association between that factor and the outcome.
• However, because prospective studies collect information about exposures and
outcomes purposefully and systematically, they provide stronger evidence for
causation.
• There are several strengths and limitations of cohort studies.
• The advantages of cohort studies are that they are easier and less expensive to
conduct than RCTs, the incidence (or rate) of exposure and outcomes can be
estimated, and subjects in cohorts can be matched to limit the influence of
confounding variables.
• In addition, enrolment criteria and outcome measures can be standardized, and—
unlike case-control studies—multiple simultaneous outcome assessment is possible.
CS
• However, cohort studies also have several disadvantages.
• Because there is no randomization, one cannot account for unmeasured
imbalances in patient characteristics.
• Blinding or masking is difficult (or impossible retrospectively), and outcomes of
interest can take a long time to occur, requiring many years of information about
exposures.
• For retrospective studies in particular, treatment selection bias and confounding
variables may lead to unmeasured differences in exposure
• groups over time that cannot be controlled for with statistical analysis
• Moreover, interpretations can be limited because of missing data that is
impossible for researchers go back in time to collect.
Case Series and Reports
• describe surgical interventions and patient outcomes.
• These have been the most highly cited type of study design in the
surgical literature until the past couple of decades.
• Observations are made on a series of individuals, usually all receiving
the same surgical intervention, but with no control group.
• The sampling of a case series is based on either exposure or outcomes,
but not both.
• For example, a series could include all patients who underwent surgery
at a single institution over a decade, or a case report might describe
patients who experienced some rare adverse event after surgery.
Case Series and Reports
Strength
• ability to provide in-depth narrative information on a given topic
• may help to generate new hypotheses.
Main limitation
• lack of ability to generalize findings
• lack of ability to establish a cause and effect relationship between exposures
and outcomes.
• cannot be comparative and no absolute risk nor relative effect measures for
an outcome can be calculated.
• nearly always considered low quality evidence.
Summery
Synthetic and Systematic Outcome Studies
• Meta Analyses
• One of the highest forms of evidence quality comes from a met
analysis
• where the results from multiple studies on the same topic are
combined.
• data generated from either experimental or observational study
designs
• have the statistical power to detect the true association.
• meta analyses allow study results to be generalized to larger
populations
• permit inconsistency of results between studies to be quantified and
analysed, and identify the presence of publication bias between
similar studies
factors that can limit the quality of
evidence derived from meta analyses
• Often, studies examining a specific topic do not have comparable
patient populations, endpoints, or exposure groups.
• Excess variation or heterogeneity in exposure or outcome variables
between individual studies beyond what chance alone would predict
indicates that the results of the studies are not compatible and should
not be pooled
• Even a well-designed meta analysis cannot correct for poor study
design or intrinsic bias that was present when the included studies
were originally designed or analyzed
Systematic Reviews