Research Methodology

and
BestBets

Dr. Frank Gama

MBBS, MSc (Derm), MSc(AesMed).

1 1
Sackett et al. defined EBM as the integration of clinical expertise and
patient preferences with the best available research evidence to make
decisions about patient care.

2
We can learn from others and their collective experiences of
treating hundreds and thousands of patients, and integrate
that with our own.

Clinical judgement is therefore based on a larger sample,

has greater validity, and has more statistical power.

3
Judging the best available research is a matter of critical appraisal.

4
 Clinical Research: Basic concepts

• Research is a process of finding things out and filling gaps in our

knowledge and understanding.
• It provides an organised systematic way of answering questions.
• It involves gathering, analysing and interpreting data to answer
the questions.
• The value of a research study is determined by the size of the gap
in knowledge which is filled.

5
Research studies are mainly two types:

I) Primary studies:
a) Experiment.
b) Clinical trials.
c) Observational studies.
d) Surveys.
e) Psychometric studies.

II) Secondary studies:

a) Reviews.
b) Systemic Reviews.
c) Meta-analysis.

6
 Study Designs

a) Case Reports: These are from clinical observation of cases with clinical/scientific
interest.

• They are not a method for answering research questions.

• They can contribute to identification of new diseases, treatments, or adverse
effects of treatments.
• They should not be used as a basis for changing medical practices.

7
b) Case Series: these are collection of data from a number of case reports
or subjects.
• Easy to carry out, and may be used to explore whether a proposed
research study is likely to be successful.
• But they are highly susceptible to bias arising from the way the cases
were obtained, and findings should be treated with caution.

8
c) Surveys: A systematic way of gathering information, and widely used in research.
It generally involves collecting data of those included in the sample.

The main biases in a survey are:

•Non-coverage bias.
•Non-response bias.

9
d) Cohort Studies: When two (or more) groups of people (cohorts) are
selected based on differences to their exposure to a medicine or toxin,
and are followed up to see the proportion in each group who develop a
particular outcome.

Advantages:
• Good for estimation of incidence and progress of disease.
• Good at examining several possible outcomes.

Disadvantages:
• Time consuming and need to follow-up over prolonged periods.

Main biases in Cohort Studies:

• Selection bias.
• Losses to follow-up.

10
e) Case Control Studies: Done by comparing a group of patients who have a
have a treatment/disease being studied with a control who do not.

• The controls should match the cases as much as possible for factors such
as age, gender, socioeconomic status etc.
• Data should be collected from both groups over the study period using
same methodology.

Advantages:
• Efficient approach to answer a research question.

Disadvantages:
• Can be difficult to find matching controls.
• Prone to bias, such as selection bias, measurement bias and confounding.

11
Clinical Trials is a method for assessing the effectiveness of health care
interventions.

Types of trials:
a) Preclinical trials: These are laboratory based and usually consist of
standard animal toxicology studies.
b) Phase I trials: Designed to estimate the maximum tolerated dose or
safety of the drug, before proceeding to phase II trials.
c) Phase II trials: To establish the clinical effectivity, short term side effects,
risks associated with increasing doses of the drug.
d) Phase III trials: Randomised comparision studies studies of a new drug
against a currently standard drug. Once phase III studies completed the
sponsor applies to authorities for approval.
e) Phase IV trials: Conducted after drug approved for marketing. Done to
provide additional information like different patient populations, other
stages of disease, or use of the drug over a longer period of time.

12
 Designing a Clinical Trial

1) Clearly defining the patient group.

2) Sample Size. A small sample size may fail to detect a treatment
effect which could be regarded as clinically significant.
3) Randomisation.
4) Blinding:
- Single-blinded.
- Double-blinded.
- Triple-blinded.

13
So what can make a clinical trial weak ?

• Poorly defined patient groups.

• Failed randomisation.
• Inadequate sample size.
• Poor blinding.
• Poor outcome measures.
• Poor compliance with treatment.
• Patients lost to follow-up.
• Poor analysis or interpretation of the data.

14
Advantages of RCT:
• Allows rigorous evaluation of a single variable in a precisely defined
patient group.
• Minimises bias by comparing two otherwise similar groups.
• Prospective design.
• Allows for meta-anaylsis.

Disadvantages of RCT:
• Expensive.
• Time consuming.
• Surrogate endpoints often used instead of clinical outcomes.
• May still have hidden bias.

15
Systematic Review: The scientific evaluation of several primary studies on
a specific research question. They are considered the gold standard
source of evidence because the evidence is based on pooling together
results from several carefully selected studies.
Biases in Systematic Reviews:
• Selection bias.
• Observer Bias.
• Publication bias.

16
Hierarchy of Study Designs.

17
Part of determining the best available research evidence is to categorise research
into the appropriate level of evidence (LOE) as defined by the American Society
of Plastic Surgeons.

18
LOE is only a guide to judging a study design’s relative quality and bias.
High LOE can be badly conducted and reported.
Conversely low LOE studies may still reliably answer the research
question they set out to.

The onus is on the reader to develop critical awareness.

19
Critical Reading. Why is this important?

• You might assume that all research papers accepted for publication
are reliable. Don’t believe everything you read!
• There are several famous cases of fraudulent research fooling the
research establishment.

eg: Leading South Korean stem cell research scientist Dr Hwang Woo-Suk,
who faked his research on cloned human stem cells.(Cloners Disgrace,
New Scientist, Jan 2010)

20
How do you evaluate the literature ?

This is a vast topic but one can start with:

• Is the research question worth asking?
• Is the question the correct one to ask?
• Who sponsored the research?
• Are the methods appropriate to the research question?
• Are the conclusions justified given the data and the methods?
• Are the groups appropriately matched?
• Were the groups treated equally to minimise bias at every stage?
• Are the outcomes objective and measurable?
• Is the follow up long enough?
• Were the limitations of the study discussed ?

21
EBM means not only to read scientific papers, but to read the right
papers at the right time, and then to alter our behaviour in the light of
what we have found.

The practice of EBM involves five steps:

1) To convert out information needs into answerable questions.
2) To search for the best evidence.
3) To appraise the evidence critically, assess its validity, and clinical
applicability.
4) To implement the results in our clinical practice.
5) To evaluate our performance (audits).

22
 BestBETs

The BETs system was set up at Manchester Royal infirmary in 1996, as a

systematic approach to answer real world questions.

It allows working clinicians to answer the important questions in their

own clinical practice.

23
Formulating the three-part question:

1) Patient Group.
2) Intervention or defining question.
3) Relevant Outcomes.
- clinically relevant.
- patient orientated.
- definable.
- measurable.

24
Write down your search strategy.

• Which databases you searched for.

• What dates were searched for.
• Which interface (eg Pubmed, Ovid, etc).
• What terms were used in each database.
• If papers were found through other means (e.g. references on
another paper).

25
You now have a long list of papers that may or may not be relevant to the
three-part question.

You need to decide which papers to include in the BET using the
following criteria.
1) Relevance (focus on the three part question, and can be done from
reading the abstract/title).
2) Quality of the paper (we’ve covered this already, but advisable to
use critical appraisal checklists from www.bestbets.org ).
3) Range of paper types: take papers from the highest level you find +
papers from one level below.
If there is a Cochrane Systematic review, use this and any papers
published after it.

26
Collate the data in a table mentioning
• Author, date, country.
• Patient Group.
• Study type, LOE.
• Outcomes. This should be directly related to the outcomes of the
three part question.
• Study weakness
• Comment on real world clinical implications such as costs, training,
resource limitations, etc.

Clinical bottom line: This is ideally a brief statement that directly answers
the three-part question.

27