Critical appraisal of clinical trials

Specific Types of Study

Randomised Controlled Trial (RCT) Population is randomly allocated to two groups One group is given a specific treatment or intervention On average the groups are identical because they are randomised and therefore any difference in the measured outcome is due to the intervention Specified follow up period and specified outcomes e.g. drug better than placebo; surgical procedure compared with sham

Randomised Controlled Trial (RCT)

Advantages
Allows rigorous evaluation of a single variable in a previously defined population e.g. a new drug. Prospective i.e. collect the information after you decide to do the study. Tries to disprove the null hypothesis Tries to eradicate bias because the two groups are identical. Allows for meta-analysis later.

RCTs are
Unnecessary if:
Clearly successful intervention Previous RCTs or meta-analyses

Impractical when:
Unethical to randomise Large number needed

Inappropriate when:
Looking at prognosis Looking at validity of diagnostic tests Looking at quality of care

Background (1)
Practicing physicians must rely on the literature to keep current on recent developments on new therapies as well as providing additional evidence on therapies which have been long used in practice
Accurate reporting of a clinical trial is important to aid the practicing physician in deciding to adopt a new therapy or modify therapies currently in use

Background (2)
Proposals for requirements for reporting of randomised trials JAMA 1994;272:1926-31 Ann Intern Med 1994;121:894-5

JAMA Editorial in 1995 suggests two groups produce a unified statement

Consolidated Standards of Reporting (CONSORT) JAMA 1996; 276:637-9

Common Elements of Reporting for All Trials

Population under study Therapy details Experimental design Patient accounting Quality control procedures Statistical analysis

Special Reporting Requirements Non-randomized trials Randomized trials

The Term Prospective Trial Is a Pleonasm

Trial used increasingly in combination with prospective A PubMed search prospective trial yielded 507 hits for the period 1999-2001. However, prospective is a pleonasm (superfluous) Cross-sectional trial!! Because all trials are prospective by definition; the only way to do a retrospective trial is for the investigator to travel back in time with a box of pills.
Letter in The Lancet, Sept 2002 by Martijn B Katan, Netherlands

Reporting in Clinical Trials

Describe the Plan Report the Results Confess to Problems Interpret Objectively (no spin!)

Reporting in Clinical Trials

The Published Paper
1.

Identify clinical investigators and institutions (experience, reputation)

2. Also, identify Sponsor (federal, industry) Data management team Statistical analysis team

Reporting in Clinical Trials

The Published Paper
3. Introduction & Background
a. Biochemical theory b. Animal work c. Phase I/II clinical studies d. Previous large clinical studies e. Other pharmaceutical analogues

Reporting in Clinical Trials

Definition of question

What was the primary question?

Clearly defined in advance

Reporting Clinical Trials

4. Methods Section
Outcome variables Eligibility criteria
Inclusion Exclusion

Randomization Procedures Sample size justification Treatment & Control

Randomization
Sequence generation
Method used to generate the allocation sequence, including any details about restriction
What does restriction mean?

Allocation concealment
Method used to implement the random allocation sequence

Implementation
Who generated the allocation sequence, who enrolled participants, and who assigned participants to their groups

Allocation concealment
A technique used to prevent selection bias by concealing the allocation sequence from those assigning participants to intervention groups, until the moment of assignment.
Allocation concealment prevents researchers from (unconsciously or otherwise) influencing which participants are assigned to a given intervention group.

Importance of allocation concealment

Unclearly concealed and inadequately concealed trials, compared to adequately concealed trials, exaggerated the estimates of an interventions effectiveness by 30% to 40%, on average.
Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408-412.

Reporting Clinical Trials

4. Methods Section (continued) Outcome assessment & blinding Measures of patient safety and adverse events Predefined Subgroups Data monitoring plan Analysis Plan summary

Reporting in Clinical Trials

Treatment groups
a. Definitions - Experimental - Control b. Dose escalation c. Withdrawal for toxicity

Reporting in Clinical Trials

5. Description of results a.Full accounting of all patients entered on trials - Completeness - LTFU - Withdrawal b. Comparison of treatment groups as assigned (ITT), baseline characteristics c. Simple comparison of primary outcome variables using means, proportions, graphs, with measures of statistical precision (SE's, P-values) d. Thorough analysis of side-effect data/adverse effects

Reporting in Clinical Trials

5. Discussion e. Adequate handling of the possible impact of missing values, dropouts, non-adherence f. Discussion, allowance for multiplicity in number of interim analyses, number of endpoints

Reporting in Clinical Trials

5. Discussion Consistency of results a. among investigators and centers b. Other independent studies of same drug or analogues c. Subgroups consistency d. Primary and secondary outcomes

Reporting in Clinical Trials

6. Conclusion
a. Brief summary b. Strengths/weaknesses consistent with data c. Generalizability d. Trade off in side effects - risk/benefit

THE LANCET
Double-blind trial of aspirin in primary prevention of myocardial infarction in patients with stable chronic angina pectoris
Clinical trials have demonstrated a prophylactic role for aspirin in myocardial infarction and in unstable angina pectoris. The Swedish Angina Pectoris Aspirin Trial (SAPAT) is the first prospective study of aspirin in stable angina.

After showing good tolerance of sotalol for at least three weeks the patients were randomised double blind to aspirin 75 mg daily (n=1009) or placebo (n=1026).

Lancet 1992;340:1421-25

Obstetrics & Gynecology

Bacterial Vaginosis: Treatment with Topical Intravaginal Clindamycin Phosphate

We tested topical intravaginal clindamycin phosphate at ac concentrations of 0.1, 1.0, and 2.0% in the treatment of Alternative 62 women with symtomatic bacterial vaginosis in a propective, randomized, double-blind, placebo-controlled efficacy practitioners use systemic metronidazole and common gastrointestinal side effects. apy, particularly for pregnant women, is highly able. This study examined the safety and

. . . prospective, randomized, double-blind, placebo-controlled trial.

Obstet Gynecol 1990;76:118-23.

British Journal of Obstetrics and Gynaecology

October 1991, Vol. 98, pp 980-987

(Patho)physiological Implications of Chronic Dietary Sodium Restriction During Pregnancy; a longitudinal prospective randomized study
Abstract: Objective--To study the possible pathophysiological implications of long continued dietary sodium restriction in pregnancy Design--Longitudinal prospective randomized study of the effects of a low sodium diet compared with unrestricted sodium intake in pregnacy. Setting--Academic Department of Obstetrics and Gynaecology at Sint Radboud Hospital, Nijmegen, The Netherlands.

Design - Longitudinal, prospective randomized . . .

Br J Obstet Gynaecol 1991;98:980-7

A Controlled Trial of Povidone-Iodine as Prophylaxis Against Ophthalmia Neonatorum

Abstract Background. Neonatal conjunctivitis Ophthalmic neonatorum) continues to cause blindness, because the agents used prophylactically to prevent this condition are not completely effective and are not widely available in many parts of the world. Povidone--iodine ophthalmic solution is an effective antibacterial agent with broad antibacterial and antiviral activity to which no bacteria are known to be resistant, and it is

Randomization was achieved by

rotating the three medications after each was used for a week.

NEJM 1995;332:562-6

March 1991, Vol. 98, pp 260-264

Stopping Smoking in Pregnancy: Effect of a Selfhelp manual in Controlled Trial

Summary. For medical reasons, encouraging women to stop smoking during pregnancy and post partum has high priority. Many smokers want to stop smoking but decline clinical treatment when it is offered. The aim of this study was to find a method which was accepted by a large number of smokers, had a high success rate and, at the same time,

Women were randomized . . . born on days 1-10 of every month formed the control group (n=231), and those born on days 1131 formed the treatment group (n=492).
Br J Obstet Gynaecol 1991; 98:260-4.

Obstetrics and Gynecology Vol. 77, No. 3, March 1991

Nifedipine in the Treatment of Severe Preeclampsia

We conducted a randomized clinical trial in which patients with severe preeclampsia between 26-36 weeks of gestation receive either nifedipine (1030 mg sublingually, then 40-120 mg/day orally; N= 24) or hydralazine (6.2512.5 mg intravenously, then 80-120 mg/day orally; N= 25)

We conducted a randomized controlled trial. . . Subjects were assigned to the nifedipine or hydralazine group according to the week of the month.
Obstet Gynecol 1991; 77:331-7

Effectiveness of antibiotic prophylaxis in preventing bacteriuria after multichannel urodynamic investigations: A blind, randomized study in 124 female patients
Am J Obstet Gynecol;1991;165:679-81

On completion of the procedures, the patients were randomly assigned to prophylaxis or nonprophylaxis groups according to hospital number. Both the physician and the nurse technician were blind as to which assignment the patient received. Patients in group A received nitrofurantoin 50 mg four times and phenazopyridine hydrochloride 200 mg three times for 1 day. Patients in group B received phenazopyridine hydrochloride only. The code was broken at the completion of the study.

Randomization Process
Proper Approach
Generation of Allocation Sequence Allocation Concealment Both
[Lancet 1990; 335: 149-153.]

4 Major General Medical Journals

49% 26% 15%

CONSORT (1)
Intent is to make experimental process more clear, flawed or not, so that users of the data can more appropriately evaluate its validity for their purposes
checklist figure available at www.consort-statement.org

CONSORT (2)
Widely adopted by medical journals
required by many from Jan 1, 1997

Available in six languages

SPECIAL COMMUNICATION

The CONSORT Statement: Revised Recommendations for Improving the Quality of Reports for Parallel-Group Randomized Trials
David Moher, MSc Kenneth F. Schulz, PhD, MBA

Douglas Altman, DSc

for the CONSORT Group

JAMA, April 18, 2001Vol 285, No. 15 1987

Dissemination
Major general & internal medicine journals endorsed CONSORT
Required authors to submit RCT reports using template

Editorial groups that have endorsed CONSORT

World Association of Medical Editors (WAME) Council of Science Editors (CSE) International Committee of Medical Journal Editors (ICMJE or Vancouver Group)

ACADEMIA AND CLINIC

The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Douglas G. Altman, DSc; Kenneth F. Schulz, PhD; David Moher, MSc; Matthias Egger, MD; Frank Davidoff, MD; Diana Elbourne, PhD; Peter C. Gtzsche, MD; and Thomas Lang, MA, for the CONSORT Group
17 April 2001 Annals of Internal Medicine Volume 134 Number 8 663

Participant flow through a randomized trial . . . important?

Revamped web site (www.consort-statement.org)

Reprints of statement and E & E document (copyright is in the public domain) Data bank of good and not so good examples of trial reporting Reference citations of new evidence

Does the CONSORT checklist work? Methods

4 general & internal medicine journals
3 endorsed CONSORT (BMJ, JAMA, Lancet) 1 did not (NEJM)

Hand searched all 4 journals for 1994 and 1998 (January to June), respectively Used number of CONSORT items, Jadad score and adequacy of allocation concealment
Trained 2 assessors

Does CONSORT work? More results

Total

Unclear allocation concealment change

(CI) 0.43 (-0.34, .20) 0.35 (-0.29, .99) ** 0.68 (0.14, .22) *** 0.45 (0.08, .82) Pre Mean 79 59 54 61

Journal

1994 N 14 29 28 71

1998 N 20 20 37 77

Pre Mean (SD) 2.07 (0.92) 3.00 (1.04) 2.75 (0.89) 2.72 (1.00)

Change (CI)
-29 (-62, 4) -14 (-43, 16) -24 (-48, 1) **** -22 (-38, -6)

BMJ JAMA Lancet

Total Adopters Total NonAdopter

26

37

3.12 (1.03)

-0.01 (-0.55,0.54)

69

-8 (-33, 17)

Moher D, Jones A, Lepage L, for the CONSORT group. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before and after evaluation? JAMA 2001;285:1992-1995.