METHODOLOGIES OF REAL WORLD STUDIES

Rizaldy Pinzon, MD, PhD

RS Bethesda Yogyakarta/ MMR FKKMK UGM
Email: drpinzon17@gmail.com
REAL WORLD EVIDENCE

¡ Real world data : is data collected from sources

outside of traditional clinical trials.
¡ These sources may include large simple trials, or
pragmatic clinical trials, prospective observational or
registry studies, retrospective database studies, case
reports, administrative and healthcare claims,
electronic health records, data obtained as part of a
public health investigation or routine public health
surveillance, and registries.
¡ Real world evidence: is the evidence derived from
aggregation and analysis of RWD elements.
Schematic flowchart of medicine/ device-development

Submission

Approval
non-clinical PMS Industry-initiated clinical
Clinical trials (post marketing
studies (feasibility/Pivotal)
surveillance)
registries

literature review Investigator-initiated

clinical registries

lsafety and efficacy data for lfocused on safety data in lsafety and efficacy data in
regulatory approval real world practice real world practice
PIVOTAL STUDY FOR REGULATORY APPLICATION

¡ Pros.
¡ controlled
¡ high level evidence
¡ Cons.
¡ limited population
¡ high cost

“real world data” is help to understand the medical front

activities
EVIDENCE GAP

Little to No
Evidence

Off-label indications
Conceptualization Approval

Evidence
Clinical Trials Evidence Post-Marketing Studies Utilization

Variances in population
characteristics from what
was studied Little to No
Differing age groups – elderly, pediatrics Evidence
Race, ethnicity, gender variances
Unstudied co-morbid conditions
Varying severity of disease
Differing concomitant medications
Varying levels of compliance – i.e. < 80%
CONTOH

¡ The risk for statin-induced myopathy:

¡ RCTs ∼3% of patients developed myalgia
¡ This condition was reported by 13% of the users in prospective clinical studies
(RWE)
¡ The 4-fold increase :
¡ The short duration of RCTs
¡ The selection criteria might have excluded older patients
¡ The use of other medication
CHALLENGE : NEEDS OF REAL WORLD EVIDENCE

¡ Need to demonstrate comparative effectiveness

¡ Cost effectiveness – Payer's willingness to pay
¡ Clinical effectiveness (long term efficacy and safety) – CER research
and patient reported outcome for decision making by healthcare
providers and patients
¡ Other related issues:
¡ Lack of standardization of data (EHRs/EMRs ) and analytical methods
¡ Uncertainty in data quality and completeness in EHRs/EMRs
¡ Uncertainty in regulatory environment
CHALLENGE: LONG TERM SAFETY

¡ Some drugs have been withdrawn from

the market because of unexpected
adverse effects that were not detected
during Phase III clinical trials and were
only apparent from long-term
surveillance data from the wider patient
community
 Emergence of the Payor in the Decision-Chain & the need for
Industry to develop products that will get reimbursed

To Innovate To Approve To Pay for To Prescribe To Adhere

Health Plan
Industry FDA Government Physician Patient

Effectiveness Population Total Cost of

Cost
Clinical

Safety

Evidence Safety/Long- Care Impact

Demands term Safety
Efficacy SAE & Overall Direct cost of
Cost
Clinical

Safety

Evidence Tolerability product

Used acquisition
1
1
DECISION-MAKING PERSPECTIVES
DECISION MAKING PERSPECTIVES

Pharma Healthcare Providers

• Understand the market • Maximum treatment
• Differentiate the product safety and effectiveness
• Address data gaps • Reduced treatment
costs

Payers/HTA Patients
• Evidence of positive • Minimal side effects
clinical, humanistic and • Cure the disease
economic outcomes • Improve quality of life
• Affordable care
Payers Pharmas

Proactive
D&I Pharmacovigilance PHC Drug Repositioning
/New Indications EBM
CER

Cost
Patient recruitment Effectiveness
¡ Real world data is useful to understand safety and efficacy
under real-world practice

Sources
Registries Large Simple Trials
observed cohorts of patients with diseased observed trials with large populations but
or treatment less cohorts than RCT

Supplement of RCTs Retrospective database studies

supplement data from RCT retrospective analysis to utilize medical
database

Medical records Others

review of medical charts of patients
TYPE OF REAL WORLD STUDIES

¡ Observational study: non-interventional clinical study designs that are not considered clinical trials
¡ Retrospective observational studies identify the population and determines the exposure/treatment
from historical data.
¡ Prospective observational studies identify the population of interest at the start of the study, and
exposure/treatment and outcome data are collected from that point forward.
¡ Pragmatic trials: clinical study designs that include elements that closely resemble routine clinical
practice
¡ Historical controls: Using data generated prior to the initiation of the study as a comparator
Randomized controlled trial

Real world use

Pragmatic trial
REAL WORLD EVIDENCE OPTIONS

¡ Pragmatic clinical trial

¡ Modified RCT
¡ population enrichment
¡ cohort multiple RCT
¡ comprehensive cohort study
¡ cluster RCT
¡ Evidence synthesis, such as NMA
¡ Modelling to predict outcomes, re-weighting trial data
Endocrine Reviews, 2021, Vol. 42, No. 5 661

Figure 1. Data sources used by RWE studies. EHR, electronic health records; NDI, National Death Index; PRO, patient-reported outcomes; Adapted
from: Franklin JM, Glynn RJ, Martin D, Schneeweiss S. Evaluating the use of nonrandomized real world data analyses for regulatory decision making.
Clin Pharm Ther 2019;105:867–77.

a mix of various data sources. Researchers then embed Perceived Barriers to Using RWE for Decision
studies within those longitudinal data collections. Making
Endocrine Reviews, 2021, Vol. 42, No. 5 663

1
Examples:
External control arms for single
Primary arm trials, RCT enrichment, RCT
approval RWE long-term follow-up

2
Examples: Indica!on expansion,
popula!on expansion (pediatrics,
Secondary stage), surrogate to clinical
indica!ons RWE endpoint claim expansion

3 Ini!al Approval Full Approval

Examples:
Biomarker to clinical endpoint,
Accelerated efficacy claim expansion,
Pathways RWE broadened popula!on

4 RWE Examples:
Safety (a) a) Post-market requirements
(PMR) or commitment (PMC),
b) Rapid regulatory response to
Safety (b) RWE a safety signal

Figure 2. Contributions of RWE for regulatory and coverage decision-making. Adapted from: Franklin JM, Glynn RJ, Martin D, Schneeweiss
S. Evaluating the use of nonrandomized real world data analyses for regulatory decision making. Clin Pharm Ther 2019;105:867–77.

Professional societies have agreed on the RWE study of RWE in healthcare still needs to be fully established
parameters that need to be disclosed in detail in order (44). Reviews comparing pairs of previously published
Endocrine Reviews, 2021, Vol. 42, No. 5 671

Downloaded from https://academic.oup.com/edrv/article/42/5/

Figure 8. The prediction of the CAROLINA RCT by a RWE study.

as Suissa pointed out (21), that if patients died on an oGLD enumerable. This is not the case in many community-based
they never had the chance to move on to an SGLT2 and or hospital-based case–control studies, where the true
were never categorized as such; in turn all oGLD users who underlying source population remains unknown.
switched to SGLT-2s did by definition survive their oGLD There are several good reasons, even in database studies
RWE THROUGHOUT THE LIFECYCLE
THE RWE JOURNEY
CONCLUSION

¡ RWD studies can provide information on a wider patient population, but an existing RWD source may
have some inherent bias that could limit its value for drawing causal inferences between medical
device exposures and outcomes.
¡ Careful study design, a study protocol and analysis plan should be created prior to accessing, retrieving,
and analyzing RWD, regardless of whether the RWD are retro- or prospectively collected
¡ Protocols and analysis plans for RWD should address the same elements that a traditional clinical trial
protocol and statistical analysis plan would cover.