6/24/2013

THE MOUSE TRAP?

March 8, 2013
PRESENTOR: ED MON D M AR ZB AN I, MD

DISCUSSANT: GREG DEL ZO P PO , MD

INTRODUCTION INTRODUCTION

 Animal experiments have  How often do animal experiments predict success in humans?
contributed significantly to our
understanding of mechanisms
of disease.  There is exceptionally limited data describing the frequency of
discordance between animal studies and human trials.
 However, their ability to
predict effectiveness of
treatment strategies for use in
clinical trials is unclear and
has remained controversial for
many models.

 Clinical trials are essential

because animal studies do not
predict with sufficient
certainty what will happen in
humans.

1

INTRODUCTION
 Highly cited systematic review performed to assess how often
high impact animal studies translate into successful human
research.
 Investigators searched 7 leading journals by impact factor b/w
1980-2000 (Science, Nature, Cell) that regularly publish animal
studies. Articles with > 500 citations were retrieved under the
assumption that prominent findings would be tested in human
trials (n=2000).
 Articles were included if they investigated a preventive or
therapeutic intervention in an in vivo animal model. If multiple
animal studies of the same inter vention were found, most cited
study was retained (n=76).
Hackam and Redelmeier. Translation of Research Evidence from Animals to Humans. JAMA 2006; 296: 1731-1732.
LOST IN TRANSLATION
 Multiple reasons for the failure to translate findings from
animal models to clinical trials.
 Internal validity refers to the extent to which design and
conduct of the trial eliminate possibility of bias.
6/24/2013
 For each included study, literature search identified human
studies that translated the animal evidence.
 “Successful” translation was defined as replication in a RCT resulting
in statistically positive results according to primary outcome.
 Of the highly cited animal studies, 37% were replicated in
human randomized trials.
 Of the interventions, 8 or one-tenth of the inter ventions were
subsequently approved for use in patients.
 Successful translation rate likely to be significantly lower.
Hackam and Redelmeier. Translation of Research Evidence from Animals to Humans. JAMA 2006; 296: 1731-1732.
LOST IN TRANSLATION
 In clinical trials, several systematic reviews and meta-
analyses have demonstrated that inadequate methodological
approaches in controlled clinical trials are associated with
bias.
 However, in animal studies, the impact has been examined
less extensively.
2
 6/24/2013

LOST IN TRANSLATION LOST IN TRANSLATION

 In a metaanalysis of animal studies investigating  External validity refers to the extent to which the results of an
interventions in emergency medicine, odds of positive result animal experiment provide a correct basis for generalizations
were more than three times as large if the publication did not to the human condition.
report randomization or blinding compared with publications
that did. 1  Reductions in external validity:
 models with diseases or injury that are dissimilar to the human
condition;
 In systematic review of animal models of hypothermia for  induction of disease under study in animals that are young and
acute ischemic stroke, an inverse relation was found between healthy while patients are elderly and with comorbidities;
study quality as assessed by a study-quality checklist and  homogenous group of animals versus heterogeneous group of
effect size. 2 patients;
 use of male or female animals only;
 treatment time course or doses that are unrealistic in humans;
 differences in outcome measures and timing of outcome assessment.

1Bebarta et al, 2003

2Van der Worp et al, 2007

LOST IN TRANSLATION LOST IN TRANSLATION

 The Inflammation and Host Response To Injur y, Large Scale  However, attempts at publishing these findings resulted in a
Collaborative Research Program was funded to study early common objection that the researchers had not shown the
inflammatory response to serious injuries. same gene response had happened in mice.

 Investigators per formed genome-wide expression analysis on  “They were so used to doing mouse studies that they thought
white blood cells from hundreds of burn, trauma, or sepsis that was how you validate things. They are so ingrained in
patients and found patterns of gene expression that predicted tr ying to cure mice that they forget we are tr ying to cure
outcome. humans.”

 “That started us thinking,” he continued. “Is it the same in the

mouse or not?”

Kolata, Gina. (2013, February 11). Mice Fall Short as Test Subjects for Humans’ Deadly Ills. New York Times. Retrieved from
http://www.nytimes.com/2013/02/12/science/testing-of-some-deadly-diseases-on-mice-mislead-report-says.html

3
 6/24/2013

PATIENT ENROLLMENT AND SAMPLING

 “Murine models have been extensively used in recent decades  Between 2003-2009, 167 patients admitted to US Level I
to identify drug candidates for subsequent human trials but Trauma Center (1 of 7) with blunt injury associated with SBP <
few have shown success.” 90, or HCO3 deficit > 6 mEQ/L, transfusion requirement < 12
h, Abbreviated injury score > 2 had blood samples taken
within 12 h and 1 , 4, 7, 14, and 21 d after injury.
 Authors discuss that of 150 clinical trials evaluating agents
intended to block inflammator y response in critically ill
patients, every single one has failed.  Between 2000-2009, 244 burn patients were enrolled if
admitted within 96 h of injur y, TBSA > 20%, and required at
least 1 excision and grafting procedure and had blood draws
 Goal was to compare the genomic response seen in human
up to 1 year after injury.
inflammatory diseases and a commonly used murine model.
 Prior to the paper, no studies had evaluated on a molecular basis
how well the murine clinical models mimic human inflammatory
diseases in patients.

PATIENT ENROLLMENT AND SAMPLING MALE MURINE INJURY MODELS

 Eight healthy male and female subjects between 18-40 years  All mice were male C57BL/6J.
of age received either IV endotoxin E. coli 0113 or 0.9% NS
and blood samples were collected at 0 hours, 2, 4, 6, 9, and  Groups of 12 mice undergoing injury were giving anesthesia
24 hours after infusion. with isofluorane and then subjected to 25% TBSA scald burn
or trauma/hemorrhage. T/H consisted of laparotomy followed
by withdrawal of sufficient blood from arterial line to
 In addition, 35 healthy control subjects were recruited decrease and maintain MAP at 45 mm Hg for 90 minutes.
between 2004 and 2007.
 After burn injury, mice were resuscitated with 1 mL NS IP.
After T/H, mice were resuscitated with Ringer’s at 4x shed
blood volume.

 Mice receiving LPS (10 ng E. coli 0113) were not

anesthetized.

4
 6/24/2013

METHODS RESULTS

 Total blood leukocytes were  5,544 genes were identified as significant between patients
isolated and total cellular RNA and healthy subjects and 4,918 of these had mouse
was extracted and hybridized orthologs.
onto an Affymetrix GeneChip.

 Baseline represented mean  Among the 4,918 genes, correlations of maximum gene
gene expression controls. changes (Pearson correlation, R 2 ) and percentages of genes
Time course data of the measured expression level of a representative
Maximum fold change (fold changed in the same direction in all six conditions were
gene. On the x-axis, points of samples of controls are at 0 hour and change) refers to maximum compared.
points of patient samples are displayed according to their sampling time
after injury. On the y-axis, each point shows the log2 fold change of deviation from baseline.  The Pearson correlation is +1 for direct linear relationships, -1 for
expression value. Maximum fold change (MFD) represents largest
deviation of the trend line from base level. Respond time and Recovery
time are defined as time when trajectory reaches half the maximum fold
inverse relationships, and and 0 for no relationship.
change before and after reaching the maximum respectively.  Response time was defined as  By random chance, 50% of genes between two uncorrelated
time to one-half maximum conditions are expected to change in the same direction.
fold change. Recover y time
was defined as time from
baseline to one-half of
maximum.

RESULTS RESULTS

 High correlation in gene  Investigators next evaluated whether genes with the greatest
response between human fold change demonstrated higher correlation between human
trauma and burns (0.91, 97%)
injuries and mouse models.
and moderate similarity
between human endotoxemia
and burn/trauma (0.47, 88%).  Correlations increased slightly to R 2 =0.11-0.28 among the top
105 genes with the greatest magnitude of changes (FC>6) in
 Low correlation in gene contrast to the top 4,918 genes (R 2 =0.00-0.09).
response between all three
murine models of injury
(R<0.13, 39-63%).

 Low correlation in gene

response between all human
injuries and mouse models
(R<0.09, 47-63%).
Scatter plots and Pearson correlations (R2) of the log twofold changes of
4,918 human genes responsive to trauma, burns of endotoxemia and
their murine orthologs.

5
 6/24/2013

RESULTS

 Reverse analysis was

performed with the 1 ,519
murine genes with FC > 2
that had human orthologs
(87%, 1 ,318).

 Investigators randomly
sampled comparable small
numbers of humans and
mice and showed
correlations remained high
Among the 4,918 human genes changed in human trauma, burns, of endotoxemia, 105 had FC between human injuries and
> 6 in human burn. Shown are bar graphs of Pearson correlations (R2) between human trauma,
endotoxemia, murine models versus human burns as a reference. low compared with mouse
conditions.
Scatter plots and Pearson correlations (R2) of the log twofold changes of
1,519 murine genes responsive to trauma, burns of endotoxemia and
their human orthologs.

RESULTS

 Next, investigators sought to evaluate differences in both

response and recovery time of gene expression between the
models.

 In all conditions, gene response time occurred within the first

6-12 hours.

 However, recovery times were significantly different.

 In murine models, recovery occurred within hours to 4 days;
 In humans, recovery time was 1-6 months.

Box plots of response and recovery times of gene expression in human

burns, trauma, and endotoxemia as well as corresponding mouse models.

6

RESULTS
 Major signaling pathways
regulated in human injuries
were identified and
compared across models
with human burns as a
reference.
 innate immunity upregulated
and adaptive immunity
downregulated.
 Evaluation of the top 40
pathways demonstrated
median R 2 of 0-0.16 and
gene percentages of genes
changed in the same
direction of 52-67%.
Pathway comparisons between the human burns, trauma, and endotoxin
and mouse models. Shown are bar graphs of Pearson correlations (R2)
for five most activated and suppressed pathways between the four model
systems with human burns as a reference.
RESULTS
 Genomic responses in
humans correlated well
with each other with R 2
from 0.47-0.91 and % of
genes changing the same
direction from 83-97%.
 In mice, R 2 ranged from
0-0.08 with 47-61% of
genes changing in the
same direction.
Comparison of the genomic response to severe acute inflammation from
different etiologies in human and murine models.
6/24/2013
RESULTS
 Investigators evaluated
additional patient and
corresponding mouse model
studies from Gene
Expression Omnibus (GEO)
for other acute inflammatory
diseases.
 Maximum fold changes from
the data sets were compared
with the 5,554 genes
significantly changed in the
three human injuries to
calculate correlations and
directionality of gene
response.
Studies available in GEO on severe acute inflammatory diseases in
humans and murine models. R2 represents Pearson correlation.
Percent represents percentages of genes changed to same direction
between datasets.
DISCUSSION
 “Studying disease in patients is much more complex than
studying model systems.”
 This may be true but it may be that model systems and human
disease are equally complex. The issue may be that model
systems and disease are just dissimilar.
 Two broad points of the article.
7

DISCUSSION
 Critically ill patients who suffer dif ferent injuries often have
similar appearing physiologic reactions, a condition known as
Systemic Inflammatory Response Syndrome (SIRS).
 An concept central to drug testing for this syndrome has been
that the molecular mechanisms underlying it are similar
regardless of the etiology.
 High correlation in response between trauma, burns, and
endotoxemia in humans despite heterogeneity tends to support this
hypothesis.
DISCUSSION
 The authors stress the point that the evolution of the immune
system for any species is a consequence of microbe-exerted
selection pressure for that species.
 In endotoxin model, mice are highly resilient where lethal
dose is 5-25 mg/kg for most strains of mice, whereas a dose
1 ,000,000-fold less (30 ng/kg) causes shock in humans.
6/24/2013
DISCUSSION
 Genomic response of peripheral WBC in the C57BL/6J mouse
strain poorly reflects human inflammatory diseases.
 Why might this be the case?
DISCUSSION
 This difference in sensitivity may result in genomic responses
that reach an upper threshold in human disease, whereas in
mice it doesn’t reach threshold and results in greater
heterogeneity in response.
 It is possible that a higher level of injur y in mouse models
would result in a transcription response that actually mimics
the response seen in patients.
8

DISCUSSION
 Multiple other potential reasons that the C57BL/6J model
poorly mimics human injury
 evolutionary difference between mice and humans,
 complexity of human disease,
 inbred nature of the mouse model,
 differences in cellular composition between mouse and human
tissues can contribute to differences seen in the molecular response;
1Reutershan
lung injury. Anesthesiology. 2006 Mar;104(3):511-7.
DISCUSSION
 While the reasons for the differences are potentially
interesting to study, ultimately this paper suggests that
C57BL/6J is a poor strain for drug development for these
inflammatory conditions.
 This is not irrelevant because C57BL/6J is often used to study
inflammatory conditions.
6/24/2013
DISCUSSION
 Multiple other potential reasons that the C57BL/6J model
poorly mimics human injury
 similarities in cellular composition between mouse and human
tissues are not reflected in peripheral WBC genomic expression
patterns,
 different temporal spans of recovery from disease,
 late events related to clinical care of patients (fluids, drugs, surgery)
may alter genomic responses not captured in murine models,
 differences in gender,
 isofluorane may have suppressed the immune response 1 ,
et al. Protective effects of isoflurane pretreatment in endotoxin-induced
DISCUSSION
 How to proceed?
 The authors note that a practical approach forward would be
to require greater validation of animal models.
 Studying whether a model mimics or fails to mimic the
molecular or cellular behavior of key genes and pathways
relevant to human disease would be critical to advance
models for drug development.
9
 6/24/2013

DISCUSSION DISCUSSION

 New approaches  “As a cornerstone of modern biomedical research, the use of

 outbred mice with greater genetic heterogeneity; mouse models has dominated scientific literature. The
 comprehensive genomic descriptions in patient studies to define prevailing assumption—that molecular results from current
human disease which could then help guide development of animal mouse models developed to mimic human diseases translate
models; directly to human conditions—is challenged by this study.”
 may require use of other organisms that model human disease more
accurately, although still require validation;
 tissue chips – miniature 3D organs made with living human cells
 NIH is dedicating $70 million over 5 years
 “humanized” mice.

REACTION REACTION

 “For decades, mice have been the  "The study’s investigators tried for
species of choice in the study of more than a year to publish their
human diseases. But now, researchers paper... They submitted it to the
report evidence that the mouse model publications Science and
has been totally misleading for at least
three major killers — sepsis, burns and Nature…reviewers did not point out
trauma. As a result, years and billions scientific errors but “the most common
of dollars have been wasted following response was, ‘It has to be wrong. I
false leads, they say.” don’t know why it is wrong, but it has
to be wrong.’”
 “The good news for mice is that
humans have spent billions of dollars
to solve their illnesses. But it seems
researchers have tortured mice in vain
for decades in the search for drugs to
help humans recover from certain
traumas...”

10
 6/24/2013

REACTION REFERENCES

 If these results are validated, how should they change 1. Bebar ta et al. Emergency medicine animal research: does use of
randomization and blinding af fect the results? Acad Emerg Med.
practice? 2003 Jun;10(6):68 4-7.
 PETA 2. Hackam and Redelmeier. Translation of research evidence from
animals to humans. JAMA 2006; 296: 1731-1732.
 Jackson Labs 3. Mestas and Hughes. Of Mice and Not Men: Dif ferences between
Mouse and Human Immunology. J ournal of Immunology 200 4; 172:
2731-2738.
 “Of mice and not men” not a novel concept, yet this article 4. Reutershan et al. Protective ef fects of isoflurane pretreatment in
has promoted a great deal of discussion. endotoxin-induced lung injur y. Anesthesiology. 2006
Mar;104(3):511-7.
 Is it because the use of this technique is new and more compelling 5. Seok et al. Genomic responses in mouse models poorly mimic
than previous comparisons? human inflammator y diseases. PNAS Epub Feb 201 3.
 Is it because the narrative the authors constructed – investigators 6. van der Worp et al. Hypothermia in animal models of acute
ischaemic stroke: a systematic review and meta-analysis. Brain.
losing sight of human disease, bias in the peer review process – is a 2007 Dec;1 30(Pt 1 2):3063-74.
compelling one? 7. van der Worp et al. Can animal models of disease reliably inform
human studies? PLoS Med 2010; 7(3): e1000245.

QUESTIONS?

11