Why does

Science keep
changing its
mind about
things?
BIOL 210
Week 5
First, a few updates
 A note on class safety during a
pandemic…
• The safety steps we are taking including requiring masks at all times and
increased ventilation wherever possible (this room has mechanical
ventilation)
• Immunization remains the best protection against catching or
transmitting SARS-CoV-2 and provides excellent protection against
serious illness or death after infection
• Your instructors and classmates might be or might live with people who
are more susceptible to Covid-19
• For these reasons, we encourage everyone in the class who has not been
fully vaccinated to do so as soon as possible, unless you are unable to
• Classes and tutorials will be recorded. If you feel unwell, please stay
home and watch these on myCourses rather than risk coming to campus
when you are ill.
But what about when videos do not
work?
• There is still an audio recording, and a transcript Can listen or read
audio (transcript) and look through slides at the same time.

• Is there a student in the class who would share notes with students
when they get sick? Ideally a few so that those students can also stay
at home if/when they get sick. Contact me!
 Reminder about upcoming deadlines
• This Wednesday– Friday (Sept 29-Oct 1) Quiz 2 (on weeks 3 & 4)

• October 15 (just over 2 weeks away) Science Communication Essay

• Next week: Teams will be assigned for Team projects

• Following week (Thanksgiving & fall break): no class entire week

• Anytime before the end of term: Scientific Seminar. Don’t put it off!

• If you are ill: do not come to class. If you cannot take a Quiz, remember that the
lowest grade is dropped.
 A question about clinical trials

Who are these people?

Who would participate in Phase 1
trial?
Here are some (Covid vaccine) volunteers
Why does
Science keep
changing its
mind about
things?
BIOL 210
Week 5
Self-correcting Science
“There are many hypotheses in
science that are wrong. That's
perfectly alright; it's the aperture to
finding out what's right. Science is a
self-correcting process. To be
accepted, new ideas must survive
the most rigorous standards of
evidence and scrutiny.” – Carl Sagan
But remember him?
But remember him?
But remember him?
 Publication bias

Journal articles typically

get bound into yearly or
monthly books that are
then kept in perpetuity in
Universities.
But remember him?
But remember him?
Publishing negative results is possible!
Let’s take a recent example
Hydroxychloroquine

• Treatment for Lupus, malaria,

Rheumatoid arthritis
• Side effects are common (nausea,
diarrhea, itching, anemia, liver
failure & more)
• WHO list of Essential Medicines

Hydroxychloroquinine
Hydroxycholorquine is approved for
treatment of Covid-19 in Canada

A. True
B. False
Why did we think of using it for Covid-19?
• In vitro means that this was
done in cell culture (derived
from monkey kidneys), not in
people
• Infect cells with SARS-CoV-2
(virus that causes Covid-19) and
measure the amount of virus
produced with and without
Hydroxychloroquine
• Observe that
Hydroxychloroquine reduces
virus more than untreated cells
So that is promising, right?
• Would be akin to a pre-clinical study from a traditional clinical trial.
But that usually lasts years, and there would be multiple studies. This
is one study, so rather preliminary.

• We are living through a pandemic, however, so publishing something

that is preliminary is not necessarily wrong, so long as the limitations
of the study are understood
Then what happened?
Open-label means that
there was no blinding.

Non-randomized means
that there was no
random sorting of
participants. The controls
were the people who
refused treatment (and
some from another
hospital)
Their findings did look promising

Short follow-up (6 days)

PCR test might also have
false negatives & false
positives

We’ve seen this before! Low N’s

N = 16 N = 20 Could be a sampling error
Take care about Type 1 & 2 errors
(False positives & false negatives)
Remember our red flags?

1. Were experiments 3. Were all the results

performed blinded? presented?

Some of the patients were not included:

1 left the hospital before the endpoint
4. Were there appropriate 1 stopped treatment because of nausea
controls?
1 died
 *
Was it wrong to publish this?
Remember, we were in the middle of a global pandemic with little
treatment to offer patients

Study was flawed but nonetheless appeared promising.....

Follow up was needed

* Let’s come back to this question….

More research was done
 June 2020

• 1446 patients (1376 remained in study)

• Mean follow-up period was 22.5 days. Looked at death

rate, intubation rate

• Hydroxychloroquine patients were more severely ill at

the beginning than those that did not receive treatment
(patients chose whether they took or not. Not random
or blinded).

• Hydroxychloroquine did not cause different death rate

or intubation rate than non-treated patients
 Aug 2020
• Random, double blind

• Patients who had been exposed to Covid-19 but did not have disease
at the time of signing up (across US and Canada)

• No difference observed in the number of patients who were positive

• Hydroxychloroquine-taking patients had more serious side effects

• More than one hospital (so any variability due to location should be reduced)

• Study done on Covid-positive, sick patients in hospital (97 control, 97 treatment)

• Randomized, control study

• No differences in mortality rate or rate of intubation

No apparent effect on sick patients

• What about patients who are not as sick? Does it prevent sickness?

• Remember, the original study in cell culture showed reduction in viral

production, so maybe it is effective in preventing severe illness, but cannot
reverse it
• Randomized, control study

• Recently-diagnosed patients (293 in total: 157 control, 136 treatment)

Nasal swab to determine viral load

Also Found

• No difference in disease duration

• No difference in risk of
hospitalization
What about a meta-analysis?
• Remember, these usually take years, not months!

• But pandemic times mean that research goes more quickly! There are
already several meta-analyses
Article highlights:

• While initial studies with hydroxychloroquine showed some ray of hope, recent studies
that have emerged found either no benefit or a possible harm in COVID-19.

• This meta-analysis showed no benefit on viral clearance, although a significant increase in

death was observed with hydroxychloroquine in patients with COVID-19, compared to the
control.
Was it wrong to publish this?
Remember, we were in the middle of a global pandemic with little
treatment to offer patients

Study was flawed but nonetheless appeared promising.....

Follow up was needed

Not everyone agrees with that.

Same journal as
original article
was published in
(perhaps trying to
right a wrong?)
 In-depth critique of original paper
• Treatment & control groups from different centres (differences in population, age,
health were not controlled for or well reported)

• Treatment group of 26: 6 were excluded (of these, 3 ICU, 1 died). Extreme selection
bias. Imagine if 25/26 died, and in the remaining 1 showed virus was undetectable.
According to researchers of original study, this would count as being 100% successful.

“This study suffers from major methodological shortcomings which make it nearly if not
completely uninformative. Hence, the tone of the report, in presenting this as evidence
of an effect of hydroxychloroquine and even recommending its use, is not only
unfounded, but, given the desperate demand for a treatment for Covid-19, coupled with
the potentially serious side-effects of hydroxychloroquine, fully irresponsible.”
Not all critique was as harsh
• No finger pointing
• Using this example as an opportunity to study how to best conduct
non-randomized studies
There’s a lot to learn, and science
changes as the world does
Time to publish (remember, which may itself not be accurate)
Cardiovascular research

Covid-19 papers

2019 all papers

Self-correcting Science
“There are many hypotheses in
science that are wrong. That's
perfectly alright; it's the aperture to
finding out what's right. Science is a
self-correcting process. To be
accepted, new ideas must survive
the most rigorous standards of
evidence and scrutiny.” – Carl Sagan
 Because of methodological flaws, the first non-
randomized hydroxychloroquinone and Covid-19
paper should not have been published

A. True
B. False
 Why or why not?
Rank Responses
Is Hydroxychloroquine and Covid-19
an example of Science self-correcting?
• Yes

• This is how science works, and the process shows

(at least to me) that science is working.

• It may be hard to understand from the outside.

Many non-scientists find it hard to understand
why “science keeps changing its mind”. This can
lead to conspiracy theories and pseudo-science
(some world leaders disagreed with their own
governmental scientists about the interpretation
of these very studies)
Conclusions
• Science is not perfect, but with time it should (and usually does) self-
correct

• Hydroxychloroquine’s lack of treatment efficacy for Covid-19 is a good

example of Science’s ability to self-correct

• It may be hard for non-scientists to understand the differences between

different methodologies and thus why some studies are weighted more
than others

This contributes to the public communications crisis facing science.