Letters

Making sense of prior probabilities in research
Lex (ALB) Rutten1, Robert T. Mathie2, and Raj K. Manchanda3
1

Homeopathic physician and independent researcher, Aard 10, 4813 NN Breda, The Netherlands
British Homeopathic Association, Hahnemann House, 29 Park Street West, Luton LUI 3BE, UK
3
Central Council for Research in Homoeopathy, Ministry of Health and Family Welfare, Government of India; Secretary for
Research, Liga Medicorum Homoeopathica Internationalis (LMHI), 61–65 Janak Puri Institutional Area, Delhi 110058, India
2

In a recent article, Gorski and Novella state that prior
probabilities can be so low that putting them to the test
makes no sense [1]. A few decades ago the randomised
controlled trial (RCT) was demanded because of the low
prior probability of clinical methods such as homeopathy.
Interestingly, the mention of prior chance and its updating using Bayes’ theorem arose after a considerable
number of RCTs of homeopathy were subjected to
meta-analysis; this concluded that the results were not
compatible with the placebo hypothesis for homeopathy
[2]. Extremely low priors, based on theory, are now
proposed as sufficient reason to disregard scientific evidence.
It is stated that a prior probability is infinitesimally low
if it ‘violates well-established laws of physics and chemistry’. Quantum mechanics seemed at one time to violate
well-established laws; it did not overturn all existing
knowledge, but supplemented it. The Copernican worldview was once ridiculed: the daily turn of the world would
cause a devastating wind. Nowadays homeopathy is ridiculed because sometimes ‘there is no molecule in it’, though
it is now importantly recognised that nanoparticles may
remain [3]. We cannot yet conclude which mechanism
might explain homeopathy, but over 2000 basic research
experiments indicate an effect of ultra-molecular dilutions
(http://www.carstens-stiftung.de/hombrex/index.php).
Prior chances must be updated by new evidence using
Bayes’ theorem. This is not a one-step process, and all
possible evidence should be used for sequential updating
[4]. In sequential updating, the posterior odds after
one piece of evidence serve as prior odds for the following
calculation based on new information. Bayes’ formula
is:
Posterior odds ¼ likelihood ratio  prior odds

(1)

where odds = chance/(1 – chance), and chance = odds/(1 +
odds). Evidence from RCTs can be translated into a
likelihood ratio (LR). Rosendaal and Bouter assigned
one significantly positive RCT an LR of 16 [5]. Their
estimate of the prior that homeopathy ‘works’ was 1 in
106. Sequential updating after eight positive trials generates the results shown in Table 1. Very low priors are
updated surprisingly quickly by Bayes’ formula [6]. In
Corresponding author: Rutten, L. (lexrutten@gmail.com)
Keywords: prior chance; homeopathy; Bayes’ theorem.
1471-4914/
ß 2014 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.molmed.2014.09.007

eight trials the probability of efficacy increased from one
in a million to more than 99%.
The only existing comparative review of homeopathy
and conventional medicine showed similar indications of
efficacy, with better quality in the homeopathy trials
[7]. The funnel plot of homeopathy trials shows no difference from the comparable plot of conventional trials. Most
spots on the graph indicate beneficial effects for both
homeopathy and conventional medicine, and the median
sample sizes are the same (65.5 vs 65.0). Gorski and
Novella, however, stated that the conclusions of the review
were negative or inconclusive. This was based on testing a
post hoc hypothesis of different efficacy of ‘larger trials of
higher methodological quality’, with unspecified subsets of
eight homeopathy and six conventional medicine trials.
These subsets appeared to be incomparable and highly
heterogeneous [8]. The negative conclusion depended on
various subjective choices.
Before RCT evidence in complementary and alternative
medicine (CAM) became available, this type of research
was requested by opponents of CAM. Declaring now that
this evidence is pseudoscientific, based only on plausibility,
is suggestive of post hoc reasoning. Is there perfection in
the evidence-based medicine (EBM) paradigm, based on
basic science, cell culture, in vitro studies, animal models,
and clinical trials? Ezzo concluded, after reviewing
160 Cochrane reviews, that 43.8% of conventional treatments were (possibly) efficacious [9]. Daily reality illustrates that CAM helps many patients after conventional
medicine has failed [10]. It is unlikely that the present
physicochemical model is complete; how unlikely, therefore, is a future explanation for homeopathy?
A low theoretical prior is a questionable argument
for rejecting CAM trials. The theory is imprecise; it is
not about whether homeopathy acts in the same way as
Table 1. Updating beliefs in eight steps after new evidence:
the posterior chance after the first evidence is the prior chance
before the second evidence, and so on [6]
RCT number
1
2
3
4
5
6
7
8
a

Prior chance
0.000001
0.000016
0.000256
0.004079
0.061505
0.511856
0.943748
0.996289

Posterior chance
0.000016
0.000256
0.004079
0.061505
0.511856
0.943748
0.996289
0.999767

LR+ a
16
16
16
16
16
16
16
16

LR+, likelihood ratio for positive results.
Trends in Molecular Medicine, November 2014, Vol. 20, No. 11

599

but reflects the possibility of another mechanism of action. MI 48201. 473–476 Trends in Molecular Medicine November 2014.1016/j. and quality bias does not explain this finding. Geneeskd. Tijdschr. such as the theory of relativity or evolution..J. 20. References 1 Gorski. 1471-4914/ ß 2014 Published by Elsevier Ltd. complementary and alternative medicine. Homeopathy 97. 685–708 4 Gill. but rather ‘a well-substantiated explanation of some aspect of the natural world that can incorporate facts. J. and de Craen. Moreover. MI 48201.10.org/10. No. BMJ 330. F. A.H. 214–219 7 Shang. and Novella.R.M.H. Barbara Ann Karmanos Cancer Institute. (2005) Why clinicians are natural bayesians. Ann. D. Med. we argued that prior plausibility. (2001) Alternative medicine: a ‘mirror image’ for scientific reasoning in conventional medicine. 457–466 10 Marian. 146. and Rutten. it demonstrates that extremely low priors are consecutively increased by new positive evidence.) S5. Assess. when used correctly. L. and tested hypotheses’ (http://www. http://dx.B. Ned. (2014) Clinical trial of integrative medicine: testing whether magic works. are now proposed as sufficient reason to disregard scientific evidence’. It is a complaint that is. [1]. when Corresponding author: Gorski. (2002) Errors in methodology (conclusion). To accept the possibility that homeopathy works because of an as-yet unestablished different mechanism of action is more compelling than to ignore the experience of patients. can be sufficient reason to conclude that clinical trials are so unlikely to be informative that carrying out such trials would be both unethical and scientifically unproductive. quantum theory was necessary to explain compelling observations that the then-current theory could not account for. A. Lancet 366. Technol. Altern. USA The response of Rutten et al. Wayne State University School of Medicine. Novella3 1 Michael and Marian Illitch Department of Surgery.edu) Keywords: evidence-based medicine. and an incorrect interpretation of Bayes’ theorem applied to clinical trials of homeopathy. A. J. J. (gorskid@med. in which we contend that it is unscientific and unethical to test highly implausible treatments such as homeopathy and reiki in randomized clinical trials (RCTs). The extremely low prior probability of homeopathy is not based solely on theory. Suite 6C. (2001) Reviewing the reviews. 4100 John R St. (Schol. Vol. They mischaracterize our core argument as stating that ‘extremely low priors. Detroit. leaving room for additional mechanisms of action that could fill gaps in medicine. Prior chance combined with Bayes’ theorem cannot be used as a one-step method to interpret scientific evidence because. inferences. Health Care 17. A. 8.2 and Steven P.2014. unjustified and relies on a misunderstanding of the history of science. Ed. G. 40 Temple St.Letters conventional medicine. Clin. Med. Contrary to how it is portrayed by Rutten et al. Existing RCT evidence seems to be incompatible with the placebo hypothesis.molmed. S. 1080–1083 5 Rosendaal. BMC Complement. The statement that accepting homeopathy would overthrow existing knowledge is unscientific. there are no such compelling observations that can only be explained by homeopathy. 3990 John R St. based on theory. (2005) Are the clinical effects of homeopathy placebo effects? Comparative study of placebo-controlled trials of homeopathy and allopathy.001 600 so low as to be indistinguishable from zero (as is the case for homeopathy). yet nowhere did we write that prior plausibility alone is sufficient reason to disregard scientific evidence. J. D. et al.L.L. (2008) Patient satisfaction and side effects in primary care: an observational study comparing homeopathy and conventional medicine. 61. 1197–1204 9 Ezzo..M. 135. 726–732 8 Lu¨dtke. laws. Patients experiencing effectiveness after conventional medicine has failed should be taken seriously. A theory. The conventional paradigm is incomplete. and Bouter. Med. Biosci. I. after all. (2013) Adaptive network nanomedicine: an integrated model for homeopathic medicine. basic research. The ultimate truth.E. science-based medicine. Detroit. et al.. 304–309 (article in Dutch) 6 Rutten. and Schwartz. Currently. New Haven CT 06510. Yale University. A scientific theory. Wayne State University. represents a common and misguided complaint by advocates of alternative medicine against interpreting clinical trials through the lens of prior plausibility.P.R. is accepted by the community of relevant scientists based on overwhelming observational and experimental evidence supporting it.edu/openbook. F. from a scientific viewpoint. USA 3 Department of Neurology. Front. is not merely a hunch. 507–513 3 Bell. in both theory and evidence. Epidemiol.J. et al. a straw man characterization of arguments for science-based medicine. to our recent article [1].php?record_id=6024&page=2).wayne. that conclude that homeopathy is about as close to impossible as one can imagine are so well established. Rather. (2008) The conclusions on the effectiveness of homeopathy highly depend on the set of analyzed trials.P. XXXIX. as implied by Rutten et al. Trends Mol. C. integrative medicine. USA 2 Molecular Therapeutics Program. Intern. R. (2008) How can we change beliefs? A Bayesian perspective. et al. 52 Considering prior plausibility in clinical trials does not mean ignoring scientific evidence David H. and RCT evidence. 11 2 Vandenbroucke. 20.nap. rooted in 200 years . How strong is the evidence? How clear are the conclusions? Int.. clinical trials. the underpinnings of physics and chemistry. Gorski1.doi.