How do you derive the Success-Run Theorem from the traditional form of

Bayes Theorem?

Taken from How do you derive the Success-Run Theorem from the traditional form
of Bayes Theorem? on StackExchange

1 Answer

Joe Blitzstein, Professor in the Harvard Statistics Department

First we need to state the problem, including its assumptions, precisely. Being clear
about the assumptions is important not only for understanding the derivation, but
also for understanding when the result is applicable.

We are studying the reliability of a product, by performing n trials. In each trial, the
product succeeds with probability p and fails with probability 1-p. Suppose that the
trials are conditionally independent given p (the distinction between conditional
independence and unconditional independence is crucial in statistics).

Let R be a desired reliability level, and C be the corresponding confidence level, in

the sense that, given the data, there is at least probability C that the true reliability
p is at least R. For example, if R=0.9, C=0.95, we want to be able to say that there
is at least a 95% chance that p is at least 0.9. Given that the product succeeds all n
times, how are R and C related?

Another assumption is still needed though: we have not assigned a probability

distribution to p, so what does P(p≥R)P(p≥R) mean? If p is a constant and does not
have a distribution, then this probability is either 0 or 1. So to reflect our
uncertainty about p, we will view it as a random variable (this is a Bayesian
approach). Assume that the prior distribution is p∼Unif(0,1)p∼Unif(0,1) (this is a
simple way of quantifying being very, very uncertain about the value of p).

The posterior distribution of p, given the data, is Beta(n+1,1). This follows from
Bayes' theorem, and more specifically from the fact that Beta is the conjugate prior
for the Binomial (I explain the Beta distribution and conjugacy in Lecture 23 of
Statistics 110: Probability).

So we want to find P(W≥R)P(W≥R) for W∼Beta(n+1,1)W∼Beta(n+1,1). The CDF of

W, evaluated at w in [0,1], is F(w)=wn+1.F(w)=wn+1. Thus,
C=P(W≥R)=1−F(R)=1−Rn+1,C=P(W≥R)=1−F(R)=1−Rn+1,

as desired.