EXAM STA 426

QUESTION 1

(i) The rationale of the Bayesian approach is to use probability as a measure of uncertainty or
degree of belief about an event or a parameter. The Bayesian approach allows us to update our
beliefs based on new evidence or data, using Bayes’ theorem. Bayes’ theorem describes how the
prior probability, which represents our initial belief, can be combined with the likelihood, which
represents the probability of the data given the parameter, to obtain the posterior probability,
which represents our updated belief after observing the data12

(ii) a) A prior probability distribution is the assumed probability distribution of an uncertain

quantity before some evidence or data is taken into account. For example, the prior could be the
probability distribution representing the relative proportions of voters who will vote for a
particular politician in a future election. The prior distribution reflects our existing knowledge or
beliefs about the parameter of interest3

b) A posterior probability distribution is the updated probability distribution of an uncertain

quantity after some evidence or data is taken into account. For example, the posterior could be
the probability distribution representing the relative proportions of voters who will vote for a
particular politician in a future election, after observing the results of a poll or a survey. The
posterior distribution incorporates the new information from the data and modifies our prior
beliefs accordingly4

(iii) Let y be a variable that has a Binomial distribution B(θ, n), where θ has U (0,1) distribution.
Using a prior distribution g (θ) =1, 0 < θ < 1, determine the posterior distribution g(θ/y) and
hence the posterior mean of θ.

The prior distribution g(θ) = 1, 0 < θ < 1 is a uniform distribution on the interval (0, 1). This
means that all values of θ between 0 and 1 are equally likely. The likelihood function of y given
θ is:

f(y∣θ)=(yn)θy(1−θ)n−y,y=0,1,…,n

Using Bayes’ theorem, the posterior distribution of θ given y is:

g(θ∣y)=∫01f(y∣θ)g(θ)dθf(y∣θ)g(θ)∝f(y∣θ)g(θ)=(yn)θy(1−θ)n−y

where the proportionality sign means that we omit the normalizing constant that makes the
posterior distribution integrate to 1. The posterior distribution g(θ|y) is a beta distribution with
parameters α = y + 1 and β = n - y + 1. This can be seen by comparing the functional form of the
posterior distribution with the general form of the beta distribution:

Beta(θ∣α,β)=B(α,β)θα−1(1−θ)β−1,0<θ<1

where B(α, β) is the beta function. The mean of the beta distribution is:
E(θ∣α,β)=α+βα

Therefore, the posterior mean of θ given y is:

E(θ∣y)=n+2y+1

This is also known as the Laplace rule of succession, which gives the probability of a success
after observing y successes and n - y failures56

QUESTION 4
QUESTION 5

a) A conjugate prior is a prior distribution that belongs to the same parametric family as the
posterior distribution, given a certain likelihood function. This means that the prior and the
posterior have the same functional form, but with different parameters. A conjugate prior is
convenient because it allows us to obtain a closed-form expression for the posterior, without the
need for numerical integration. It also helps us to interpret how the prior information is updated
by the data12

QUESTION 6

a) The probability of 3 wins for Bob and 5 wins for Alice in the first 8 rolls can be calculated
using the binomial distribution. The binomial distribution gives the probability of getting a
certain number of successes in a fixed number of independent trials, each with the same
probability of success. In this case, the number of trials is 8, the number of successes is 3, and
the probability of success is θ, which is the probability that Bob wins. The binomial probability
formula is:

P(X=k)=(kn)pk(1−p)n−k

where X is the random variable that counts the number of successes, k is the desired number of
successes, n is the number of trials, and p is the probability of success. Plugging in the values
from the question, we get:

P(X=3)=(38)θ3(1−θ)5

This is the probability of 3 wins for Bob and 5 wins for Alice in the first 8 rolls.

b) The posterior probability of Bob going ahead to win is the probability that Bob wins 6 times
before Alice does, given the information from the first 8 rolls. To calculate this probability, we
can use Bayes’ theorem, which is a rule that relates the conditional probability of an event to the
prior probability and the likelihood. Bayes’ theorem states that:

P(A∣B)=P(B)P(B∣A)P(A)

where A and B are any two events, P(A∣B) is the posterior probability of A given B, P(B∣A) is
the likelihood of B given A, P(A) is the prior probability of A, and P(B) is the marginal
probability of B. In this case, the event A is that Bob goes ahead to win, and the event B is that
Bob has 3 wins and Alice has 5 wins in the first 8 rolls. We can use the binomial probability
formula from part a) to calculate the likelihood and the marginal probability, and we can use the
geometric distribution to calculate the prior probability. The geometric distribution gives the
probability of getting the first success on the k-th trial, where each trial has the same probability
of success. The geometric probability formula is:

P(Y=k)=p(1−p)k−1

where Y is the random variable that counts the number of trials until the first success, k is the
desired number of trials, and p is the probability of success. Using these formulas, we can write
the posterior probability as:

P(A∣B)=P(B)P(B∣A)P(A)

=(38)θ3(1−θ)5+(58)(1−θ)5θ3(38)θ3(1−θ)5∑k=15θ(1−θ)k−1

=θ+(1−θ)∑k=15(1−θ)k−1θ∑k=15(1−θ)k−1

This is the posterior probability of Bob going ahead to win, given the information from the first 8
rolls. I hope this helps you with your assignment. If you want to learn more about probability
distributions and Bayes’ theorem, you can check out these web pages: Binomial Distribution
Calculator, Geometric Distribution Calculator, and Bayes’ Theorem Calculator. Have a nice day!
😊

QUESTION 7

a) Here are the definitions of the terms:

(i) A loss function is a function that measures the discrepancy between an estimated value and the
true value of a parameter or a variable. It is usually denoted by L and depends on the estimator and
the true value. For example, L(θ, θ̂) is the loss function for estimating θ with θ̂1

(ii) Bayes risk is the expected value of the loss function with respect to the posterior distribution of
the parameter. It is a measure of the performance of a Bayesian estimator or decision rule. It is
usually denoted by R and depends on the prior distribution and the loss function. For example, R(π, δ)
is the Bayes risk for using the decision rule δ with the prior distribution π2

(iii) Squared error loss is a type of loss function that is defined as the square of the difference between
the estimated value and the true value. It is commonly used for regression problems, where the goal
is to minimize the mean squared error (MSE) of the estimator. It is usually denoted by L(θ, θ̂) = (θ -
θ̂)^23

b) Here are the solutions for the sub-questions:

(i) To determine the beta(a, b) prior that matches her belief, we need to find the values of a and b that
satisfy the following equations:

The mean of the beta distribution is 0.5, which implies a / (a + b) = 0.5.

The standard deviation of the beta distribution is 0.15, which implies √(ab / ((a + b)^2 (a + b + 1))) =
0.15.
Solving these equations, we get a = 8.333 and b = 8.333. Therefore, the beta(a, b) prior that matches
her belief is beta(8.333, 8.333).

(ii) To determine the posterior distribution, we need to use Bayes’ theorem and the properties of the
beta and binomial distributions. The posterior distribution is proportional to the product of the prior
distribution and the likelihood function, that is:

π(θ|y) ∝ π(θ) p(y|θ)

The prior distribution is beta(8.333, 8.333), which has the following density function:

π(θ) = θ^7.333 (1 - θ)^7.333 / B(8.333, 8.333)

The likelihood function is binomial(68, θ), which has the following probability mass function:

p(y|θ) = (68 choose 21) θ^21 (1 - θ)^47

Multiplying the prior and the likelihood, we get the posterior distribution as:

π(θ|y) ∝ θ^28.333 (1 - θ)^55.333

Comparing this expression with the beta distribution, we can see that the posterior distribution is also
beta with parameters 29.333 and 56.333. Therefore, the posterior distribution is beta(29.333, 56.333).