SMBI - Simulation Methods and Bayesian Inference 2023-2024

SMBI - TD1
Introduction to Bayesian Statistics

1. A coin is tossed three times and the sequence of heads and tails is recorded.

A) List the sample space.

B) List the elements that make up the following events: A = at least two heads, B = the
first two tosses are heads, and C = the last toss is a tail.
C) List the elements of the following events: Ā, A ∩ B, and A ∪ C.

2. The weather forecaster says that the probability of rain on Saturday is 25% and that the
probability of rain on Sunday is 25%. Is the probability of rain during the weekend 50%?
Why or why not?

3. If two events, A and B, are such that P (A) = 0.5, P (B) = 0.3, and P (A ∩ B) = 0.1, find the
following:

A) P (A|B)
B) P (B|A)
C) P (A|A ∪ B)
D) P (A|A ∩ B)
E) P (A ∩ B|A ∪ B)

4. Three brands of coffee, X, Y , and Z, are to be ranked according to taste by a judge. Define
the following events:

A: Brand X is preferred to Y .
B: Brand X is ranked best.
C: Brand X is ranked second best.
D: Brand X is ranked third best.

If the judge actually has no taste preference and randomly assigns ranks to the brands, is
event A independent of events B, C, and D?

5. During an influenza epidemic, we denoted that in a household of a mother, father, and two
children, the probability that the mother gets influenza is 10%. Furthermore, we see that the
probability that the father get influenza is also 10% while the probability that both mother
and father get influenza is 2%. For the children, we assume a probability of 20% that each
child will get influenza and a probability of 10% that both children get the disease at the same
time.

A) Are the events that mother and father get influenza independent of each other?
B) What is the probability that at least one child will get influenza?
C) What is the conditional probability that the father has influenza given the mother has
influenza?
D) What is the conditional probability that one child has influenza given that the other
child does not have influenza?

Prepared by Nhim Malai 1

SMBI - Simulation Methods and Bayesian Inference 2023-2024

6. Polygraph tests (lie-detector tests) are often routinely administered to employees or prospec-
tive employees in sensitive positions. Let + denote the events that the polygraph reading
is positive, indicating that the subject is lying; let T denote the event that the subject is
telling the truth; and let L denote the event that the subject is lying. According to studies of
polygraph reliability (Gastwirth 1987), the sensitivity is 88% and the specificity is 86%. Now
suppose that polygraphs are routinely administered to screen employees for security reasons
and that on a particular question, the vast majority of subjects have no reason to lie so that
prevalence is 1%. A subject produces a positive response on the polygraph. What is the
probability that the polygraph is incorrect and that she is in fact telling?

7. A doctor is called to see a sick child. The doctor has prior information that 90% of sick
children in the neighborhood have the flu, while the other 10% are sick with measles. Let F
stand for an event of a child being sick with flu and M stand for an event of a child being sick
with measles. Assume for simplicity that F ∪ M = Ω, i.e., that there is no other maladies in
that neighborhood. A well-known symptom of measles is a rash (the event of having which we
denoted R). Assume that the probability of having a rash if one has measles is P (R|M ) = 0.95.
However, occasionally children with flu also develop rash, and the probability of having a rash
if one has flu is P (R|F ) = 0.08. Upon examining the child, the doctor finds a rash. What is
the probability that the child has measles?

8. In a study, physicians were asked what the odds of breast cancer would be in a woman who was
initially thought to have a 1% risk of cancer but who ended up with a positive mammogram
result (a mammogram accurately classifies about 80% of cancerous tumors and 90% of benign
tumors.) 95 out of a hundred physicians estimated the probability of cancer to be about 75%.
Do you agree?

9. Suppose that X ∼ bin(n, p).

A) Drive the likelihood function of binomial distribution.

B) Show that the maximum likelihood estimate (mle) of p is p̂ = X/n
C) If n = 10 and X = 5, plot the log-likelihood function.

10. Suppose that X1 , X2 , ..., Xn are i.i.d. N (µ, σ 2 ).

A) If µ is known, what the mle of σ?

B) If σ is known, what is the mle of µ?

11. Consider an i.i.d. sample of random variables with density function

1 |x|
f (x|σ) = exp(− )
2σ σ
A) Obtain the likelihood function.
B) Find the mle of σ.

12. Let X1 , ..., Xn be an i.i.d. sample from a Poisson distribution with probability mass function

e−λ
f (x|λ) = λx
x!

Prepared by Nhim Malai 2

SMBI - Simulation Methods and Bayesian Inference 2023-2024

A) Obtain the likelihood function.

B) Find mle of λ.
1
C) Prove that the Fisher information is I(λ) = λ

13. Suppose that X1 , ..., Xn are i.i.d. with density function

f (x|θ) = e−(x−θ) , x≥θ

and f (x|θ) = 0 otherwise.

A) Obtain and plot the likelihood and log-likelihood functions.
B) Find the mle of θ. (Hint: Be careful, and don’t differentiate before thinking. For what
values of θ is the likelihood positive?)
C) Find the Fisher information.
14. Write a program in R or Python, which reproduces Figure 2.3 below from the Stroke study
example.

15. Write a program in R or Python, which reproduces Figure 2.7 below from ECASS 2 study
example.

Prepared by Nhim Malai 3

SMBI - Simulation Methods and Bayesian Inference 2023-2024

16. Consider you are doing a coin toss experiment. You are given a presumably unfair coin with
p(heads) = 0.80 from 20 coins tossed. You are now collecting new data and analyzing the
posterior by doing 10 coin tosses and getting 4 heads.

A) Choose the distribution for your prior and construct your posterior distribution.
B) In case no prior information is available, construct your posterior distribution.

17. Grogan and Wirth (1981) provided data on the wing length in millimeters of nine members
of a species of midge (small, two-winged flies). From these nine measurements (1.64, 1.70,
1.72, 1.74, 1.82, 1.82, 1.82, 1.90. 2.08) we wish to make an inference on the population mean
θ. Studies from other populations suggest that wing lengths are typically around 1.9mm and
must be positive. We also know that most of the probability (95%) is within two standard
deviations of the mean. Obtain and plot the posterior distribution of midge wing length.

18. The bar chart on the right shows the number of injuries caused by road accidents in Cambodia
from 2014 to 2019. We are interested in making inferences on the average of fatalities at year
level. The experts suggest that road accidents kill more or less 2,000 people every year.
Construct the posterior distribution of the average fatality.

Figure 1: Source: UNDP: Road Traffic Accidents in Cambodia

19. The negative binomial distribution

 
y+r−1 r
p(y|θ, r) = θ (1 − θ)y , y = 0, 1, ..., (0 < θ < 1)
r−1

expresses the distribution of the number of failures until r successes have been observed. Show
that a beta prior distribution combined with a negative binomial likelihood results in a beta
posterior distribution.

20. The exponential distribution is a candidate distribution for a positive random variable y.
Show that a gamma prior distribution combined with an exponential likelihood, results in a
gamma posterior distribution.

Prepared by Nhim Malai 4