• Embed Doc
  • Copy Link
  • Readcast
  • Collections
  • 2
    CommentGo Back
Download
 
Introduction to Bayesian Statistics
Harvey ThornburgCenter for Computer Research in Music and Acoustics (CCRMA)Department of Music,Stanford UniversityStanford, California 94305February 19, 2006
Statistical approaches to parameter estimation andhypothesis testing which use
prior distributions 
overparameters are known as
Bayesian
methods. Thefollowing notes briefly summarize some important facts.
Outline
•
Bayesian Parameter Estimation
•
Bayesian Hypothesis Testing
•
Bayesian Sequential Hypothesis Testing
1
 
Bayesian Parameter Estimation
•
Let
y
be distributed according to a
parametric family 
:
y
∼
f 
θ
(
y
)
. The goal is, given iid observations
{
y
i
}
, toestimate
θ
. For instance, let
{
y
i
}
be a series of coinflips where
y
i
= 1
denotes “heads” and
y
i
= 0
denotes “tails”. The coin is weighted, so
P 
(
y
i
= 1)
can be other than
1
/
2
. Let us define
θ
=
P 
(
y
i
= 1)
;our goal is to estimate
θ
. This simple distribution isgiven the name “Bernoulli”.
•
Without prior information, we use the
maximumlikelihood 
approach. Let the observations be
y
1
...y
H 
+
T 
. Let
H 
be the number of heads observedand
T 
be the number of tails.
ˆ
θ
= argmax
f 
θ
(
y
1:
H 
+
T 
)= argmax
θ
H 
(1
−
θ
)
T 
=
H/
(
H 
+
T 
)
•
Not surprisingly, the probability of heads is estimatedas the empirical frequency of heads in the datasample.
•
Suppose we remember that yesterday, using the samecoin, we recorded 10 heads and 20 tails. This is one
2
 
way to indicate “prior information” about
θ
. Wesimply include these past trials in our estimate:
ˆ
θ
= (10 +
H 
)
/
(10 +
H 
+ 20 +
T 
)
•
As (H+T) goes to infinity, the effect of the past trialswill wash out.
3
of 00

Leave a Comment

You must be to leave a comment.
Submit
Characters: ...
morning1234

Those who know this subject, know how useful it is in any science. I have used Bayesian estimation for assessing diversity and obtained excellent results. There is nothing funny about my earlier comments except for ignorant

reply09 / 02 / 2009
jamesyu

I believe this was given as a handout for a lecture in one of Julius Smith's or Bosi's courses at CCRMA. I don't recall which one though.

reply11 / 09 / 2007
You must be to leave a comment.
Submit
Characters: ...