aASSMAte

Dat: 2/9/2022
Page

Baye's heorem-
In heodis cussion of conditional pobabiIty
ven tndicokes Hhat i v i s i n probabiliy ohen e .
inkrmochio obtain is Cm mprtont Phse
ot b: omdlysí30fter we begum ur ouMayafs
wih MiHo or Pmor prebabiliies eiimote
Speciticevent fwterest Thes tro SDurce Such
C mple a special report produc tes OUM
So Om We bbta Some additiona mfr maHo
about the event G i v e n ths ne m t r nahom e
update
update iorpmb. values by calculatiyreye pba
blities rettered to as pesterir pzobabil+hes
Bayetherenm ohich wagive by Th0mas Bcyess
aBriHsh Mothe maticiam in 1963 poyjdes a memA
forrmaking thí prbeubiit calutatima. The steps t
h3 bability Yevision p ces are shd wn i n _he
h
followjha diagom

Prior Ne Applicahom t| |Pas teriur

probabi1,ties nfrwahion Bauyes1hm| p bab)h

Bayes hm
Statement'> If EL E2 Enare mutua d3-
0itevens wth P(E:) # o
L 2,- n Then
oM arr bi tran even ALOhich S a Subset
Ei St CA) >0. We have P (Ei (A) = PEi) PCA
IPE:).P
PlA Ei)
P(E JR) PCE). PCAJ Ei) P(Ei) P (AEi) E/2-n
2 PCEi) P (hEi) P CA)
ohere, CEs), PEa), PEn) are ter m ed aA he
pnpr pod babi Hes
 PA|Ei) i1,2, n are
cal ed telhd be ca use
be caue
they dr po vidiw
thuy
indicate infurwnathin brz
trey
ho uo
litely the event A
cons iderahhon
qiven each Omd Ocu mdar
OCCUr
Umdpr
omd every o privr po
eyer PmU p
be cauae
cauae are Coll ed
be thuydre determined posteriY p bakii,
expenment are knon ofter th
the resuut
resuwt
EqThe probdbiliHes o x, ,2 be conai mano4s dre
Yesp The pro babilities thaE
bonus Scheme
a LO be
MamAqer dre 3lto,uhroduced
i 3 2 beco
thibo bo w /2 1S respechively
what the
scheme
has been inhroduteo
pmbability tha the wana qer appoinua
let E E2 Ea denotes eyenh_ that
managers res pechvel K2beconw
3

A - denotes the even- that Bonu

intoduced Scheme is
Given
PCE) PCEa) 2 PCE3) =
P(A E) 3 o PCAEa) Ph| Es)
P(En A) PLEDPA JE) 3
PIA)
2 PEi). P(P] Ei); i: 1,2

P(EA)= 12 b

4/go
PEA) 6
23

SR sB
6R B
R B
To find the prob hatsecd b bal is drcwon
henthah un s V
e t E, E2 E3 denotees Yes p

P CE) /3 PE): V3,PCEs) 3

P (AE) = PAEa) /6 PCAE) = Y3
/S
PCE ) : 63
2to
An
 9122
SubjecHve CUnd quen Hstic
Probability has Vaiowm Coprobabilit
notahtns
=
Hotesi caly it ha been (onneched ih both
personaevaluation
Ox
other
o
ncertinity a a

predichius abou
decisim making under wncertainty 40mli
Ond
LOcertam prportion of occurance ot
OCCUrance of SDrne
event Thus, wben a
probabiliy 2thatthat personSyS The
this
turnupa head theni will
pariuer Loin
wil
that in
be oually meon
.megm
wany tosses
about half t timt
will be
head But it Can alo meon mt
het it
Somecn pus thi bet n head t
he vins head
othe loss a dller The
gMable 3 fur
stiwher pre.tahion s requenHSH fhe setmd
bsubfecive Similayone COU have both
nter predahm wid hen a oeathey urecash
Says thene i a probability of 60-. o rain
but the Subjechive
belps
inerpreBahon matters mr
pu D dedde tuDu LMI_ talee
wwbrlle
Hnalyone Can think up
Stuahons
edionc poatiular candidate o
ucess ot pochcular suden is a
es where only poritula
3/20222 valid. Hhesubjechive interpre tahbn P

Classical Inferene Model

ltt usConsider fer Th pblen otelh
ahon ot reaparameter eis unknoun in nar
Daka x a et
T(X) = eshmatoY
a AuDd esHmatr T(x) oTCx)-Bf shoul b
 Small o equ valent ly (TIR) 0) should be Snall
-

Classi cal stais H averoge this quauhy over

ohole
Lohole Sample space (se of al posible values
x)
R(T) E ( T) - )

Note A Bayesiam won't do this he sne wne

she
COndider daa depend measure af pertr wana
an H m a t Some thinq tha depends o porth l a r
data in
in hand, the data he has dbservd e
he has paiular data, that he ha coserv ed. He i
net qo)ng to average over al other possible values
data ha he had nat observeRd

Baesiam Inference Model

Bayesi om Model The pupose of stahshical analys)s
fundamertally inversion Purpae since i a m
its re Hving The ceurse redue to pardLwMeter
Th probabilisic g enerating mechanmS m trom the
the

ettect summai2e by The observahion

In
Ln other wds. when observing a randa
phhoMenon directed by porameter stahs ica me-
Thod allow to reduc ftom tbese observahn t e r
-ene Ctbout A Genera discripHen of inverdicn ot
pro babiliies IS given by Baye's togiluula
I A E d re events sh Such that pröbabihy
of A shold not equal to zero PCE) #0 Then prob-

ability P(h/E) S PCEA) dre related by.

P(AE) PEA) P (A)
P(EJA) P(A) + P(EA) PLA)
 Date
Page

This resuult
descri bes th
A from PA) to updahnq Of iteli hotd ef
observed PLAE) one E has been
Thomas Baye's actualY
version ot result proved a Continuouw
wih Conditiena maìnly. given
qiven to XX amd
Md y
disibuhion Fx) 9 wMargm
9N) The (ondithonal
dishribution t
margnd
9Ix) Y)X s
9(Ix)=F({x|): qY)
fCX). SA) dy
hhile
yhile this inversion is quile
iHespoint of yie. Bayes Omd natural trom a prtbabe
fur Hher amd onsider that Laplace went
mter of a model
th certanty of
Could be model pam
biLishe disnbution 1 to a pmba
Cn col
dishribuhion" edthe"prur
The inferen ce then bap
tiona) on
econ dhsm
off 6 condi
x_say
given by T (6|z) (6|x)_call ed th
f():T_ ()
the pos
posteni
en d disf

The (1e) TTO) de

dishribution s
Called a priur
Prir becaue E quunHHes ome disibuhiom m
wncertainty abou
prior t observìnq data. The wLertainty abo
the
Bauyesian subjech to believeprioron may represen
represent
i which ase it knowledg
know ledge
ouldbe subject to prior.
Al ter naively,
ît (old be aq
sec
sed to
represent small
conven honal pn
p n Supp
no nfer
TT (lk) maio we called
n
=
TTCG). f(x]6)_
TT9) Fe) de
 e xte TT(6|) T (6)
F(16)
The nuumerater in is joined
and the denominat ie Mx )
and
density o 6 x
TT (6) F (1p) d
is marginal derIH of X
The posterirdistribuhitm ombine
about e 0ith tbe
toprpr belief
inforr mauhion abou Com au n in
the data x IE gives a omposi nq picure of tte
final belief's abou J uJuE Os dsT
pir reflecs
beliefs about after dbserving data X
T9
T(G)

Note let X_be_a rdndom vamable hosevaluu

wen to estimate
e b e th observed r V that is we hav te
observed amd i we wWould ike to eshmate
X aSsum ing boh Xand y are discrete rv.then
hen
O Can rite,
PLx : E
P[x=X, y]1
P(N )
PLY]x:*]P(x: *)
PY ]
l(x9) P x y|x).()
Py () discrete)
f either x ov dre continuous v . we an replace
COesponding P:mf änd p4f in the termula
fur eg Ix S COnhuOa rY YS discrete
we Can write
 fxt(I9) P (tx) f(x)
If X and
P()
dre both ConinuDu TV then
fx (|9)
f)
To find ) and f 9) -wt ofteno se
ot total pn babiiy.

e k x . U(O,1) Suppose ha we tnou x :

Geomehic (x) nd the pos keriordensiky f x:2

ing Baye's rue we have

fx l2) x (2) fr(z)
P (2)
n 0 hat x =« Gx)
x y|*) =(I-x)9- fer y I ,2

1x 91<) x -x) er :2.

and X U0,1)
fx) L
Noo, to Py (2) we se the ot bfa pyod
pyob

P(e)

y (2) (x-* d

2
2
 Date
Pag2

The posteripr densi ty of 1 2 s

fxIy ( |2) Ptix 2 | ) f « )
PC2)

txIN (x12)
6
t x (X|2) 6* (1-x)

egle X Bn, P) and f (x[P) = "CP (1-P)

CUMd 9(p)= (1p)B- hen fAnd
f(P)
have
X B h P)
F(ap) nC P (1-p) n -X
9P) (1-P)
B(P)
to Hnd f (PIx 2
PzIp (XIP). 9 CP)
f (Px)
Px ( )
We n o that
P(P) B (n P)

5PxA) nC PA - *

f (PI) Chp* (I-P)- P p1-P)S

B(, P) p (1-P)?**

f (Plx) P CI-P)P

B (P)
 eASSMA
Dat
Page
1912012

f(aP) P)
f x P) 9P) dp
nCx p -P) h
n-a
B(AB)- p -P)
Cxp(-P)" /B(dB) P
-P)P1 d
t (-P)n- . ' B

p -
(T-P)

P (1-P)-**B

de b

t-
B (x*, n-*4 P)

?Fta (x+d, n-x4B)

distibuion
Bayes ESHmation
Setting
0ur qbal tb
pnr disibuhon ith elHmate
t
eletP(e) be the
ieii.d. f( |e). Derive Collec data *, X:
+(6|) Use th the posterior dstibuh
uHmatt fr 6posterior distibuh fo
o denve
dehe
let f , 6 e® be the
et ,
X2--7 n be rand m
family ot Pdf
pdt mf
One Sanmple (4, 2y--,n) Samplefrom f8
lam is observed the
observeol the sta
has to take ,

an acon th baas o#
this SCMple
1e dehote
the set ot all acHons
achms (T
(or deciSI
pen fo Th
SFahs icia a
 Loss tuncion
defdn
deftmihon of Decision funchion
A deusìm fun Bd' is a Stahshc that takes
valuesof i : e . dz) E a ) .
fe eq A-}a1, aaf then anydecisin funchun d
d
divides the space ot values (X1Xz X ) nto t o
disfoiut sets C nd CSt iE EC d d(z) = a
Cie e chose a achions) md t 6 C d)-az
This is the problem of tesing of hypothesi
Similaxy it d then the problem 1S thepoblem
o pointelHoaim

Loss function &

Def A funcHum
non-_neqohve L
thak map the
parameter H X into R called a loss funchm.
L: xa R (risk )
The yalue L(B, a) s the loss cUSociated_wTH
Hu acHtm a when the true panameter valuee 1S 8
If d dx) is a decisio funchon to be Lsed
LAed to
hoDse an ahion, then note that
L (,4) is C yand omn vamable
Enamplu of loss funchion
Squared eros loss funchom (uadratic loss Fun")
L(6,a) = (6-a) CMSG)
DR
LC6) ( - )2

Absplute eroY loss function

L(, a) 6 -aa)
(oR)
LCB, ) 1e 6
 asurt afed
clAsSMte
Dote

Risk tunction
Def: Rted}
Th isk funchion assoaated wih decision funchio
R(0, d) E L(,d))
It is a riterion to cbmpare different deision fun
fund
Homs a deuis jon funthon i th Smaller risl
12912822 funtion is differ .
that wtth larger decisjon tun
vth
Suppose
Suppa X B 0,0).
3/4 Hee desctmple
the we space
naider
is fo
X ur
o,k

deusim tunioms folows

d,x)E
34

d (

dalx) /2

d )

So1? e the
osS tundiun a
1 Squaedero loss funhm-
ie L6,a) (6,a)
Thun
LCe, a)E i(,4):
(GiG)
6 it te,4)= (G)(G
No Suppose the tm value o s /2 . Then
 :SMte
Data

the rsk fun" ofdi i

R d
(x))E le, dox)||
B d, EyL2, d,)

EL( PCA) * L(2PX)+L3P)

\ly R d a ) EyLL(L, da)]

L Px)+L(4) P

d a ) E y L ( , da)
(
c0,
4)
) P)

, l Va4«
, d)
( a d ) PO)
0,
 Daie
Pago

) P( L( PX)

T
No (muider he absol ute errer losS funion
e L B, 4) = 9 a.
We have,
L(D,a) = 0 , f (0,a) e ( G
i t f(0,4) )

R d) ( ft)L
= EL(% d )

1/2
2+o

R 2 da): E,L.d): ft

3192

So foar we have
Censidered e a a fixed (omr
Ond fe ) a the p-dt Or pmt ot . Tn
Bayesjan esinaaath We reat e 0 a r V. Ch
j hawn P:4: (e). Oih suppot .
The called a pn disth e So her
atter F(X/0) represents the comdihma pdf
Pdf Y

pmt o V. fa given e ee b e l d fin

Since TT (8) S the P-d.f CY pmf) ot e,th
Dint pd
Pdf Y pmt) o t ( e ) S
F ,e) IT ) f(x [e).
n thi framewn RCO,dD is the COmd
 113g20 22
Page
r6, dr ye
qvq oSS e. d) = E,L(6,dcn)e

Baye's Risl
DefiniHon
Bayesrisk ot a decision funchiom d s
defined a ,
R(TT, d) ER (6,d)]
=ErEl LE,d(«))_le
HE Lb case Ohere both 4 e are ContinuOA
we ha ve
RCTTd) Er ExLCed()) |e}
ET LC,dCz))E(xe) dx
L(,dz)£x e )e ) dx de

R CT d) = (L(E,d(z)).f(x,
e ) dx d e

Similarlyhen bothb a r e discrete R have

RCTd): 22 ( , d (y)) +(x,e) dde

eg7Suppese X B(n,6) and L(B, d(s) = (-d())

T (0)

S he piuy dish of Cal cuulat e bayes sk.

fr dCX) = ( h
 Date
Page-

sob The Bayesisk S ,

R4) ET EL(6,d(:)) |o] - 0-0
2
0here ExLce,dcy)) JD]= E -d x))|e]
Ex(dx)-e3)2 |e].

Var

ne(i-e) B(7, e). 1

v(x) ne (i-e)

(1-66)
n

NOw,_RLT d) E_E 6(1-6)

h

(-9) T(0) d

(1-e):1 de_

6n

61R e (1-0)21 de
n
B(,B)J»*-as
B(2, 2)
2 4+B
6n +B-1)L
 YAssMAt
Dat2
Pa

Suppose XP(A) ound L(A.dtx)) = (A- dz)) Th

pri
pror dishibuhim o t a is Gamma ( a e)Caluulate
Byes nsk fur d(x) *
Here X PA)
O,

- -A|B
CA) D<A< co

W
O

No Ex LO, dcs)) | 6] = Ey (dx) -a ) a

Ex (xA) Ja
voa A)

The Bayes S,

B(T) ETA
-AtB
ae

Mean o Gamma ddist

R[Td)
A 6(«, P)
 cSAAte
pote 141322

Def: Baye's Rule

The decision fun d" is called Baye's rule tr

bayes roced urt it 1 i n i miLes bayes

bayes ris
rsk
e is t
R(TT,d): in R(T,d)
d

osteri Dism buhm

x
A onditional disth o e qiven
s caulled a postert dismbu of 6 note h
the
pri daibutium ot e sthe dsmbuHen o
8 bete ta the Sampl Ohilethepesten
dish ot e Bthe d ot e after tak
sCUpleThuw, the postent pdf (n pt)
Pnf)

T ( ) =t e).TT (6) f(x, e)

9 (x) 9x)
petterit dath,
oR
h x) FX,e)
3(x)
LOnere
9(x) the pdf pmnt otXqiven b.
g() fCxe) de

Scuyes rult

Note that
R(T4) L , do) f(x,e) dx de

L(0,de ) f(x, e) de dx

L(da)) g(x) T (e/x) de dx

 s5Ate
Date

L(0,d x) TT(e (z) d 6 g(<) da

* Posterir Expected Less i- (oliernahre det b

Posterior expected 6ss deined O
T(|x) L(, dox)de
C n d auternaiv defn_ Df Bayesrule ecHmatr
Sth estimato that _minimi2es thepoterir ezpeted
lesS
Note Bayes estimater ot e is
E(e)=(
TT(6| x) de

Bayes EsHmaor
Bayes estimattr Om Squared era SS
Statement T nb e problem ot etmation ot poram-
eer e E the bayes eHmatr wY't quadraH
lsS fun s,
E )
et
pos Feri oY) S R ( e ) = (T(elx) L(e,8) de
epeched
c) ET(6)]x-x

Ce-8) (e|a) de
pesttri
ted
\GSS* 6 TT ( de -eeTT (e|*) de+e Tr(e|)de
ep

2 b +c
 aiASSiM

Sd the zoal is to minimi 2e,

R El(6-6) |x= *J.

d RCO) 2 2b

Te x) de

E el)
the m ean ot )
NOW
d2 R( = 2 he mmi uum

E ((-)|x=*) Lohicb is minimum_ a

Ee!x):_ mean the peiten da
S9122

H:P
* BayesEsHmat wh absolulte e r o r loss
l0ss tundh
Statem en
The Bayes ehmater is 6 w.rE
absolute
e r o oss funhm is the meclian cf the
distibuion t posten
pasten

root
L(o.B) l6-8
R(11,) E e -®

6-61 t (®|<) d de
 eAsSMAte
Dat2
Page

(e-) TI (o]<) de t (-6) T (e]<) de

T (0|x) de TT(ez) de
+ T
(e|x) de eTT (e|<) d
BTT (e ) de-&- F(61«) FE)- TT(e2) de
-
No OUY am isto Mìniui 2e R LTI )

R (T )_ s mimi.mize e, mediam ofthe

t 1 3 1 2 2
posten dist.

(eT el<) de
-co
e | x ) de
46 +)dt= f(x) .

-F6)] 6)] d FO
-
TT (x)+f ()
1-
(B |x)- f 6)
F
de ) 6T(83) -+f(6)
de
C(ely (elx) de ô T L6I)
-00

- (6lx) - It
6 T (E |<) +
F®) + T (E|)
 Pa

2
F(e) - =0
Fe) 2
is the mediaw ot T ( )
NoLO
dTTd 2TT (6 2) >o winimu d f(G).
d
R(Tè) is minimum e m ediam o posternor
di ot

L(e) I (6 +e):

BauesEsHmater 0-1 oith

erOr loss
Statement
The Baue estimaka of e
tuncHam is the mode of thewTt 0-1 e r r s e

Prot Ptenr clistn a-

L(el6) I(B+ ).

I
-6P (ete) T(0|x) de

T(6| z) de + T
(ex) de
+ T
(s|) de
A
 ASSMAC

-A C

im
A0
TT (e ) de+ TT (6|4) de
tA
tA
li T (e]x) de
ne
A O
e-4

m T ( e ) e

A 0
-A

-T(e|z) de rde

T ominimize
R()De must mdii2e T E|x)_
Th IMplies Must be th mode ot T (el).

cedure to obtain Bayes Estimatr

12 0b tain the jout pdf o pmf dt (xe) a
f (xe) =f(xe) T e)_

62 Obtain the wMarginal_doth of x a

S) (E(1|e) (0)de

S 3) btain he posterio distibutim ot a

h Celz) = f(3, e)
()

S. 4Under squa red emoY loss funchinm the

the Bayes es Hiade
Under e r r funcium the.
men s E (6J«) amd uunder dbsolute
Bayesesti mcutr is the med tan o the posterir
Bayes
distribuhom otB.
 Lele19 3la
Paga

PLP-dx)] omd
eg, XBn,P) Let L(f, d(x))
DC P
T1(P)
D W

Hnd Bayes esHmakr ot P

So Wehave, XB(n.P

f(xP) ( OP)

0.

o<pe
TTLP)

Tbe ont p.d f distihutin t x_oumd Pis

F(aP)E f(x le) TP)
-
M:O1 2-1
OCP

- W
pw)
The Marna Tdistribuhomot is aiven
f(*.P) dp
-
O
dP

-t1-1

P dP

t1 n-x+1) 0,,-D
6P<l
0 W

The pesterim dis hb uhon oPs given bybu

tP]a) = f(x.P)
 eAsSMALe
Date
pag

( f Ix) :
P " 1-
Bxtl, n-nt1)

0,1,2. - -D
f(Px) B(+1,n-*t))

ohich is pdf ofB.(x+1, n-+)

Hence Umderthe squared e r r Lo fun cHm s
BauyesesH maute o P iS,
EP|x) = +
E (C)
E (P )
n+2

5ayesnsk iS
R(T d)_ Er RP, d)
ETECLP, d(x)) |e
xE t2-

ET E
a
RTd)- Em)E AL-2P. Xt|+P ht2
P
o h wn
E(X+1 E E()+1

(D42)2

(n4 2)