7 views

Uploaded by Aarthy

EM

- 16-581AppendixB
- isical1
- Likelihood analysis of species occurrence probability from presence-only data for modelling species distributions
- Aistats2012
- Estimating time to pregnancy.pdf
- The Effectiveness of Customized Promotions
- Editorial
- ZhongZhongNi_Pres2005
- RSC Final Cvpr Submit
- POSTERIOR RESOLUTION AND STRUCTURAL MODIFICATION FOR PARAMETER DETERMINATION IN BAYESIAN MODEL UPDATING
- 2-Agri Sci - Ijasr - Efficiency - Ogunniyi - Nigeria
- Panel Data on Eviews
- Eeg Interface Module for Cognitive Assessment Through Neurophysiologic Tests
- Semiparametric Model and Bayesian Analysis for Clustered Accelerated Life Testing Data
- Parametric Survial Model E
- 11 Maths Notes 16 Probability
- 3-17
- Mds Self Test
- algebra2-probabilitystatisticssampleunit
- Bowerman3Ce SSM Chapter15

You are on page 1of 25

Intro: Expectation Maximization Algorithm

presence of unobserved variables.

relevant features or variables might be observable.

Eg: Hidden Markov, Bayesian Belief Networks

Simple Example: Coin Flipping

landing heads, , .

1 ;

Where , = {

0 ;

1

No hidden variables easy solution. =

=1 ,

; sample

mean

Simplified MLE

parameters without knowing

the identity of each data sets

coin.

Solution: Expectation-

maximization

Coin Flip With hidden variables

What if you were given the same dataset of coin flip results,

but no coin identities defining the datasets?

1,1 ,1

1 ;

= , where , =

1, , 0;

But Z is not known. (Ie: hidden / latent variable)

EM Algorithm

= 1 , , coin flip example: = { , } = {0.6, 0.5}

1) Expectation (E-step)

= = )

, =

=1 = = )

If , is known:

2) Maximization (M-step)

=1 ,

=1

= =

=1 ,

EM- Coin Flip example

value

Ex: A=0.6, B= 0.5

Compute a probability

distribution of possible

completions of the data using

current parameters

EM- Coin Flip example

Set 1

What is the probability that I observe 5 heads and 5 tails in coin A and B

given the initializing parameters A=0.6, B= 0.5?

Compute likelihood of set 1 coming from coin A or B using the binomial

distribution with mean probability on n trials with k successes

Likelihood of A=0.00079

Likelihood of B=0.00097

Normalize to get probabilities A=0.45, B=0.55

The E-step

Set 1

P(Coin=A)=0.45; P(Coin=B)=0.55

Estimate how these probabilities can account for the number of observed

heads and tails in the coin flip set

The M-step

Summary

2. Estimate probability using these parameters that each data set

( ) came from coin ([, ])

3. Use these probability values ([, ]) as weights on each data

point when computing a new to describe each distribution

4. Summate these expected values, use maximum likelihood

estimation to derive new parameter values to repeat process

Gaussian Mixture Models

Distributions

Gaussian Mixture Models

= =

= = ; , 2

EM algorithm in Gaussian Mixtures

1 , ,

Step 0) Initialize = 12 , , 2 (assuming k clusters)

1 , ,

, (| = )

, =

=1 , (| = )

EM algorithm for Gaussian Mixture

Step 2) Maximization:

= ,

=

1

= ,

2 1 2

= ,

Example of EM in Gaussian Mixtures

Overfitting through convergence

Initializing Parameters

complex likelihood functions w/ many local optima

Since EM only solves for a single local optima, choosing a

good initial parameter estimation is critical

Strategies to improve initialization

Multiple random restarts

Use prior knowledge

Output of a simpler, though less robust algorithm

Resources

Matlab EM Algorithm

Tom Mitchell- Machine Learning: Chapter 6 (on lab wiki)

EM Algorithm Derivation, Convergence, Hidden Markov

and GMM Applications

Nature Review Article

- 16-581AppendixBUploaded byangeljosechuquiure
- isical1Uploaded byapi-3715848
- Likelihood analysis of species occurrence probability from presence-only data for modelling species distributionsUploaded bySantiago Vega
- Aistats2012Uploaded byGautam Dematti
- Estimating time to pregnancy.pdfUploaded byEly Danel
- The Effectiveness of Customized PromotionsUploaded byAyçennur Ytgn
- EditorialUploaded byAbd Lathif Al Basyir
- ZhongZhongNi_Pres2005Uploaded byNiyas Ahamed
- RSC Final Cvpr SubmitUploaded byVijay Sagar
- POSTERIOR RESOLUTION AND STRUCTURAL MODIFICATION FOR PARAMETER DETERMINATION IN BAYESIAN MODEL UPDATINGUploaded byJames Moreno
- 2-Agri Sci - Ijasr - Efficiency - Ogunniyi - NigeriaUploaded byTJPRC Publications
- Panel Data on EviewsUploaded byIndiane Rios
- Eeg Interface Module for Cognitive Assessment Through Neurophysiologic TestsUploaded bylazarstosic
- Semiparametric Model and Bayesian Analysis for Clustered Accelerated Life Testing DataUploaded bySEP-Publisher
- Parametric Survial Model EUploaded byJoseph Lim
- 11 Maths Notes 16 ProbabilityUploaded byniteshg11
- 3-17Uploaded byKabilan
- Mds Self TestUploaded bysahil
- algebra2-probabilitystatisticssampleunitUploaded byapi-292276730
- Bowerman3Ce SSM Chapter15Uploaded byamul65
- hw2Uploaded byTristan Wa
- RTS_2Uploaded byयोगेशअर्यल
- Ulrich Hoffmann et al- Bayesian Feature Selection Applied in a P300 Brain-Computer InterfaceUploaded byAsvcxv
- Estimative ProbabilityUploaded byBisto Masilo
- GEM2900 probabilityUploaded bywakakka
- Data and Statistical NotesUploaded byVineet Khandelwal
- DSC4213-2016-4 notesUploaded bySabina Tan
- Mudersbach and Jensen - An Advanced Statistical Extreme Value Model for Evaluating Storm Surge With Sea Level Rise - 2010Uploaded byAnonymous p9SPc84
- ngramsUploaded byOuladKaddourAhmed
- 4.1 notes2Uploaded bySunil Kumar

- 34 NP CompletenessUploaded byAarthy
- Inspiron 15 3567 8th Generation i5Uploaded byAarthy
- ElectionUploaded byAarthy
- EC6312 SET1Uploaded byAarthy
- SimilarityAnalysis.pdfUploaded byAarthy
- SimilarityAnalysis.pdfUploaded byAarthy
- EC6312 SET1.pdfUploaded byManju Nathan
- Gate New Syllabus Computer Science and Information Technology for 2016Uploaded byersayantan

- TuringUploaded byAldo Raquita
- BE10-R3Uploaded byprashantmp298
- Elisabethcairesume TNR (2 17 14)_updateUploaded byAndrea Ning
- tucilUploaded bySePtia LoKa
- LN_Week1Uploaded bywilltuna
- M WAY TREES PDFUploaded bymush76here
- FM Topic 5 Time Value of MoneyUploaded bySozia Tan
- 1digital System DesignUploaded byharisri269
- An Introduction to Numerical Methods and Analysis, 2nd EditionUploaded byRicardo
- Python Machine Learning - Sample ChapterUploaded byPackt Publishing
- Poster cfdUploaded byDanielle Hays
- Seznam Literatury o StatisticeUploaded bycherokez
- RTSsolUploaded byAmit Negi
- Express Learning Automata Theory and Formal Languages PDFUploaded byKim
- DSP Complete ManualUploaded byAnonymous Q6WJ2JU
- Simplex FullUploaded bySayed Alhussaini
- Kimia Komputasi B_zevano c. Sibarani_1506740654)Uploaded byZevano C. Sibarani
- Fuzzy Cognitive Maps With Genetic Algorithm for Goal-Oriented Decision SupportUploaded bypavithramasi
- Modeling of a SEPIC Converter Operating in Discontinuous Conduction ModeUploaded bytybonescribd
- DAA Imprt QnsUploaded byammurasikan6477
- NewsboyUploaded byFlorentina Alina Popa
- 351_27435_EE411_2015_1__1_1_0 3 EE411 Lec6,7 compensation RL (1)Uploaded byMohamed Saeed
- III-12 (1)Uploaded bybastian89_ripa
- 175 Quantization Error and NoiseUploaded byKhánh Sơn Ma
- Algorithm Design SlidesUploaded byThangNgô
- soalGOJEK.rtfUploaded byRyan Hegar Suryadinatha
- Validation and Updating Ansys Model Using Expt Modal DataUploaded byKiran Kolluru
- Assignment 3 SolUploaded byChuah Chian Yeong
- BSTUploaded byGiri Prasad
- Research MethodologyUploaded byLawal Abimbola