3 views

Uploaded by Krishnavishal Vemula

Downloaded and Uploaded xDDDDDD

- Trends in Audio and Acoustic Signal Processing
- 38.an Unsupervised Approach to Cochannel
- Cdc2011 Slides
- Alexander Hcii 2009
- Socially Shared Images with Automated Annotation Process by Using Improved User Preferences
- Floridi.the Philosophy of Information_ocr
- In Memory of Clive l Dym
- PRESENTATION - Cognitive Computing for Anticipatory Risk Analytics [44]
- role of statistics in education
- 7.if Peter Drucker Was a Software Developer How Would He Use Artificial Intelligence to Help Managers
- bayesian nonparametrics
- finding answers through data collection
- Learning Scripts as HMMs
- Importance of Statistics
- Chords
- An Introduction to Expert System and Artificialintelligence
- Emerging Jobs Report India Sept2018-D5B5
- Pmd Measurment
- Chapter 1
- tts

You are on page 1of 28

Steven Salzberg

CMSC 828H, Univ. of Maryland

Fall 2010

Real time continuous speech recognition

(HMMs are the basis for all the leading

products)

Eukaryotic and prokaryotic gene finding

(HMMs are the basis of GENSCAN, Genie,

VEIL, GlimmerHMM, TwinScan, etc.)

Multiple sequence alignment

Identification of sequence motifs

Prediction of protein structure

2

S. Salzberg CMSC 828H

What is an HMM?

Essentially, an HMM is just

A set of states

A set of transitions between states

Transitions have

A probability of taking a transition (moving from

one state to another)

A set of possible outputs

Probabilities for each of the outputs

attached to the states rather than the

transitions

3

S. Salzberg CMSC 828H

HMM notation

The set of all states: {s}

Initial states: SI

Final states: SF

Probability of making the transition

from state i to j: aij

A set of output symbols

Probability of emitting the symbol k

while making the transition from

state i to j: bij(k)

4

S. Salzberg CMSC 828H

0.9

Fair

Two

TwoCDF

CDFtables

tables

0.2

0.1

Unfair

States

0.8

0.5

H

0.5

T

0.7

H

0.3

Observation Symbols

HTHHTTHHHTHTHTHHTHHHHHHTHTHH

FFFFFFUUUFFFFFFUUUUUUUFFFFFF

Observation Sequence

State Sequence

Motivation: Given a sequence of H & Ts, can you tell at what times

the casino cheated?

5

S. Salzberg CMSC 828H

Consider the sequence AAACCC, and assume that you observed this

output from this HMM. What sequence of states is most likely?

6

S. Salzberg CMSC 828H

Properties of an HMM

First-order Markov process

st only depends on st-1

However, note that probability

distributions may contain conditional

probabilities

Time is discrete

7

S. Salzberg CMSC 828H

1. Evaluation: given a model and an output

sequence, what is the probability that the

model generated that output?

To answer this, we consider all possible paths

through the model

A solution to this problem gives us a way of

scoring the match between an HMM and an

observed sequence

Example: we might have a set of HMMs

representing protein families

8

S. Salzberg CMSC 828H

2. Decoding: given a model and an output

sequence, what is the most likely state

sequence through the model that generated

the output?

A solution to this problem gives us a way to

match up an observed sequence and the

states in the model.

In gene finding, the states correspond to

sequence features such as start codons,

stop codons, and splice sites

9

S. Salzberg CMSC 828H

3. Learning: given a model and a set of

observed sequences, how do we set the

models parameters so that it has a high

probability of generating those sequences?

This is perhaps the most important, and most

difficult problem.

A solution to this problem allows us to

determine all the probabilities in an HMMs

by using an ensemble of training data

10

S. Salzberg CMSC 828H

An untrained HMM

11

S. Salzberg CMSC 828H

The sum of the probabilities on all the

edges leaving a state is 1

ij

=1

12

S. Salzberg CMSC 828H

The sum of all the output probabilities

attached to any edge is 1

b (k) = 1

ij

13

S. Salzberg CMSC 828H

aij is a conditional probability; i.e., the

probablity that the model is in state j at

time t+1 given that it was in state i at

time t

aij = P ( X t +1 = j | X t = i)

14

S. Salzberg CMSC 828H

bij(k) is a conditional probability; i.e., the

probablity that the model generated k as

output, given that it made the transition

ij at time t

15

S. Salzberg CMSC 828H

Probability of taking a transition depends only

on the current state

This is sometimes called the Markov assumption

only on the transition ij, not on previous

outputs

This is sometimes called the output independence

assumption

nth order HMM using a 0th order HMM

This is how some actual gene finders (e.g., VEIL)

work

16

S. Salzberg CMSC 828H

the Forward algorithm

To solve the Evaluation problem, we use the

HMM and the data to build a trellis

Filling in the trellis will give tell us the

probability that the HMM generated the data by

finding all possible paths that could do it

17

S. Salzberg CMSC 828H

18

S. Salzberg CMSC 828H

Time

t=1

t=0

1.0

(0.6)(0.8)(1.0)

t=3

0.48

State

4)

(0.

(0.

1)(

0.1

)(0

)

S1

t=2

)

1.0

5)(

(0.

S2

0.0

Output:

+

(0.9)(0.3)(0)

0.20

C

19

S. Salzberg CMSC 828H

Time

t=1

t=0

1.0

(0.6)(0.8)(1.0)

0.48

(0.6)(0.2)(0.48)

t=3

.0576

.0756 + .018 = .0756

)(0

( 1.

.5 )

0)

0.0

Output:

+

(0.9)(0.3)(0)

0.20

(0.9)(0.7)(0.2)

S2

.48

)(0

0.5

4)(

(0.

4

(0.

(0.

1

)(0

.1 )

State

(0.

1)(

0.9

(0)

)(0

.2

S1

t=2

.126

.222 + .096 = .222

C

20

S. Salzberg CMSC 828H

Time

t=1

1.0

(0.6)(0.8)(1.0)

0.48

(0.6)(0.2)(0.48)

t=3

(0.6)(0.2)(.0756)

.0756

.029

.009072

+ .01998 =+.029052

+

Output:

(0.9)(0.3)(0)

(0.

1)(

0.9

)(0

.2

(0.

1)(

0.9

+

+

.222 + (0.9)(0.7)(0.222)

.01512 = .15498

.155

(0.9)(0.7)(0.2).13986

)

0.0

56)

.0)

)(1

0.5

4)(

(0.

S2

.07

)(0

0.5

4)(

(0.

State

.48

)(0

0.5

4)(

(0.

(0.

1)(

0.1

)(0

)

)(0

.2

S1

t=2

22)

t=0

0.20

C

21

S. Salzberg CMSC 828H

sequence of length T:

T

1

T

1

All paths:

T +1

1

T +1

1

22

S. Salzberg CMSC 828H

P(Y = y ) = P(X

T

1

T

1

T +1

1

T +1

1

=x

T

1

T

1

T +1

1

)P(Y = y | X

T +1

1

=x

x1T +1

HMM is the sum of the probabilities that we took any path that

emitted that sequence.

* Note that all paths are disjoint - we only take 1 - so you can add

their probabilities

23

S. Salzberg CMSC 828H

probabilities

T +1

1

P(X

T +1

1

=x

) = P(X t +1 = x t +1 | X t = x t )

t=1

Markov assumption, which allows us to multiply probabilities just

as we do for Markov chains

24

S. Salzberg CMSC 828H

probabilities

T

t=1

another Markov assumption, that the output at any time is

dependent only on the transition being taken at that time

25

S. Salzberg CMSC 828H

computable formula

P(Y1T = y1T ) =

x1T +1

P(X

t +1

= x t +1 | X t = x t )P(Yt = y t | X t = x t, X t +1 = x t +1 )

t=1

computes, recursively.

*Note that the only variables we need to

consider at each step are yt, xt, and xt+1

26

S. Salzberg CMSC 828H

formulation

0 : t = 0 i SI

i ( t ) =

1 : t = 0 i = SI

(t 1)a b (y) : t > 0

ji ji

j j

Where i(t) is the probability that the HMM is in

state i after generating the sequence y1,y2,,yt

27

S. Salzberg CMSC 828H

The Forward algorithm computes P(y|M)

If we are comparing two or more models,

we want the likelihood that each model

generated the data: P(M|y)

Use Bayes law:

P(y | M)P(M)

P(M | y) =

P(y)

just need to maximize P(y|M)P(M)

28

S. Salzberg CMSC 828H

- Trends in Audio and Acoustic Signal ProcessingUploaded byAl Strings
- 38.an Unsupervised Approach to CochannelUploaded byjagadeesh jagade
- Cdc2011 SlidesUploaded bykundeng2
- Alexander Hcii 2009Uploaded byMostafa Sataki
- Socially Shared Images with Automated Annotation Process by Using Improved User PreferencesUploaded byAnonymous 7VPPkWS8O
- Floridi.the Philosophy of Information_ocrUploaded byPabloAPacheco
- In Memory of Clive l DymUploaded bymarcelo3com
- PRESENTATION - Cognitive Computing for Anticipatory Risk Analytics [44]Uploaded byrobertkover1964
- role of statistics in educationUploaded bySahir Khan
- 7.if Peter Drucker Was a Software Developer How Would He Use Artificial Intelligence to Help ManagersUploaded byAtul Mishra
- bayesian nonparametricsUploaded byZeferinix
- finding answers through data collectionUploaded byapi-339611548
- Learning Scripts as HMMsUploaded bywalker
- Importance of StatisticsUploaded byTritoy
- ChordsUploaded byDANA
- An Introduction to Expert System and ArtificialintelligenceUploaded byMuamer Besic
- Emerging Jobs Report India Sept2018-D5B5Uploaded bySunny Verma
- Pmd MeasurmentUploaded byhas
- Chapter 1Uploaded byNeupane Er Krishna
- ttsUploaded byMohammad Rzzaz
- Bridging AI Skills GapUploaded byishaan90
- AI IntroductionUploaded bymahmoud_mhosny2717
- cst300l delgado paper2Uploaded byapi-428318320
- Hidden Markov Model ImplementationUploaded byJay Singh
- review.pdfUploaded byJulio Cesar Sucasaca Rodriguez
- Fuzzy Model Optimisatgion Using GaUploaded byAbhijit Singh Uppal
- “One Panchayat, One Product” a Strategy for Sustainable Rural EconomyUploaded byMANAV SAHU
- Opportunities for Artificial Intelligence[1]Uploaded byMatthew Reece
- Writing Problem and Hypothesis Statements for Business Research(13)Uploaded by柯泰德 (Ted Knoy)
- AISO-2015Uploaded byCS & IT

- New MicrosoftUploaded byKrishnavishal Vemula
- Chapter1.pdfUploaded byKrishnavishal Vemula
- ARGA Investment Management (India) Pvt. Ltd._Global Research Associate.docxUploaded byKrishnavishal Vemula
- Tutorial 1 EE55100Uploaded byKrishnavishal Vemula
- Murali AssignUploaded byKrishnavishal Vemula
- SAS93_9B3PQZ_70109264_Win_WrkstnUploaded byKrishnavishal Vemula
- IIT Madras Academic CalenderUploaded byKrishnavishal Vemula

- How to Face Questions in QTPUploaded byRam_p
- Automatic Pneumatic Bumper Adjust MechanismUploaded byieee4mybusinessonly
- MIS QuestionUploaded byr_ponnusamy
- Cable RacewayUploaded by54045114
- DDS-50-2007(part-1)Uploaded byTalal Ghazi
- Preliminary Study on the Use of Computational Fluid Dynamics to Determine the Frictional Resistance of a Trimaran ShipUploaded bysm8575
- Sample Unit 3 Life 5 AmeUploaded bymariojt2
- 9781849696623_Raspberry_Pi_Cookbook_for_Python_Programmers_Sample_ChapterUploaded byPackt Publishing
- Arrancadores Suaves ATS01 Telemecanique Coeva Barcelona SAUploaded byHugo Flores
- PicoBlaze Tutorial 1v2Uploaded bydbinnie
- Training Feed Forward Networks With the Marquardt AlgorithmUploaded bysamijaba
- Rac on VM Best PracticesUploaded byRavi Kasibhatla
- Chapter 14.pdfUploaded byDana Ajouz
- Custom Chart - Min Max Average Bps in Out (4)Uploaded byGabriel Celis Salguero
- Using Haar-like Feature Classifiers for Hand Tracking in Tabletop Augmented RealityUploaded byMohammad Bassam El-Halabi
- GleeUploaded byslofire94
- Hfm Cell Text AddendumUploaded bykonda83
- Smokeping InstallUploaded byAndi Jaf
- 05483285 wUploaded byAju John
- Produktblad Elcon DCM_v5pdfUploaded bySunil Limaye
- cUploaded byBharath Meduri
- MM QUESTIONBANKUploaded byPolisetti Sn Bhaskar
- Assessment Progress0512Uploaded byuogufc1
- Dwl-2200ap Guia RapidaUploaded byDrak'Thul Walter Quintana Franco
- Hadoop World: Monitoring Best Practices, Ed Capriolo, About.comUploaded byOleksiy Kovyrin
- PolymerUploaded byivorisek
- A Computer Program for Cylindrical Shell AnalysisUploaded byYogesh Baheti
- Collaborative Action PlanUploaded bypi1tama2
- Ez PDF to Word Converter for Doc 5 0 Keygen At4reUploaded byAngelina
- Python RequestsUploaded bymalliwi88