12 views

Uploaded by Narasimharajan Mani

save

- Stochastic Hw
- 3Vol23No2
- chap8_p1_s
- Energy Consumption Model in Ad Hoc Mobile Network
- Pattern and Predictability in Music.pdf
- Uslu
- 621Syllabus
- Lecture 2006
- 1968 Prospects for Experimental Games
- DM52 Finish
- Stochastic Calculus 2015
- Judgement
- ou
- probability-theory-an-analytic-view (1).pdf
- Stochastic Calculus
- API Testing Method
- STOCHASTIC MODELING TECHNOLOGY FOR GRAIN CROPS STORAGE APPLICATION: REVIEW
- Hans Marko The bidirectional Communication Theroy - A generalization of information theory 1973
- Review of Fundamental Problems and Solutions in Fractal Surface Growth
- A Stochastic Approach to Computational Fluid Dynamics
- Brownian Motion
- Yes Please
- The Unwinding: An Inner History of the New America
- Sapiens: A Brief History of Humankind
- Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic Future
- Dispatches from Pluto: Lost and Found in the Mississippi Delta
- The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution
- Devil in the Grove: Thurgood Marshall, the Groveland Boys, and the Dawn of a New America
- John Adams
- The Emperor of All Maladies: A Biography of Cancer
- Grand Pursuit: The Story of Economic Genius
- This Changes Everything: Capitalism vs. The Climate
- A Heartbreaking Work Of Staggering Genius: A Memoir Based on a True Story
- The Prize: The Epic Quest for Oil, Money & Power
- The New Confessions of an Economic Hit Man
- Team of Rivals: The Political Genius of Abraham Lincoln
- The World Is Flat 3.0: A Brief History of the Twenty-first Century
- The Hard Thing About Hard Things: Building a Business When There Are No Easy Answers
- Smart People Should Build Things: How to Restore Our Culture of Achievement, Build a Path for Entrepreneurs, and Create New Jobs in America
- Rise of ISIS: A Threat We Can't Ignore
- Angela's Ashes: A Memoir
- Steve Jobs
- How To Win Friends and Influence People
- Bad Feminist: Essays
- The Light Between Oceans: A Novel
- The Sympathizer: A Novel (Pulitzer Prize for Fiction)
- Extremely Loud and Incredibly Close: A Novel
- Leaving Berlin: A Novel
- The Silver Linings Playbook: A Novel
- You Too Can Have a Body Like Mine: A Novel
- The Incarnations: A Novel
- The Love Affairs of Nathaniel P.: A Novel
- Life of Pi
- The Master
- Bel Canto
- A Man Called Ove: A Novel
- Brooklyn: A Novel
- The Flamethrowers: A Novel
- The First Bad Man: A Novel
- We Are Not Ourselves: A Novel
- The Blazing World: A Novel
- The Rosie Project: A Novel
- The Wallcreeper
- The Art of Racing in the Rain: A Novel
- Wolf Hall: A Novel
- The Kitchen House: A Novel
- Beautiful Ruins: A Novel
- Interpreter of Maladies
- Lovers at the Chameleon Club, Paris 1932: A Novel
- The Bonfire of the Vanities: A Novel
- The Perks of Being a Wallflower
- The Cider House Rules
- A Prayer for Owen Meany: A Novel
- My Sister's Keeper: A Novel
- Quimica Ao Primeiro Odor
- Star Wars i Jensaarai by Rodney Thompson
- Bases de Datos-cun
- Magelis Sto Stu Hmistu855
- Gómez, L. et al. Nanopartículas consolidación mat. pétreos. 2011
- 4. Guias de Onda 14
- Triac
- Absorcion Atòmica
- Deep Learning in Character Recognition
- USB Booteable Con Varios Sistemas Operativos (ACTUALIZADO)
- Cuestionario para estudiar Programas de Alfabetización Digital
- Pierre Bourdieu, une pensée en mouvement - Débats
- EMC Secure Remote Services 3.04 Port Requirements
- Andy Field Using Spss
- Sample Report Briefing 2015-05-18
- Lista de Exercícios 4º Bimestre (2015) COMENTADA
- INFORME DE INNOVACIÓN DE AREA CHELE
- E os Campos de Predominância Elementais - I
- UT Dallas Syllabus for biol3350.501.10f taught by Ilya Sapozhnikov (isapoz)
- 313177423-55-Modos-de-Servicio-Chinos-Por-Micros.pdf
- Ppt-u03 Estructuras de Control Selectivas - Copia
- Trabajo Aborto 1.1
- COSTOS TERCER EXAMEN
- MÓDULO - Psicología Del Desarrollo Humano
- Technology, Globalisation and the Future of Work in Europe
- greedy-dp
- Machupicchu Icono de La Identidad Peruana
- Cliente Liviano
- Java_Bible
- Dispense di Fisica 2

You are on page 1of 4

Stochastic Process

Stochastic Processes, Markov Chains, and Markov Models

Stochastic Process Markov Chain

Stochastic Process

◮

**Stochastic Processes, Markov Chains, and Markov Models
**

L645 Dept. of Linguistics, Indiana University Fall 2009

Markov Models

◮

A stochastic or random process is a sequence ξ1 , ξ2 , . . . of random variables based on the same sample space Ω. The possible outcomes of the random variables are called the set of possible states of the process.

◮

Markov Chain Markov Models

The process will be said to be in state ξt at time t.

◮

**Note that the random variables are in general not independent.
**

◮

In fact, the interesting thing about stochastic process is the dependence between the random variables.

1 / 22

2 / 22

Stochastic Process: Example 1

Stochastic Processes, Markov Chains, and Markov Models

Stochastic Process: Example 2

Stochastic Processes, Markov Chains, and Markov Models

Stochastic Process Markov Chain Markov Models

Stochastic Process Markov Chain

The sequence of outcomes when repeatedly casting the die is a stochastic process with discrete random variables and a discrete time parameter.

There are three telephone lines, and at any given moment 0, 1, 2 or 3 of them can be busy. Once every minute we will observe how many of them are busy. This will be a random variable with Ωξ = {0, 1, 2, 3}. Let ξ1 be the number of busy lines at the ﬁrst observation time, ξ2 the number of busy lines at the second observation time, etc. The sequence of the number of the busy lines then forms a random process with discrete random variables and a discrete time parameter.

Markov Models

3 / 22

4 / 22

Characterizing a random process

Stochastic Processes, Markov Chains, and Markov Models

Markov Chains and the Markov Property (1)

Stochastic Processes, Markov Chains, and Markov Models

Stochastic Process

Stochastic Process

To fully characterize a random process with time parameter t, we specify: 1. the probability P(ξ1 = xj ) of each outcome xj for the ﬁrst observation, i.e., the initial state ξ1 2. for each subsequent observation/state ξt+1 : t = 1, 2, ... the conditional probabilities P(ξt+1 = xit+1 |ξ1 = xi , ..., ξt = xit ) Terminate after some ﬁnite number of steps T

Markov Chain Markov Models

A Markov chain is a special type of stochastic process where the probability of the next state conditional on the entire sequence of previous states up to the current state is in fact only dependent on the current state. This is called the Markov property and can be stated as:

Markov Chain Markov Models

P(ξt+1 = xit+1 |ξ1 = xi1 , . . . , ξt = xit ) =

P(ξt+1 = xit+1 |ξt = xit )

5 / 22

6 / 22

and Markov Models Stochastic Process Markov Chain Markov Models Stochastic Process Markov Chain In the telephone example. pij ≥ 0. (Note that we have numbered the rows and columns 0 through 3.2 0. Markov Chains.2 0. a matrix with these properties is called a stochastic matrix. . a possible transition matrix could for example be: s0 0. . . Markov Chains.1 0. Markov Chains. . i. . . . and Markov Models Transition Matrix: Example Stochastic Processes.2 0.1 0. . .1. A ﬁnite Markov chain must at each time be in one of a ﬁnite number of states.. n j=1 s0 s1 s2 s3 In general.5 0. Markov Chains. .1 s1 0. 7 / 22 8 / 22 Transition Matrix for a Markov Process Consider a Markov chain with n states s1 . p1n P = . pn1 . .3 s3 0. . and Markov Models Stochastic Process Markov Chain Stochastic Process (2) pij Assume that currently. ξ2 . More generally. We can turn the telephone example with 0. Column 1 (p31 ) is 0. What is then the probability that at the next point in time exactly one line is busy? The element in Row 3.2 0.Markov Chains and the Markov Property (2) Stochastic Processes. . ξt = xit ) = = P(ξ1 = xi1 ) · P(ξ2 = xi2 |ξ1 = xi1 ) · . and not on the previous history.4 0. .1 s2 0.) Markov Models = P(ξt+2 = sj |ξt = si ) n Markov Chain Markov Models = r =1 n P(ξt+1 = sr |ξt = si ) · P(ξt+2 = sj |ξt+1 = sr ) pir · prj = r =1 The element pij can be determined by matrix multiplication as the value in row i and column j of the transition matrix P 2 = PP. .4 0. . .3 0. sn . pnn n Stochastic Processes. .e. 2 or 3 busy lines into a (ﬁnite) Markov chain by assuming that the number of busy lines will depend only on the number of lines that were busy the last time we observed them. i = 1. . P(ξt+1 = sj |ξt = si ).1. . The transition matrix for this Markov process is then deﬁned as p11 . .1 0. and Markov Models Transition Matrix after Two Steps Stochastic Processes. . 9 / 22 10 / 22 Transition Matrix: Example (2) Stochastic Processes. .5 Markov Models P= pij = 1. . Markov Chains. 11 / 22 12 / 22 (2) . the transition matrix for t steps is P t . can be calculated as: Stochastic Process Markov Chain Markov Models Stochastic Process Markov Chain Markov Models P(ξ1 = xi1 . and thus p(1|3) = 0. all three lines are busy. 1. . . · P(ξt = xit |ξt−1 = xit−1 ) The conditional probabilities P(ξt+1 = xit+1 |ξt = xit ) are called the transition probabilities of the Markov chain. and Markov Models The probability of a Markov chain ξ1 . . Markov Chains. . Let pij denote the transition probability from state si to state sj . and Markov Models Markov Chains: Example Stochastic Processes. .3 0. .

24 Stochastic Process Markov Chain Markov Models Let’s go to http://people. . Markov Chains. .2 0. and Markov Models Stochastic Process Markov Chain Markov Models For the telephone example: Assuming that currently all three lines are busy.5 0.12 ] Markov Chain Markov Models Markov models add the following to a Markov chain: ◮ ◮ a sequence of random variables ηt for t = 1. and Markov Models A vector v = [v1 . ..31 0. and Markov Models Stochastic Process Stochastic Process Markov Chain Markov Models Let v = [0.. P 2 = PP = s0 s1 s2 s3 ⇒ p31 = 0.15 0. T representing the signal emitted at time t ◮ We will use σj to refer to a particular signal ⇒ p(ξ2 = s2 ) = 0.html and practice. Markov Chains. then [p t (s1 ). is called an initial probability vector.22 s2 0.21 0.e.edu/Stefan Waner/realWorld/tutorialsf1/frames3 2. .22 (2) 13 / 22 14 / 22 Initial Probability Vector Stochastic Processes. n i=1 vi Stochastic Process Stochastic Process Markov Chain Markov Models = 1 is Markov Chain Markov Models The probability vector that determines the state probabilities of the observations of the ﬁrst element (state) in a Markov chain.12 s1 0. i.15 0.38 0.24 17 / 22 18 / 22 . .. . What is then the probability that after two steps exactly two lines are busy? vP 2 = vPP = [ s0 0. Markov Chains. . Markov Chains. vn ] with vi ≥ 0 and called a probability vector.22 0. .24 s3 0. and Markov Models Initial Probability Vector Stochastic Processes.25 0.20 0.hofstra.30 0. what is the probability of exactly one line being busy after two steps in time? s0 0. i. p t (sn )] = vP t−1 15 / 22 16 / 22 Initial Probability Vector: Example Stochastic Processes.0] be the initial probability vector for the telephone example of a Markov chain.37 0..21 s1 0. The initial probability vector and the transition matrix together determine the probability of the chain in a particular state at a particular point in time. and Markov Models Matrix after Two Steps: Example Stochastic Processes.Matrix Multiplication Stochastic Processes.17 0. Markov Chains. If p t (si ) is the probability of a Markov chain being in state si at time t.25 0. Markov Chains. where vi = P(ξ1 = si ).28 s3 0.3 0.. and Markov Models Markov Models Stochastic Processes.e. after t − 1 steps.43 s2 0. .

and Markov Models Markov Models (3) Stochastic Processes. .7 ColaPref. with emission probabilities: ◮ CP IP cola ice-t lem 0. after t − 1 steps. .Markov Models (1) A Markov Model consists of: ◮ ◮ ◮ ◮ Stochastic Processes. p.3 Iced Tea Pref.e.6 0.1 0. . and Markov Models Stochastic Process Markov Chain Markov Models Stochastic Process a ﬁnite set of states Ω = {s1 . Markov Chains. . and an initial vector v = [v1 . Markov Chain Markov Models 0. . p (t) (σm )] = vPt−1 A 21 / 22 22 / 22 . and Markov Models Stochastic Process Markov Chain Markov Models The probability that signal σj will be emitted at time t is then: n n Stochastic Process Markov Chain Markov Models p (si . .5 an n × m signal matrix A = [aij ].5 0.2 from: Manning/Schutze. . vn ] where vi = P(ξ1 = si ). σm }. an n × n state transition matrix P = [pij ] where pij = P(ξt+1 = sj | ξt = si ). . Markov Chains. then [p (t) (σ1 ). i. Markov Chains. . . . . Markov Chains. .7 0. an signal alphabet Σ = {σ1 .. σj ) = p (si ) · p(ηt = σj | ξt = si ) where ◮ (t) (t) p (t) (si ) is the ith element of the vector vPt−1 p (t) (σj ) = i=1 p (t) (si . . and Markov Models HMM Example 1: Crazy Softdrink Machine Stochastic Processes. 321 ¨ 19 / 22 20 / 22 Markov Models (2) Stochastic Processes. . σj ) = i=1 This means that the probability of emitting a particular signal depends only on the current state ◮ p (t) (si ) · p(ηt = σj | ξt = si ) . . 0.1 0. sn }.3 0. which for each state-signal pair determines the probability aij = p(ηt = σj | ξt = si ) that signal σj will be emitted given that the current state is si . 0. and not on the previous states Thus if p (t) (σj ) is the probability of the model emitting signal σj at time t. . .

- Stochastic HwUploaded byHaniyaAngel
- 3Vol23No2Uploaded byMaha Bose
- chap8_p1_sUploaded bychitta
- Energy Consumption Model in Ad Hoc Mobile NetworkUploaded byAIRCC - IJCNC
- Pattern and Predictability in Music.pdfUploaded byorestis willemen
- UsluUploaded byRajnish Vivekanand Singh
- 621SyllabusUploaded bySatyesh Jha
- Lecture 2006Uploaded byCarlos Reyes
- 1968 Prospects for Experimental GamesUploaded byPeng Wang
- DM52 FinishUploaded byVenkat Mec
- Stochastic Calculus 2015Uploaded byEric Parker
- JudgementUploaded bymadan321
- ouUploaded byRoadstar2008
- probability-theory-an-analytic-view (1).pdfUploaded byAnonymous 1OGFvtb0wL
- Stochastic CalculusUploaded byarthit.i
- API Testing MethodUploaded bypavankrk
- STOCHASTIC MODELING TECHNOLOGY FOR GRAIN CROPS STORAGE APPLICATION: REVIEWUploaded byAdam Hansen
- Hans Marko The bidirectional Communication Theroy - A generalization of information theory 1973Uploaded byjas_i_ti9
- Review of Fundamental Problems and Solutions in Fractal Surface GrowthUploaded byChristian Lau
- A Stochastic Approach to Computational Fluid DynamicsUploaded byLuis Horacio Martínez Martínez
- Brownian MotionUploaded byAhmed Mikaeil

- Quimica Ao Primeiro OdorUploaded byraioric
- Star Wars i Jensaarai by Rodney ThompsonUploaded byJuan Luis Herrera Cortijo
- Bases de Datos-cunUploaded byvanne_hurtado1668
- Magelis Sto Stu Hmistu855Uploaded bylexstorm
- Gómez, L. et al. Nanopartículas consolidación mat. pétreos. 2011Uploaded byTrinidad Pasíes Arqueología-Conservación
- 4. Guias de Onda 14Uploaded byLuis Torres
- TriacUploaded byperroqwerty
- Absorcion AtòmicaUploaded byEdwinMori
- Deep Learning in Character RecognitionUploaded byJuan Andres
- USB Booteable Con Varios Sistemas Operativos (ACTUALIZADO)Uploaded byDAFRANCAR
- Cuestionario para estudiar Programas de Alfabetización DigitalUploaded byjmcflly
- Pierre Bourdieu, une pensée en mouvement - DébatsUploaded byLorena Caldas
- EMC Secure Remote Services 3.04 Port RequirementsUploaded byAnthony Thomas
- Andy Field Using SpssUploaded byRomer Gesmundo
- Sample Report Briefing 2015-05-18Uploaded bySL
- Lista de Exercícios 4º Bimestre (2015) COMENTADAUploaded byprofnonocmm
- INFORME DE INNOVACIÓN DE AREA CHELEUploaded byPatricia Ruby Guzman Venegas
- E os Campos de Predominância Elementais - IUploaded byMarcelo Costa
- UT Dallas Syllabus for biol3350.501.10f taught by Ilya Sapozhnikov (isapoz)Uploaded byUT Dallas Provost's Technology Group
- 313177423-55-Modos-de-Servicio-Chinos-Por-Micros.pdfUploaded byErnesto Suarez
- Ppt-u03 Estructuras de Control Selectivas - CopiaUploaded bymikedanger
- Trabajo Aborto 1.1Uploaded byVictor Bladimir Abundez Mederos
- COSTOS TERCER EXAMENUploaded byKarlitox Tomaya Porgusto
- MÓDULO - Psicología Del Desarrollo HumanoUploaded byCarlos Garcia
- Technology, Globalisation and the Future of Work in EuropeUploaded byJoaquín Vicente Ramos Rodríguez
- greedy-dpUploaded bychirusagar
- Machupicchu Icono de La Identidad PeruanaUploaded byLucy L. Ancasi
- Cliente LivianoUploaded byfx4ea
- Java_BibleUploaded byCraftMaster SilverSmith
- Dispense di Fisica 2Uploaded byemilio