2.0K views

Uploaded by pi194043

A implementation of Continuous Emission Hidden Markov Model for sequence classification using C++,Eigen for OpenVision Machine Learning Package https://github.com/pi19404/OpenVision

save

You are on page 1of 8

Pi19404

March 13, 2014

Contents

Contents

Hidden Markov Models with continuous emission

0.1 Introduction . . . . . . . . . . . . . . . 0.2 Continuous Emission . . . . . . . . 0.2.1 Gaussian Mixture Models 0.2.2 Forward Algorithm . . . . 0.3 Code . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 3 5 5 8 8

2|8

0.1 Introduction

In this article we will look at hidden Markov models with continuous emission and its application in classification of D dimensional sequential data . The emission probabilities are modeled using Gaussian mixture model.The implementation is developed as part of OpenVision -Vision and Machine learning Library.

Let us assume that observed process Y ; n 2 N is vector valued in a euclidean space O R and X ; n 2 N are the hidden states.

n d n

n n

In the case of discrete hidden Markov model we had considered the emission to be discrete random variance and specified a emission matrix. In the case of continuous hidden Markov model the observed random variables are continuous in nature.Let the continuous random variable be defined in a dimensional Euclidean space. Y = y1 ; : : : ; y is the observed random variable.

At each time step the system generates a hidden state cording to a a state to state transition matrix T .

i;j t

X act

Once x has been generated the system generates a hidden cluster y according to the state to emission probability distribution.

t

Gaussian mixture models are used to model a multi-modal density function in Euclidean space. Each hidden state is associated with emission probability distribution.

3|8

The emission probability distribution are specified in terms of conditional distribution conditioned on the hidden state. The HMM can be viewed as a parametric model whose parameters vary with the state of the underlying hidden Markov chain. Let N be number of hidden states ,X

= x1; : : : ; x ; n 2 N

n

P (yjx ) =

Y i

X=1

k

a ( ;

i;k i;k

i;k

i;k

is a vector of dimension

i;k

is a matrix of size

DxD

Given we have a observation Y we need to determine the probability that observation is emitted by the hidden state X that is given by the modeled conditional distribution. This distribution is used to model the emission probability. Thus the continuous emission hidden Markov model is specified by transition matrix,initial probability matrix and emission probability distribution specified by Gaussian mixture model.

MatrixXf _transition; MatrixXf _initial; vector<GaussianMixture> _emission; int _nstates; int _nobs;

Let M be the number of hidden states,possible discrete value the random variable X can take.

n

Let

p y M

With a continuous HMM B corresponds to a family of parametric distributions p () where 2 R and B = (1 ; : : : ; ). For each hidden state we have a associated parametric distribution.

P (y; ) =

Y i

Xa

k

i;k

( ;

i;k

i;k

Another interpretation of above representation is,that hidden state i consists of hidden subsets k 2 N

4|8

And each of these subsets k can generate the observed symbol Y with probabilities a and probabilities that observed symbol is generate from specific hidden state is given by ( ; ).Thus total probability can be specified as a product a ( ; ),and average probability that observed symbol is generate by the hidden state i is given by the mixture model.

i;k i;k i;k i;k i;k i;k

Again the task is the present article is sequence classification. For the implementation we require implementation of Gaussian Mixture model and then see the modifications required in the forward/backward algorithm for continuous emission models.

The mixture models are specified by number of mixture components and parameters for each components k 2 M and parameter of mixture components ( ; )

i;k i;k

We can consider this to be a soft-clustering process where a continuous random vector x is assigned to clusters specified with the mixture models with some probability.

n

The clusters can be considers as hidden discrete emission which again emits a continuous random vector which is truly being observed. Thus we can consider than we have a another hidden latent variable which is the discrete emission specified by the cluter. The Class GaussianMixture which can be found in OpenVision Library which encapsulates Gaussian Mixture model is used to model the emission probabilities

The goal in the forward algorithm is to compute the probability

P (y1 ; : : : ; y

(z

) = P (x1; : : : ; x ; z )(z ) =

n n n n n

X (Z

n1

1 )P (x jz )P (z jz 1 )

n n n n

The quantity P (x jz ) was take from emission matrix in case of discrete hidden Markov model.

5|8

In case of continuous HMM this is required to be computed from the parametric distribution P (x = xjz = z ) = a ( ;

n n i k i;k i;k

i;k

Remaining computation remain the same,just instead of a lookup table in case of discrete distribution we have to perform computation according to model of parametric distribution in case of continuous observation variable. To verify the code the scikit-learn -a ptyhon machine learning package was used consider the following hidden Markov model for sequence classification.

#number of hidden states _nstates=2; #number of emissions per state _nmix=2; #length of the feature vector ndim=3; #transition probability model.transmat_=[[0.9,0.1],[0.1,0.9]]; #inital state probability model.startprob=[0.5,0.5]; #multivariate Gaussian for emission 1 a1=GMM(n_components=_nmix,covariance_type='full'); #the mean vector a1.means_=np.array([[ 0.08513302,-0.02109036,-0.54525889], [-0.02646832,0.01030935,0.55921764]]); #the covariance vector a1.covars_=np.array([[ [1.6045371 ,0,0],[0, 0.95491166,0],[0,0, 0.76062784]] [ [0.63467475,0,0],[0, 1.03482635,0],[0,0, 0.53010913]]]); #the mixture weights a1.weights_=[0.48161747,0.51838253];

#multivariate Gaussian for emission 2 a2=GMM(n_components=_nmix,covariance_type='full'); a2.means_=np.array([[ 0.06263125, -0.01238598, -0.38820614], [ 0.3781768 , -0.29539352, 0.84720285]]); a2.covars_=np.array([[ [0.66081899, 0,0],[0,1.3027587,0],[0,0, 0.68465955]], [ [1.23561827,0,0],[ 0, 0.55178899,0],[ 0,0,0.80040462]]]); a2.weights_=[ 0.74733838, 0.25266162]; model.gmms_[0]=a1; model.gmms_[1]=a2; #input feature vector sequence x=[[0.3,0.4,0.5],[0.1,0.2,0.3],[0.4,0.4,0.6]];

6|8

The libconfig++ library is used to load the parameters of the HMM from the configuration files. A sample code for testing the Continuous emission HMM is shown below

int hmm_test1(int argc,char **argv) { Config cfg; // Read the file. If there is an error, report it and exit. try { cfg.readFile(argv[1]); } catch(const FileIOException &fioex) { std::cerr << "I/O error while reading file." << std::endl; return(EXIT_FAILURE); } catch(const ParseException &pex) { std::cerr << "Parse error at " << pex.getFile() << ":" << pex.getLine() << " - " << pex.getError() << std::endl; return(EXIT_FAILURE); } //defining the object for continuous emission hmm ocv::CHMM hmm1; //load the parameters of hmm hmm1.loadConf(cfg,"hmm1"); Mat sequence=Mat(3,3,CV_32FC1); sequence.at<float>(0,0)=0.3; sequence.at<float>(0,1)=0.4; sequence.at<float>(0,2)=0.5; sequence.at<float>(1,0)=0.1; sequence.at<float>(1,1)=0.2; sequence.at<float>(1,2)=0.3; sequence.at<float>(2,0)=0.4; sequence.at<float>(2,1)=0.4; sequence.at<float>(2,2)=0.6;

7|8

The result estimated from the developed code is in accordance with the results provided by the scikit-learn package.

0.3 Code

The class CHMM encapsulates the continuous emission hidden Markov model.The class can be found in files hmm.cpp and hmm.hpp files in the OpenVision :A Machine vision library at https:// github.com/pi19404/OpenVision in ImgML subdirectory. The configuration files can be found in the conf subdirectory of the repository.

8|8

- Implementation of discrete hidden markov model for sequence classification in C++ using EigenUploaded bypi194043
- Normalized convolution for image interpolationUploaded bypi194043
- Adaptive Skin Color DetectorUploaded bypi194043
- A linear channel filterUploaded bypi194043
- ARM Neon Optimization for image interleaving and deinterleavingUploaded bypi194043
- OpenVision Library Gaussian Mixture Model ImplementationUploaded bypi194043
- Fast 2D Separable Symmetric/Anti-Symmmetric ConvolutionUploaded bypi194043
- Image enhancement using FusionUploaded bypi194043
- Multi Class Logistic Regression Training and TestingUploaded bypi194043
- Fast Asymmetric Learning for Cascade Face Detection Training/Testing UtilityUploaded bypi194043
- Gaussian Multivariate Distribution -Part 1Uploaded bypi194043
- Quick Sort algorithm in HaskellUploaded bypi194043
- Modified Canny Edge DetectionUploaded bypi194043
- polynomial approximation of a 2D signalUploaded bypi194043
- Polynomial Approximation of 1D signalUploaded bypi194043
- SBF Model Selection MKSC 14_2.pdfUploaded bySandeep Das
- Markov chain implementation in C++ using EigenUploaded bypi194043
- Seeded Region Growing using Line Scan algorithm - Stack base ImplementationUploaded bypi194043
- 07406701Uploaded bymani manis
- automatic white balance algorithm 1Uploaded bypi194043
- Control Limited Adaptive Histogram Equalization for Image EnhancementUploaded bypi194043
- 07814226Uploaded byBadharudheen V.K
- Dense optical flow expansion based on polynomial basis approximationUploaded bypi194043
- Region Growing Algorithm For UnderWater Image SegmentationUploaded bypi194043
- A simple color balance algorithmUploaded bypi194043
- The TertiaryUploaded byjuanpapo
- cvdm-iccv99Uploaded byjimakosjp
- Android OpenCV Face detectionUploaded bypi194043
- Computer Aided Diagnosis for Liver Cancer Using Statistical ModelUploaded byesatjournals
- Automatic Speech RecognitionUploaded byjuan

- Markov chain implementation in C++ using EigenUploaded bypi194043
- OpenVision Library Gaussian Mixture Model ImplementationUploaded bypi194043
- Fast 2D Separable Symmetric/Anti-Symmmetric ConvolutionUploaded bypi194043
- Dense optical flow expansion based on polynomial basis approximationUploaded bypi194043
- Polynomial Approximation of 2D image patch -Part 2Uploaded bypi194043
- Multi Class Logistic Regression Training and TestingUploaded bypi194043
- Compiling Native C/C++ library for AndroidUploaded bypi194043
- Gaussian Multivariate Distribution -Part 1Uploaded bypi194043
- Modified Canny Edge DetectionUploaded bypi194043
- polynomial approximation of a 2D signalUploaded bypi194043
- Polynomial Approximation of 1D signalUploaded bypi194043
- Uniform Local Binary Pattern and Spatial Histogram ComputationUploaded bypi194043
- Mean Shift TrackingUploaded bypi194043
- Tan and Triggs Illumination normalizationUploaded bypi194043
- C++ virtual functions and abstract classUploaded bypi194043
- C++ static members and functionUploaded bypi194043
- Random Ferns for Patch DescriptionUploaded bypi194043
- Integral Image for Computation of Mean And VarianceUploaded bypi194043
- C++ Class Members and friendsUploaded bypi194043
- Embedded Systems Programming with ARM on Linux - Blinking LEDUploaded bypi194043
- C++ InheritanceUploaded bypi194043
- Mean Shift AlgorithmUploaded bypi194043
- Local Binary PatternUploaded bypi194043
- C++ Const,Volatile Type QualifiersUploaded bypi194043

- Ecuación cuadráticaUploaded byEmma Suarez Viillegas
- Actividades de Refuerzo. Comunicación Oral y EscritaUploaded byenhiestopresto
- ar2011-0177-0186_Navarro-AstorUploaded bySikandar Khan
- innovacion y tecnologia en educacion infantil.pdfUploaded bypacoperez2008
- RAN 281 N10, Redução no cubo, recondicionamentoUploaded byedivaldo3veronese
- La Filosofía Del GéneroUploaded bybiblia1969
- Star of Emmy Winning Documentary Joins Forces with Tyler Perry Studios Alum to Create Inclusive Animated SeriesUploaded byPR.com
- D63114GC10_sg2Uploaded byAlexandre Marazo
- endoscop 335v23n01a90024179pdf001.pdfUploaded byandrea
- BibliologiaUploaded byRosmel Castellano
- De La Creacion Del Vinculo a La Perdida Del Si MismoUploaded byDavid Drah
- 50 Años RayuelaUploaded byvicox20056207
- Stoffels J Asset Pricing of Cryptocurrencies and Momentum based PatternsUploaded byAkshay Rao
- Final+Project+RFP+_+presentationUploaded byJuan Pablo Arciniegas
- Trans_v40_n1_2017 - 00 - CompletoUploaded byeudumal
- Rolemaster - Character Law (1st Edition)Uploaded bySincerus Renatus
- Global Lessons Learned NexteerUploaded byDearRed Frank
- Ch5 Retaining Walls (1-37)Uploaded byRafi Sulaiman
- Chupa CAUploaded byAnonymous 5JsYbv00GC
- Sole Joan-giro CopernicanoUploaded byrudy
- Tabulação geral 2011.1º e 2º semestreUploaded byJéssica Neto
- Clase01.pptUploaded byZofiia Rosero
- APRESENTACAO_DE_DADOS_NUM_RELATORIO_DE_ANALISEUploaded bysemsentido
- Solucion Septiembre 2002Uploaded bymvillaconejos
- Cours sur la force de vente.pdfUploaded byASSANA
- eficiencia y equidad economiaUploaded byErick LR
- etnosains.pdfUploaded byRatna Pujiwati
- Analisis Bayesiano Para Datos PoissonUploaded byCarlos Hurtado
- LogotiposUploaded byanapaula7
- EMENTA de DISCIPLINA - Topicos Em Mecanica Dos SolosUploaded byadilton25