# Hidden Markov model

1

Hidden Markov model
A hidden Markov model (HMM) is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobserved (hidden) states. An HMM can be considered as the simplest dynamic Bayesian network. The mathematics behind the HMM was developed by L. E. Baum and coworkers.[1][2][3][4][5] In a regular Markov model, the state is directly visible to the observer, and therefore the state transition probabilities are the only parameters. In a hidden Markov model, the state is not directly visible, but output, dependent on the state, is visible. Each state has a probability distribution over the possible output tokens. Therefore the sequence of tokens generated by an HMM gives some information about the sequence of states. Note that the adjective 'hidden' refers to the state sequence through which the model passes, not to the parameters of the model; even if the model parameters are known exactly, the model is still 'hidden'. Hidden Markov models are especially known for their application in temporal pattern recognition such as speech, handwriting, gesture recognition,[6] part-of-speech tagging, musical score following,[7] partial discharges[8] and bioinformatics. A hidden Markov model can be considered a generalization of a mixture model where the hidden variables (or latent variables), which control the mixture component to be selected for each observation, are related through a Markov process rather than independent of each other.

Description in terms of urns
In its discrete form, a hidden Markov process can be visualized as a generalization of the Urn problem. For instance:[9] A genie is in a room that is not visible to an observer. The genie is drawing balls labeled y1, y2, y3, ... from the urns X1, X2, X3, ... in that room and putting the balls onto a conveyor belt, where the observer can observe the sequence of the balls but not the sequence of urns from which they were drawn. The genie has some procedure to choose urns; the choice of the urn for the n-th ball depends upon only a random number and the choice of the urn for the (n − 1)-th ball. The choice of urn does not directly depend on the urns further previous, therefore this is called a Markov process. It can be described by the upper part of Figure 1.

Figure 1. Probabilistic parameters of a hidden Markov model (example) x — states y — possible observations a — state transition probabilities b — output probabilities

The Markov process itself cannot be observed, and only the sequence of labeled balls can be observed, thus this arrangement is called a "hidden Markov process". This is illustrated by the lower part of the diagram shown in Figure 1, where one can see that balls y1, y2, y3, y4 can be drawn at each state. Even if the observer knows the composition of the urns and has just observed a sequence of three balls, e.g. y1, y2 and y3 on the conveyor belt, the observer still cannot be sure which urn (i.e., at

The random variable y(t) is the observation at time t (with y(t) ∈ { y1.) . for a total of can be in. x2. e.Hidden Markov model which state) the genie has drawn the third ball from. However.g. y4 }). y2. From the diagram. transition probabilities and emission probabilities (also known as output probabilities). y3.) In addition. The hidden state space is assumed to consist of one of hidden variable at time possible values. x(t) ∈ { x1. if the observed variable is an distributed according to an arbitrary multivariate Gaussian distribution. that the set of transition probabilities for transitions from any given state must sum to 1. The size of this set depends on the nature of the observed variable. given the values of the hidden variable x at all times. governed by a categorical distribution. This is called the Markov property. meaning that any one transition probability can be determined once the others are known. the observer can work out other details. by assuming that the elements are independent of each other. there is a transition probability from this state to each of the transition probabilities. On the other hand. modeled as a categorical distribution. 2 Architecture of a hidden Markov model The diagram below shows the general architecture of an instantiated HMM. leaving a total of transition parameters. The arrows in the diagram (often called a trellis diagram) denote conditional dependencies. however. if the observed variable is discrete with possible values. it is clear that the conditional probability distribution of the hidden variable x(t) at time t. while the observations themselves can either be discrete (typically generated from a categorical distribution) or continuous (typically from a Gaussian distribution). Similarly. depends only on the value of the hidden variable x(t − 1): the values at time t − 2 and before have no influence. are independent of all but a fixed number of adjacent elements. For example. there will be means and parameters controlling the covariance matrix. for each of the possible states. The transition probabilities control the way the hidden state at time is chosen given the hidden state at time . The random variable x(t) is the hidden state at time t (with the model from the above diagram. the state space of the hidden variables is discrete. there is a set of emission probabilities governing the distribution of the observed variable at a particular time given the state of the hidden variable at that time. such as the identity of the urn the genie is most likely to have drawn the third ball from. unless the value of is small. In the standard type of hidden Markov model considered here. emission parameters. the value of the observed variable y(t) only depends on the value of the hidden variable x(t) (both at time t). The parameters of a hidden Markov model are of two types. for a total of -dimensional vector for a total of parameters controlling the emission parameters over all hidden states. (In such a case. (Note. (See possible states that a possible states the section below on extensions for other possibilities.) This means that for each of the of the hidden variable at time . x3 }). or less restrictively. it may be more practical to restrict the nature of the covariances between individual elements of the observation vector. Each oval shape represents a random variable that can adopt any of a number of values. there will be separate parameters.

. as follows: These characterizations use respectively.Hidden Markov model 3 Mathematical description of a hidden Markov model General description A basic. It is useful to compare the above characterizations for an HMM with the corresponding characterizations. the distribution of each observation in a hidden Markov model is a mixture density. In a Bayesian setting. and to describe arbitrary distributions over observations and parameters. see below. all parameters are associated with random variables. The two most common choices of are Gaussian and will be the conjugate prior of Compared with a simple mixture model As mentioned above. Typically categorical. the prior distribution of the initial state is not specified. in the above model (and also the one below). with the states of the HMM corresponding to mixture components. using the same notation. of a mixture model. no particular prior distribution is assumed). Typical learning models correspond to assuming a discrete uniform distribution over possible states (i.e. non-Bayesian hidden Markov model can be described as follows: Note that. A non-Bayesian mixture model: A Bayesian mixture model: .

Hidden Markov model 4 Examples of HMMs The following mathematical descriptions are fully written out and explained. for ease of implementation. A typical non-Bayesian HMM with Gaussian observations looks like this: A typical Bayesian HMM with Gaussian observations looks like this: A typical non-Bayesian HMM with categorical observations looks like this: .

A two-level Bayesian HMM An alternative for the above two Bayesian examples would be to add another level of prior parameters for the transition matrix. with a high value of (a concentration parameter) controls the density of the (significantly above 1). That is. the path followed by the Markov chain of hidden states will be highly random. transition matrix. With a low value of (significantly below 1). specifying which states are inherently likely. meaning there will be a significantly probability of transitioning to any of the other states. meaning that the path followed by the hidden states will be somewhat predictable. replace the lines with the following: What this means is the following: 1. The greater the probability of a given state in this vector. the more likely is a transition to that state (regardless of the starting state). is a probability distribution over states. only a small number of the possible transitions out of a given state will have significant probability. the probabilities controlling the transition out of a particular state will all be similar. In other words. That is.Hidden Markov model 5 A typical Bayesian HMM with categorical observations looks like this: Note that in the above Bayesian characterizations. .

hence. i. if and. so are unbalanced. or more specifically. the set of states the above models without typically having only one or two members. 3. No tractable algorithm is known for solving this problem exactly.1. a general Dirichlet with a vector all of whose values are ). the best set of state transition and output probabilities. for all of the vectors. then for different i. then each will be sparse different vectors may distribute the mass to different ending states. Then the different vectors will be dense. almost all the probability mass is distributed over a small number of states. controls the density of . Values significantly below 1 cause a sparse vector where only a few states are inherently likely (have prior probabilities significantly above 0). imagine instead that is significantly below 1. the probability vector can be directly specified.e. This willl make the vectors sparse. a two-level model such as just described allows independent control over (1) the overall density of the transition matrix. are all the same (or equivalently. almost all will contain this state. transitions will nearly always occur to this given state. regardless of the starting state. However. or. and for the rest.Hidden Markov model 2. and (2) the density of states to which transitions are likely (i. given an output sequence or a set of such sequences. Now. the density of the N different probability vectors specifying the probability of transitions out of state i to any other state. If it is desired to inject this information into the model. so is used). . The task is usually to derive the maximum likelihood estimate of the parameters of the HMM given the set of output sequences. The Baum–Welch algorithm is an example of a forward-backward algorithm.e. controls is 0. a non-symmetric Dirichlet distribution can be used as the prior distribution over distribution with a single parameter equal to is more or less preferred.e. Values significantly above 1 cause a dense vector where all states will have similar 6 prior probabilities. On the other hand. However. but a local maximum likelihood can be derived efficiently using the Baum–Welch algorithm or the Baldi–Chauvin algorithm. controls which states are likely to get more mass than others. instead of using a symmetric Dirichlet . In both cases this is done while still assuming ignorance over which particular states are more likely than others. if the probabilities in that all states are equally likely to occur in any given to which transitions are likely to occur will be very small. to the extent that this mass is unevenly spread. and so even if all the vectors are sparse. controls the density of the transition matrix. a transition to that state will be very unlikely. one of . Now. is significantly above 1. and is a special case of the expectation-maximization algorithm. . if there is less certainty about these relative probabilities. That is. according to which state (or equivalently. Notice that there are different vectors for each starting state. if the values in that one state has a much higher probability than others. For example. the probability Imagine that the value of mass will be spread out fairly evenly over all states. there will be different states in the corresponding . for any given starting state i. i. use a general Dirichlet with values that are variously greater or less than Learning The parameter learning task in HMMs is to find. which ending states are likely to get mass assigned to them. the density of the prior distribution of states in any particular hidden variable ). Hence.

but can similarly be solved efficiently by the Viterbi algorithm. this type of problem (i. given the model's parameters and a sequence of observations. the distribution over hidden states at the end of the sequence. the state sequence that is most likely to have generated that output sequence (see illustration on the right). . i. to compute .e. given the parameters of the model and a particular output sequence. this problem. Most likely explanation The task is to compute. which here correspond to the opacities of the arrows involved). the following state sequences are candidates: 5 3 2 5 3 2 4 3 2 5 3 2 3 1 2 5 3 2 We can find the most likely sequence by evaluating the joint probability of both the state sequence and the observations for each case (simply by multiplying the probability values. as outlined below. given the parameters of the model. This problem can be handled efficiently using the forward algorithm. can be handled efficiently using the forward algorithm. Probability of an observed sequence The task is to compute. we may be interested in the most likely sequence of states that could have produced it. the probability of a particular output sequence. finding the most likely explanation for an observation sequence) can be solved efficiently using the Viterbi algorithm.Hidden Markov model 7 Inference Several inference problems are associated with hidden Markov models. This requires finding a maximum over all possible state sequences. too. This requires summation over all possible state sequences: The probability sequence of observing a of length L is given by The state transition and output probabilities of an HMM are indicated by the line opacity in the upper part of the diagram. In general. Given that we have observed the output sequence in the lower part of the diagram.e. Filtering The task is to compute. where the sum runs over all possible hidden-node sequences Applying the principle of dynamic programming. Based on the arrows that are present in the diagram.

The forward-backward algorithm is an efficient method for computing the smoothed values for all hidden state variables. there is a certain chance that Bob will perform one of the following activities. They can be represented as follows in the Python programming language: states = ('Rainy'.43}. but she knows general trends. and what Bob likes to do on average. The choice of what to do is determined exclusively by the weather on a given day. On each day. A concrete example Consider two friends. which is (given the transition probabilities) approximately {'Rainy': 0. The entire system is that of a hidden Markov model (HMM). given the parameters of the model and a particular output sequence up to time . 'Sunny' : {'walk': 0. the probability distribution over hidden states for a point in time in the past. "shop". Since Bob tells Alice about his activities. The transition_probability represents the change of the weather in the underlying . 'Sunny': 0. the parameters of the HMM are known. 'shop'.3}. Alice has no definite information about the weather where Bob lives. or "clean". the statistical significance indicates the false positive rate associated with accepting the hypothesis for the output sequence. and cleaning his apartment. Alice knows the general weather trends in the area. The particular probability distribution used here is not the equilibrium one. it may also be interesting to ask about statistical significance. Statistical significance For some of the above problems. Based on what Bob tells her he did each day. start_probability represents Alice's belief about which state the HMM is in when Bob first calls her (all she knows is that it tends to be rainy on average).Hidden Markov model 8 Smoothing The task is to compute. 'Sunny' : {'Rainy': 0. they are hidden from her. 'clean': 0. There are two states.4.3.6. shopping. What is the probability that a sequence drawn from some null distribution will have an HMM probability (in the case of the forward algorithm) or a maximum state sequence probability (in the case of the Viterbi algorithm) at least as large as that of a particular output sequence?[10] When an HMM is used to evaluate the relevance of a hypothesis for a particular output sequence. i. 'Sunny': 0.4. 'clean') start_probability = {'Rainy': 0. In other words. 'Sunny') observations = ('walk'. } emission_probability = { 'Rainy' : {'walk': 0. who live far apart from each other and who talk together daily over the telephone about what they did that day. those are the observations. to compute for some . but she cannot observe them directly.6}. depending on the weather: "walk".57. } In this piece of code.7. Bob is only interested in three activities: walking in the park.6.e. Alice and Bob. Alice tries to guess what the weather must have been like. Alice believes that the weather operates as a discrete Markov chain. "Rainy" and "Sunny".4} transition_probability = { 'Rainy' : {'Rainy': 0. 'Sunny': 0.5}. 'clean': 0.1}. 'Sunny': 0.1. 'shop': 0. that is. 'shop': 0.

9 This example is further elaborated in the Viterbi algorithm page. there is only a 30% chance that tomorrow will be sunny if today is rainy. there is a 50% chance that he is cleaning his apartment. there is a 60% chance that he is outside for a walk. Applications include: • • • • • • • • • • • Cryptanalysis Speech recognition Speech synthesis Part-of-speech tagging Machine translation Partial discharge Gene prediction Alignment of bio-sequences Activity recognition Protein folding Metamorphic Virus Detection[11] .Hidden Markov model Markov chain. In this example. Applications of hidden Markov models HMMs can be applied in many fields where the goal is to recover a data sequence that is not immediately observable (but other data that depends on the sequence is). if it is sunny. The emission_probability represents how likely Bob is to perform a certain activity on each day. If it is rainy.

is the Dirichlet distribution. The upper distribution governs the overall distribution of states. with a linear relationship among related variables and where all hidden and observed variables follow a Gaussian distribution. for each given source state.Hidden Markov model 10 History Hidden Markov Models were first described in a series of statistical papers by Leonard E. only a small number of destination states have non-negligible transition probabilities. reflecting ignorance about which states are inherently more likely than others. HMMs began to be applied to the analysis of biological sequences. the state space of the hidden variables is discrete. it is also possible to create hidden Markov models with other types of prior distributions.[12][13][14][15] In the second half of the 1980s. in which the transition probabilities between pairs of states are likely to be nearly equal. . One of the first applications of HMMs was speech recognition. such as the linear dynamical system just. Values less than 1 result in a sparse matrix in which. with non-uniform prior distributions. The single parameter of this distribution (termed the concentration parameter) controls the relative density or sparseness of the resulting transition matrix. in which case the probability of generating an observation is the product of the probability of first selecting one of the Gaussians and the probability of generating that observation from that Gaussian. It is also possible to use a two-level prior Dirichlet distribution. or equivalently both the prior distribution of hidden states (the transition probabilities) and conditional distribution of observations given states (the emission probabilities). An obvious candidate. in which the joint distribution of observations and hidden states. starting in the mid-1970s. in which one Dirichlet distribution (the upper distribution) governs the parameters of another Dirichlet distribution (the lower distribution). determining how likely each state is to occur. However. Examples of such models are those where the Markov process over hidden variables is a linear dynamical system. and approximate methods must be used. which in turn governs the transition probabilities. In simple cases. might be useful for example in unsupervised part-of-speech tagging.[16] in particular DNA. where both concentration parameters are set to produce sparse distributions. exact inference is tractable (in this case. is modeled. Since then. learning algorithms that assume a uniform prior distribution generally perform poorly on this task. a symmetric Dirichlet distribution is chosen. can be learned using Gibbs sampling or extended versions of the expectation-maximization algorithm. Hidden Markov models are generative models. The above algorithms implicitly assume a uniform prior distribution over the transition probabilities. Baum and other authors in the second half of the 1960s. Values greater than 1 produce a dense matrix. Moreover it could represent even more complex behavior when the output of the states is represented as mixture of two or more Gaussians. where some parts of speech occur much more commonly than others. such as the extended Kalman filter or the particle filter. Extensions In the hidden Markov models considered above. A choice of 1 yields a uniform distribution. One such example of distribution is Gaussian distribution. they have become ubiquitous in the field of bioinformatics. which is the conjugate prior distribution of the categorical distribution. in general. given the categorical distribution of the transition probabilities. Typically. however. using the Kalman filter). its concentration parameter determines the density or sparseness of states. Hidden Markov models can also be generalized to allow continuous state spaces. Such a two-level prior distribution.[17] Types of hidden Markov models Hidden Markov models can model complex Markov processes where the states emit the observations according to some probability distribution. in such a Hidden Markov Model the states output is represented by a Gaussian distribution. exact inference in HMMs with continuous latent variables is infeasible. The parameters of models of this sort. while the observations themselves can either be discrete (typically generated from a categorical distribution) or continuous (typically from a Gaussian distribution).

approximate techniques. Yet another variant is the factorial hidden Markov model. This second limitation is often not an issue in practice. This type of model directly models the conditional distribution of the hidden states given the observations. In practice. such as variational approaches.Markov chain). arbitrary features over pairs of adjacent hidden states can be used rather than simple transition probabilities. The disadvantage is that training can be slower than for MEMM's. could be used.e. (2) It is not possible to predict the probability of seeing an arbitrary observation.Hidden Markov model An extension of the previously-described hidden Markov models with Dirichlet priors uses a Dirichlet process in place of a Dirichlet distribution. the transition probabilities are extended to encompass sets of three or four adjacent states (or in general adjacent states). since many common usages of HMM's do not require such predictive probabilities. i.g. features of nearby observations.e. The disadvantages of such models are: (1) The types of prior distributions that can be placed on hidden states are severely limited. which models the conditional distribution of the states using logistic regression (also known as a "maximum entropy model"). The advantage of this type of model is that arbitrary features (i. This uses an undirected graphical model (aka Markov random field) rather than the directed graphical models of MEMM's and similar models. An example of this model is the so-called maximum entropy Markov model (MEMM). rather than a single Markov chain. allowing for a given state to be dependent on the previous two or three states rather than a single previous state. A variant of the previously described discriminative model is the linear-chain conditional random field. rather than modeling the joint distribution. learning in such a model is difficult: for a sequence of length . there is no need for these features to be statistically independent of each other. [18] . a junction tree algorithm could be used. The disadvantage of such models is that dynamic-programming algorithms for training them have an running time. and thus may make more accurate predictions. To find an exact solution. Such a model is called a hierarchical Dirichlet process hidden Markov model. e. a straightforward Viterbi algorithm has complexity 11 . rather. It is common to use a two-level Dirichlet process. functions) of the observations can be modeled. allowing domain-specific knowledge of the problem at hand to be injected into the model. Furthermore. or HDP-HMM for short. but it results in an complexity. All of the above models can be extended to allow for more distant dependencies among hidden states. as would be the case if such features were used in a generative model. and therefore.e. a length. The advantage of this type of model is that it does not suffer from the so-called label bias problem of MEMM's. Models of this sort are not limited to modeling direct dependencies between a hidden state and its associated observation. with states (assuming there are states for each chain). of combinations of the associated observation and nearby observations. similar to the previously-described model with two levels of Dirichlet distributions. This type of model allows for an unknown and potentially infinite number of states. It is equivalent to a single HMM. or in fact of arbitrary observations at any distance from a given hidden state can be included in the process used to determine the value of a hidden state. Finally. for adjacent states and total observations (i. which allows for a single observation to be conditioned on the corresponding hidden variables of a set of independent Markov chains. A different type of extension uses a discriminative model in place of the generative model of standard HMM's.

ucsb. pdf)..1109/TIT. PMID 19589158.1214/aoms/1177699147. edu/ ~thad/ p/ 031_10_SL/ real-time-asl-recognition-from video-using-hmm-ISCV95. doi:10.html) (by Narada Warakagoda) .1007/s11416-006-0028-7. The Annals of Mathematical Statistics 41: 164. (1966). ISBN 0-521-62971-3. M. "A tutorial on Hidden Markov Models and selected applications in speech recognition" (http:/ / www.phy. "Error statistics of hidden Markov model and hidden Boltzmann model results". cs.html) (an exposition using basic mathematics) • Hidden Markov Models (http://jedlik. L. pdf). L. Modeling Form for On-line Following of Musical Performances (http:/ / www. [3] Baum. doi:10. [4] Baum. Weiss.Hidden Markov model 12 References [1] Baum. Program in Media Arts [7] B. Master's Thesis..ac. Bishop and E. ISBN 0-13-022616-5. doi:10. (1967).1023/A:1007425814087. Rabiner (February 1989).. org/ xpl/ freeabs_all. (1972). Soules.pdf) by Mark Stamp. edu/ ~pardo/ publications/ pardo-birmingham-aaai-05.. ieee.tristanfletcher. Journal of Molecular Biology 190 (2): 159–165. [15] Xuedong Huang. Alex Pentland. Eddy. [6] Thad Starner.. J. IEEE Transactions on Information Theory 21 (3): 250. "Growth transformations for functions on manifolds" (http:/ / www. L. Anders Krogh. Hidden Markov Models for Speech Recognition. [8] Satish L. Pardo and W.18626. W. scribd. MIT. PMID 3641921.comp. G. (2006). (1975). Jordan. T. Sean R. Ariki (1990). Petrie. Bahl. cc. [12] Baker. [13] Jelinek. jsp?arnumber=212242)". Sell. [16] M. edu/ Faculty/ Rabiner/ ece259/ Reprints/ tutorial on hmm and applications. Gururaj BI (April 2003). (1975). pdf) [10] Newberg.brown. L. northwestern. M. J. (1970). .html) (University of Leeds) • Hidden Markov Models (http://www. Proceedings of the IEEE 77 (2): 257–286.1109/5. IEEE Transactions on Acoustics. "An Inequality and Associated Maximization Technique in Statistical Estimation of Probabilistic Functions of a Markov Process". Stamp. Feb 1995. External links Concepts • A Revealing Introduction to Hidden Markov Models (http://www.bme.1214/aoms/1177697196.sjsu. doi:10. pdf). Bulletin of the American Mathematical Society 73 (3): 360. "Statistical Inference for Probabilistic Functions of Finite State Markov Chains" (http:/ / projecteuclid. Mercer. (1968). (http:/ / www. "A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains". BMC Bioinformatics 10: 212. Thompson (1986). F.1162650.cs. [17] Richard Durbin. and Y. AAAI-05 Proc. PMC 2722652. [9] Lawrence R. Real-Time American Sign Language Visual Recognition From Video Using Hidden Markov Models (http:/ / www. Jack.E. doi:10. ISBN 0748601627. and Hsiao-Wuen Hon (2001). R. Spoken Language Processing. R.1975. cornell. doi:10. Machine Learning 29 (2/3): 245–273. [5] Baum. Zoubin. org/ DPubS/ Repository/ 1. The Annals of Mathematical Statistics 37 (6): 1554–1563. doi:10. aoms/ 1177699147& view=body& content-type=pdf_1).uk/roger/HiddenMarkovModels/html_dev/ main. Retrieved 28 November 2011. Cambridge University Press. San Jose State University. G. doi:10. E. com/ doc/ 6369908/ Growth-Functions-for-Transformations-on-Manifolds).cs.1090/S0002-9904-1967-11751-8.. ece.1186/1471-2105-10-212. July 2005. .1975. Eagon.. "Design of a linguistic statistical decoder for the recognition of continuous speech".edu/research/ai/dynamics/tutorial/Documents/ HiddenMarkovModels. "An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology". Alex Acero. E. Prentice Hall. [11] Wong. E. Petrie. Michael I. L. Birmingham. Retrieved 28 November 2011. • Switching Autoregressive Hidden Markov Model (SAR HMM) (http://www. N.leeds. Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. "Factorial Hidden Markov Models".edu/~stamp/RUA/HMM. L. "The DRAGON system--An overview". Inequalities 3: 1–8.1109/TASSP. doi:10. 0/ Disseminate?handle=euclid. [14] Xuedong Huang. .hu/~gerjanos/HMM/node2. (1997). cs.1016/0022-2836(86)90289-5. [18] Ghahramani. edu/ courses/ cs481/ 2004fa/ rabiner. doi:10. and Signal Processing 23: 24–29. L. Speech. Pacific Journal of Mathematics 27 (2): 211–227.uk/SAR HMM. (2009). "Maximum Likelihood Alignment of DNA Sequences".co.. " Use of hidden Markov models for partial discharge pattern classification (http:/ / ieeexplore. pdf) • A step-by-step tutorial on HMMs (http://www.. E. T. Graeme Mitchison (1999). "Hunting for metamorphic engines". [2] Baum. IEEE Transactions on Dielectrics and Electrical Insulation.1055384. Edinburgh University Press. A. gatech.. Journal in Computer Virology 2 (3): 211-229.

org/web/packages/HMM/index. (http://www.Viterbi path and probabilities.org) library for working with Hidden Markov Models.org) (home page of the GHMM Library project) • CL-HMM Library (http://code.ghmm.eng. Examples with perl source code.cs. html) (by Kevin Murphy) • Hidden Markov Model Toolkit (HTK) (http://htk.edu/~karamano/Code/HMMdotEM.html) (Implementation in C by Tapas Kanungo) • The hmm package (http://hackage. • GT2K (http://gt2k.lwebzem.kanungo. Currently only Matlab) • Hidden Markov Model (HMM) Toolbox for Matlab (http://www.ac.googlecode.cgi) .org/cgi-bin/hackage-scripts/package/hmm) A Haskell (http:// www.gatech.com/software/software.toronto.html) General Discrete-State HMM Toolbox (released under 3-clause BSD-like License.haskell.cam.com/p/cl-hmm/) (HMM Library for Common Lisp) • Jahmm Java Library (http://jahmm.haskell.com/cgi-bin/courses/hidden_markov_model_online.ca/~murphyk/Software/HMM/hmm.r-project.html) to set up.cc.Hidden Markov model 13 Software • HMMdotEM (http://www.com/) (general-purpose Java library) • HMM and other statistical programs (http://www.google.edu/) Georgia Tech Gesture Toolkit (referred to as GT2K) • Hidden Markov Models -online calculator for HMM .uk/) (a portable toolkit for building and manipulating hidden Markov models) • Hidden Markov Model R-Package (http://cran. apply and make inference with discrete time and discrete space Hidden Markov Models • GHMM Library (http://www.ubc.cs.