0 Up votes0 Down votes

2 views10 pagesstatistics notes

Jun 12, 2015

© © All Rights Reserved

PDF, TXT or read online from Scribd

statistics notes

© All Rights Reserved

2 views

statistics notes

© All Rights Reserved

- Jim Dai Textbook
- A Location Predictor
- Hmms (Hidden Markov Models) Based on Anomaly Intrusion Detection Method+
- HMM.pdf
- EM Algorithm
- markov chain
- SSRN-id2472768.pdf
- Intrusion Detection Using Time-Inhomogeneous Hidden Bernoulli Model
- Sun.expert01
- Example Likelihood-Newton-Raphson
- 10.1.1.85
- 1-s2.0-S0165168405000733-main
- Modified Modified Modified Page Page Rank Rank Model Model in Investigation InvestigationInvestigationInvestigation Investigation of Criminal Criminal Criminal Gang
- Hindi Speech Recognition and Online Speaker Adaptation
- Voice Portals
- Stochastic Processes With Applications
- Curve Fitting Approximation in Internet Traffic Distribution in Computer Network in Two Market Environment
- Stochastic Solutions Manual
- 3maf
- Untitled

You are on page 1of 10

Rachel MacKay A LTMAN

Hidden Markov models (HMMs) are a useful tool for capturing the behavior of overdispersed, autocorrelated data. These models have

been applied to many different problems, including speech recognition, precipitation modeling, and gene finding and profiling. Typically,

HMMs are applied to individual stochastic processes; HMMs for simultaneously modeling multiple processesas in the longitudinal data

settinghave not been widely studied. In this article I present a new class of models, mixed HMMs (MHMMs), where I use both covariates

and random effects to capture differences among processes. I define the models using the framework of generalized linear mixed models and

discuss their interpretation. I then provide algorithms for parameter estimation and illustrate the properties of the estimators via a simulation

study. Finally, to demonstrate the practical uses of MHMMs, I provide an application to data on lesion counts in multiple sclerosis patients.

I show that my model, while parsimonious, can describe the heterogeneity among such patients.

KEY WORDS: Hidden Markov model; Latent process; Longitudinal model; Mixed model; Random effect.

1. INTRODUCTION

Hidden Markov models (HMMs) describe the relationship

between two stochastic processes: an observed process and an

underlying hidden (unobserved) process. The hidden process

is assumed to follow a Markov chain, and the observed data are

modeled as independent conditional on the sequence of hidden

states. The separability of the model for the hidden process and

the conditional model for the observed data leads to great flexibility in the overall model structure.

In particular, the observed data {Yt }nt=1 follow a HMM if

1. The hidden states, {Zt }nt=1 , follow a Markov chain.

2. Given Zt , Yt is independent of Y1 , . . . , Yt1 , Yt+1 , . . . , Yn

and Z1 , . . . , Zt1 , Zt+1 , . . . , Zn .

The HMM is fully specified by the initial and transition probabilities of the hidden Markov chain and by the distribution of

Yt given Zt . Typically, the latter would be chosen from a family

of distributions with mean depending on Zt .

Under the preceding conditions, the observed data may be

autocorrelated, but do not have the Markov property. Furthermore, the marginal distribution of Yt is a finite mixture. For

example, if Yt is a count, one might choose the distribution of

Yt given Zt = z to be Poisson with mean z . In this case, the

marginal distribution of Yt would be a finite mixture of Poisson

distributions. Thus, HMMs are one way of describing overdispersion in count (or binary) data.

HMMs have many areas of application, including speech

recognition (e.g., Levinson, Rabiner, and Sondhi 1983), gene

profiling and recognition (e.g., Krogh 1998), and the modeling

of fetal lamb movements (Leroux and Puterman 1992). Albert,

McFarland, Smith, and Frank (1994) used HMMs to model lesion counts observed on a multiple sclerosis (MS) patientan

Rachel Altman is Assistant Professor, Department of Statistics and Actuarial Science, Simon Fraser University, Burnaby, British Columbia, Canada

V5A 1S6 (E-mail: raltman@stat.sfu.ca). This article includes work from the

authors Ph.D. dissertation and was supported in part by a research grant from

the Natural Sciences and Engineering Research Council of Canada. The author

gratefully acknowledges her advisor, John Petkau, for his assistance, both academic and financial. In addition, the author thanks Farouk Nathoo for helpful

conversations, and the editors and referees whose comments led to substantial

improvement of the original manuscript. The author also expresses her appreciation to Paul Albert, Henry McFarland, and the Joseph Frank Experimental

Neuroimaging Section, Laboratory of Diagnostic Radiology Research, Clinical

Center, NIH, for sharing their MS/MRI data. Finally, she thanks Drs. Don Paty

and David Li of the UBC MS/MRI Research Group and Serono International

S.A. for providing the PRISMS data.

feature of these models is that they have been developed for

single processes.

Few HMMs for multiple processes have been considered in

the literature. Most have been developed in the context of specific applications and, hence have not been posed in their full

generality. Little is known about the theory surrounding these

models.

Hughes and Guttorp (1994) presented one example of a

HMM for multiple processes: a multivariate HMM for data consisting of daily binary rainfall observations (rain or no rain)

at four different stations. These time series are assumed to be

functions of the same underlying process. Given the hidden

state at time t, the observations at this time are considered to

be independent of all observations at other time points. They

may, however, depend on one another. Turner, Cameron, and

Thomson (1998) and Wang and Puterman (1999), working in

the setting of Poisson count data, developed models for independent processes, each of which depends on a different underlying process. To account for between-subject differences, these

authors incorporated covariates in the conditional model for

the observed process. MacDonald and Zucchini (1997, chap. 3)

also provided a brief discussion of this idea. Between-subject

differences can also occur in the transition probabilities; for example, in their rainfall model, Hughes and Guttorp (1994) included covariates in the model for the hidden process.

The addition of random effects is a natural extension of these

models. In the subject-area literature, HMMs with random effects have appeared in a limited way. For instance, Humphreys

(1997, 1998) suggested such a model for representing labor

force transition data. He worked with binary observations on

employment status (employed or unemployed) that are assumed

to contain reporting errors. The misclassification probabilities,

as well as the transition probabilities, depend on a subjectspecific random effect. Seltman (2002) proposed a complex

biological model for describing cortisol levels over time in a

group of patients. The initial concentration of cortisol in each

patient is modeled as a random effect.

The goal of this article is to develop a new class of models,

mixed hidden Markov models (MHMMs), which unify existing HMMs for multiple processes and provide a general framework for working in this context. These models extend the class

201

Journal of the American Statistical Association

March 2007, Vol. 102, No. 477, Theory and Methods

DOI 10.1198/016214506000001086

202

of HMMs by allowing the incorporation of covariates and random effects in both the conditional and the hidden parts of the

model. The advantages of MHMMs are numerous. First, modeling multiple processes simultaneously permits the estimation

of population-level effects, as well as more efficient estimation

of parameters that are common to all processes. Second, these

models are relatively easy to interpret. Finally, MHMMs permit greater flexibility in modeling correlation structure because

they relax the assumption that the observations are independent

given the hidden states.

This article is organized as follows. In Section 2 I describe

the MS study that provided the practical motivation for this

work. In Section 3 I define the class of MHMMs and give some

insight into their interpretation. I discuss estimation of the parameters of these models in Section 4, and in Section 5, for

some simple models, conduct a simulation study to investigate

small sample properties. I illustrate the application of MHMMs

to the MS data in Section 6 and conclude with a discussion in

Section 7.

2. MULTIPLE SCLEROSIS DATA

Magnetic resonance imaging (MRI) scans of relapsing

remitting multiple sclerosis (RRMS) patients are one source

of data that may be appropriately modeled using the HMM

framework. Patients afflicted with this disease have symptoms

that worsen and then improve in alternating periods of relapse

and remission. One such symptom is lesions in the brain; it is

now believed that exacerbations are associated with increased

numbers of lesions (e.g., Sormani, Bruzzi, Comi, and Filippi

2002). Interferon -1a (Rebif) is a common treatment and has

been shown to reduce MRI activity, relapse rate, and clinical

progression according to the Expanded Disability Status Scale

(PRISMS Study Group 1998; Li and Paty 1999; PRISMS Study

Group and the University of British Columbia MS/MRI Analysis Group 2001).

Because the number of lesions depends on whether the patient is in relapse or remission, one would expect the distribution of the lesion counts to be a finite mixture. In addition,

one would expect counts observed on the same patient to be

autocorrelated. Thus, a two-state HMM may be reasonable for

describing such data. Indeed, Albert et al. (1994) used these

ideas in the development of a model for lesion counts observed

monthly on three RRMS patients over a period of approximately 2 years. They used a HMM to analyze each patients

data individually, treating the data as three unrelated processes.

In particular, for a given patient, they let {Zt } be an unobserved,

stationary Markov chain with P(Zt = 1|Zt1 = 1) = P(Zt =

1|Zt1 = 1) = and P(Zt = 1) = P(Zt = 1) = 1/2. They

then modeled the observed lesion count at time t, Yt , conditional

on Zt as Poisson (t ), with

Zt = 1

t1 ,

t =

(1/ )t1 ,

Zt = 1.

Here 0 is an unknown parameter. At first glance, it may appear that this model is not a HMM, because t depends on

Z1 , . . . , Zt1 as well as on Zt . However, Altman and Petkau

(2005), by reparameterizing the model, demonstrated that it is,

in fact, a nonhomogeneous HMM. These authors also pointed

Different Models

AIC

Patient

HMM

Poisson

Mixture

1

2

3

138.54

151.40

137.20

134.54

163.66

139.08

138.94

150.66

132.36

out that, under this model, t is restricted to a discrete number of values (evenly spaced on the log scale). Therefore, if the

process {Yt } is stationary, one might expect the model of Albert

et al. (1994) to be similar to a stationary Poisson HMM with

K < hidden states.

There are a number of limitations to the approach of Albert

et al. (1994). First, for each patient, the authors compared the fit

of their HMM to the model that assumes independent Poisson

counts. Based on the Akaike information criterion (AIC) values (reproduced in columns 2 and 3 of Table 1), they claimed

evidence of autocorrelation in the data from patients 2 and 3.

However, the HMM differs from the Poisson model in that it allows not only for autocorrelation but for overdispersion as well.

When one fits a mixture of two Poisson distributions (a model

that allows for overdispersion but not autocorrelation), one sees

that, of the three models, the mixture model actually yields the

lowest AIC values for patients 2 and 3 (column 4 of Table 1). In

other words, although one expects autocorrelation to be present,

it is not detectable using individual HMMs.

Second, as Albert et al. (1994) noted, MS is a very heterogeneous disease, and its degree and behavior are expected to vary

considerably among patients. Although modeling each patients

data separately certainly allows for interpatient differences, individual models require a large number of parameters, which

leads to increased uncertainty in all parameter estimates. In

addition, this approach prevents the assessment of populationlevel effects. For example, in an MS clinical trial, models for

individual patients would typically not be sufficient given the

usual sample size per patient and heterogeneity among patients.

Third, despite its simple form, the model of Albert et al.

(1994) is hard to interpret. Specifically, consider the implied

marginal moment structure:

1

1

,

E[Y1 ] = 0 +

2

1

1

E[Y2 ] = 0 2 + 2(1 ) + 2 ,

2

1

1

E[Y3 ] = 0 3 2 + +

2

1 2

2

[2(1 ) + (1 ) ] + 3 ,

..

.

The model assumes that the mean lesion counts follow a very

complicated trend. It would be difficult to justify this assumption in practice.

Finally, little is known about the theoretical properties of

nonhomogeneous HMMs. For instance, there are no tools for

model assessment in this setting.

203

of MHMMs: the need for an interpretable, parsimonious model

that allows interprocess differences while borrowing strength

across processes. MHMMs are most advantageous in the setting where there are many processes. Therefore, to demonstrate the performance of my models in the MS setting, rather

than continuing to work with the data of Albert et al. (1994),

I use a larger MS/MRI dataset that was generated as part of

the PRISMS study (PRISMS Study Group 1998). This study

was a multicenter trial designed to investigate the use of interferon -1a in treating RRMS. Patients were randomly assigned to the placebo group, the low-dose group, or high-dose

group. The 39 patients who participated at the University of

British Columbia (UBC) received monthly scans for approximately two years (similar to the data of Albert et al. 1994 data).

demonstrate both the heterogeneity among patients and the effectiveness of the treatment in reducing mean lesion count. The

counts in each treatment group are most frequently 0, but range

up to 18 in the placebo group, 8 in the low-dose group, and 7 in

the high-dose group. Some scans are missing in each treatment

group (assumed at random). The PRISMS data are analyzed

further in Section 6.

3. MODEL DEVELOPMENT

In this section I specify the MHMM class and discuss the

interpretation of these new models. For concreteness, I present

my ideas in the context where each process corresponds to repeated observations on a patient.

Let Yit be the observation and let Zit be the hidden state associated with patient i, i = 1, . . . , N, at time t, t = 1, . . . , ni .

204

equally

spaced. However, missing data are permissible. Let N

i=1 ni =

nT . Yi denotes the ni -dimensional vector of observations on patient i and Y the nT -dimensional vector of all observations. The

vectors of hidden states, Zi and Z, are defined analogously. The

generic notation f (x) is used to denote the density (or probability mass function) of a random variable (or vector), X.

I make the following additional assumptions:

1. Zit takes on values from a finite set, {1, 2, . . . , K}, where

K is known.

i

2. Given the random effects, {Zit }nt=1

are Markov chains.

These processes may or may not be stationary.

3. Conditional on the random effects, the ith process,

i

, is a HMM, and observations on different pro{Yit }nt=1

cesses are independent.

i

, conditional on the random effects,

If the processes {Zit }nt=1

are stationary with unique stationary distributions, one may use

these as the initial probabilities. Otherwise, the initial probabilities are typically nuisance parameters. In this case, it may be

sensible to assume that they are fixed parameters and are the

same for all patients (or perhaps the same for all patients in a

given treatment group). One generally has very little information with which to estimate these probabilities, so allowing for

further complexity does not seem necessary.

for the Observed Data

I first focus on the addition of random effects to the conditional model for the observed data and assume that random

effects do not appear in the model for the hidden processes.

In particular, I assume that the hidden processes are homogeneous with transition probabilities, {Pk }, and initial probabilities, {k }, common to all subjects. In the RRMS setting, such

a model would allow the mean lesion count to vary among patients.

Borrowing from the theory of generalized linear mixed models (GLMMs) (e.g., McCulloch and Searle 2001) and letting

be the vector of all model parameters, I assume that, conditional

on the random effects, u, and the hidden states Z, {Yit } are independent with distribution in the exponential family:

f (yit |Zit = k, u, )

= exp (yit itk c(itk ))/a() + d(yit , )

(1)

and

itk = k + xit k + witk u.

model could be extended further by allowing the parameter to

vary among patientsa case I do not consider here. Henceforth,

this model will be referred to as Model I.

The likelihood for this model is

L(; y)

f (y|z, u, )f (z; )f (u; ) du

=

u z

ni

N

u z

f (yit |zit , u, )

i=1 t=1

zi1

i=1

N

u i=1

ni

Pzi,t1 ,zit f (u; ) du

t=2

zi

ni

Pzi,t1 ,zit f (yit |zit , u, ) f (u; ) du.

(3)

t=2

standard HMM and can be expressed more simply as a product

of matrices. Specifically, for a given value of u, let Ai1 be the

it

vector with elements Ai1

k = k f (yi1 |Zi1 = k, u) and let A be the

it

matrix with elements Ak = Pk f (yit |Zit = , u), t > 1. Let 1 be

the K-dimensional vector of 1s. Then

n

N

i

i1

it

A 1 f (u; ) du.

L(; y) =

(A )

(4)

u i=1

t=2

Thus, the integrand is simple to compute, and it is only the complexity of the integral that can make evaluating and maximizing

(4) challenging. In most applications, however, one would expect (4) to reduce to a much simpler form.

Example 1 (Single, patient-specific random effect). Let ui be

a random effect associated with the ith patient, i = 1, . . . , N,

and let {ui } be iid. Then, observations on different patients are

independent, and, given ui and the sequences of hidden states,

the observations on patient i are independent. In this case, (3)

and (4) simplify to a one-dimensional integral:

L(; y)

=

i=1 ui

(2)

Here k is the fixed effect when Zit = k, xit are covariates for

patient i at time t, and witk is the row of the model matrix for

the random effects for patient i at time t in state k. I denote

the distribution of the random effects by f (u; ) and assume

that the random effects are independent of the hidden states.

I take E[u] 0 to avoid problems with model identifiability.

The notation u (as opposed to ui ) indicates that a random effect

could be common to more than one patienta generalization

that would be helpful if, for example, data were collected on

patients from multiple centers. The form of (1) assumes that

the link function is canonical, an assumption that is not strictly

i=1 ui

ni

Pzt1 ,zit f (yit |zit , ui , ) f (ui ; ) dui

t=2

i1

(A )

ni

it

1f (ui ; ) dui .

(5)

t=2

It may also be desirable to allow the parameters of the hidden

Markov chain to vary randomly among patients. For example,

in the RRMS context, patients may spend differing proportions

205

of time in relapse and remission. To explore this class of models, I again specify the conditional model for the observed data

by (1) and (2), but I now allow the model for the hidden process

to vary among patients.

i

In particular, I assume that {Zit |u}nt=1

is a Markov chain and

that Zit |u is independent of Zjs |u for i = j. Any model for these

Markov chains must satisfy the constraints that the transition

probabilities lie between 0 and 1 and that the rows of the transition probability matrices sum to 1. Thus, I propose modeling

the transition probabilities as

P(Zit = |Zi,t1 = k, u, )

+ x + w u}

exp{k

it k

itk

= K

h=1 exp{kh

+ w u}

+ xit kh

itkh

(6)

from those in (2). The vector u now contains the random effects associated with both the hidden process and the conditional model for the observed data. To prevent overparameteri 0 for all k and set w to be a row

zation, I define kK

kK

itkK

of 0s for all i, t, k. I call this model Model II.

The likelihood associated with Model II is very similar to (4).

Again, I can write

n

N

i

i1

it

L(; y) =

A 1 f (u; ) du,

(A )

(7)

u i=1

t=2

it

i1

where now I define the quantities Ai1

k and Ak as Ak =

i

it

k f (yi1 |Zi1 = k, u, ) and Ak = P(Zit = |Zi,t1 = k, u, )

f (yit |Zit = , u, ), t > 1.

the random effects are patient specific, so that observations on

different patients are independent. In particular, for patient i,

I model the transition probabilities as

+ u }

exp{k

ik

P(Zit = |Zi,t1 = k, uik , ) = K

,

+ u }

exp{

h=1

kh

ikh

u 0 for all i, k. The likelihood for this model

where kK

ikK

can be simplified as in (5). However, I now need K(K 1) (possibly correlated) random effects for each patient. For K > 2, the

resulting integral could be prohibitively complex.

It is easy to create complex models using latent variables.

However, caution must be exercised in order to avoid unwanted

implications of ones modeling choices. One way of understanding the impact of the fixed and random effects is to examine the resulting marginal moments of the observed process.

In the MHMM setting, the marginal moments do not, in general, have closed forms. Nonetheless, in the case of Model I,

i

when {Zit }nt=1

is stationary, closed forms do exist for certain

common distributions of Yit |Zit , u (e.g., Poisson, normal) and of

the random effects (e.g., multivariate normal). Interpreting the

model under these circumstances is relatively easy (see, e.g.,

Sec. 6).

One can also interpret the impact of the random effects on

the asymptotic covariance. Consider the case where {Zit |u} is

homogeneous and stationary with unique stationary distribution

Under mild conditions, it can then be shown that

cov[Yit , Yis ]

as |t s| ,

there are no random effects in the model (i.e., when one assumes the same model for each patient). Thus, the random effects allow a long-range, positive dependence in each patients

observations.

In addition, the different roles played by the random effects

in the two parts of the model are noteworthy. Specifically, the

effect of including random effects in the conditional model for

the observed data is to relax the assumption that the observations are conditionally independent given the hidden states. In

contrast, the role of the random effects in the model for the

hidden process is to relax the assumption that this process is

Markovian.

4. ESTIMATION

Traditionally, the expectationmaximization (EM) algorithm

has been used to estimate the parameters of a HMM. However,

the EM algorithm is notoriously slow to converge, and thus,

in this setting, I prefer to maximize the likelihood directly. In

particular, I evaluate the likelihood as a product of matrices

(as described in Sec. 3) and then use a quasi-Newton method

(e.g., Nash 1979) to locate the maximum likelihood estimators

(MLEs). For standard HMMs, I have found this approach to

be much more efficient. Moreover, the quasi-Newton routine

has the added benefit of producing, as a by-product, the estimated variancecovariance matrix of the parameter estimators

(as given by the inverse of the observed Fisher information matrix).

The estimation of MHMMs poses a more challenging problem. In the special cases considered by Humphreys (1997,

1998), (4) can be evaluated analytically. In these cases, the response is binary and the hidden Markov chain has two states.

The random effects are assumed to follow a log-Gamma distribution, and the complementary log-log link is used. Typically,

though, neither (4) nor (7) will have a closed form. Seltman

(2002) took a Bayesian approach to estimating the parameters

of his specialized model (an example of a MHMM with one

random effect), claiming that the frequentist approach is intractable. I disagree with this claim, having successfully implemented two of the frequentist estimation methods discussed

below.

4.1 Numerical Integration

In the case where there are only a few random effects, numerical methods of integration work well. In particular, for

common choices of distribution for u (e.g., multivariate normal), Gaussian quadrature methods offer both accuracy and efficiency. The quasi-Newton method can then be used to maximize the approximated likelihood.

Two issues concerning such methods are the starting values

and the number of quadrature points. With respect to the former, I recommend trying a variety of starting values to improve

the chances of locating the global maximum. With respect to

the latter, I recommend increasing the number of quadrature

206

points with the number of iterations of the quasi-Newton routine. To choose the various numbers of points, the likelihood

can be evaluated (at the starting values) for different numbers

of quadrature points, q1 < q2 < . I select qk as the number

of quadrature points if the use of qk+1 , qk+2 , . . . points does not

lead to a substantial change in the value of the integral, where

the definition of substantial depends on the number of iterations executed by the quasi-Newton algorithm. (I accept less

accuracy for early iterations, but demand high accuracy for the

final iterations.)

I have used these techniques to estimate a variety of MHMMs

in the RRMS context. Using a dual 1.2-GHz Athlon processor,

the fitting of a model with no random effects takes less than

1 second, that with one random effect takes approximately 12

seconds, that with two or three correlated random effects takes

several hours, and that with four random effects (two correlated

random effects in the conditional model, independent of two

additional correlated random effects in the hidden model) takes

several days.

4.2 Monte Carlo Methods

For larger numbers of random effects, numerical integration

methods are no longer appropriate. For such complex models,

estimation is significantly more difficult. Of existing estimation

methods, the Monte Carlo expectation-maximization (MCEM)

algorithm (McCulloch 1997) seems to be the most feasible. In

this setting, both the states of the hidden Markov chain and the

random effects are treated as missing data. The complete loglikelihood is then given by

log Lc (; y, z, u)

= log f (y|z, u, ) + log f (z|u, ) + log f (u; )

=

ni

N

i=1 t=1

log zi1

i=1

ni

N

i=1 t=2

y, z, u) conditional on the observed data and parameter estimates at iteration p (denoted by p ). Note that

p

f (z, u|y, )

=

where

f (y|z, uj , p )f (z|uj , p )

.

k

p

k

p

k=1

z f (y|z, u , )f (z|u , )

hj (z) = B

The values of hj (z) are easy to compute using the fact that the

standard HMM likelihood can be written as a product of matrices (see Sec. 3).

For the M-step, note that, typically, the parameters involved in f (yit |zit , u, ), the initial probabilities, the transition probabilities, and f (u; ) form disjoint sets. (One notable exception occurs when the hidden Markov chains are

stationary conditional on u, in which case the initial probabilities are functions of the

probabilities.) In other

ni

transition

log

f (yit |zit , u, )|y, p ],

words, the expressions E[ N

i=1

N t=1

N

ni

p

E[ i=1 log zi1 |y, ], E[ i=1 t=2 log Pzi,t1 ,zit |y, p ], and

E[log f (u; )|y, p ] can usually be maximized separately, improving the efficiency of the procedure.

Assuming that the s are treated as unknown parameters, it

is straightforward to show that

N B

z hj (z)1(zi1 = )

i=1

j=1

p+1

,

=

N

where 1(zi1 = ) = 1 if zi1 = and 0 otherwise. Numerical

maximization (e.g., via the quasi-Newton routine) would ordinarily be required in order to obtain updates for the other parameter estimates.

Although this method is flexible and theoretically sound, the

number of samples, B, required to approximate the E-step accurately is an important practical consideration. For integrals of

high dimension, B may be very large, resulting in a prohibitive

computational burden. I recommend the same approach as for

the quadrature method discussed in Section 4.1, namely, to increase the values of B with the number of EM iterations. To

choose these different values, I suggest, given B and starting

values, estimating

N

p

log zi1 |y, ,

E

E

E

i=1

ni

N

log f (yit |zit , u, )|y,

i=1 t=1

ni

N

log Pzi,t1 ,zit |y, p ,

and

i=1 t=2

f (y; p )

=

.

p

p

p

z f (y|z, u, )f (z|u, )f (u; ) du

Therefore, if one generates samples u1 , . . . , uB from f (u; p )

(via a random number generator or more sophisticated methods

such as Gibbs sampling), one obtains the approximation

E[log Lc (; y, z, u)|y, p ]

1

log Lc ( p ; y, z, uj )hj (z),

B

z

B

j=1

several times. The accuracy of the E-step is reflected in the

amount of variation in these estimates.

I have successfully used the MCEM algorithm to fit several different models in the RRMS setting. The most complex

of these included three random effects (one in the conditional

model, independent of two additional correlated random effects

in the hidden model). In this case, I used B = 50 for the first 5

iterations, B = 5,000 for the next 10 iterations, and B = 50,000

for subsequent iterations. The computational time required was

approximately 3 days.

It is of interest to note that, in their estimation of a GLMM using the MCEM method, Chan and Kuk (1997) selected a much

of the complexity of the MHMM as compared to the GLMM

in particular, of the difficulty in parameter estimation when the

observations are not assumed independent given the random effects.

4.3 Simulated Maximum Likelihood

Another way to estimate the parameters of a MHMM is to approximate the likelihood directly, that is, by generating samples

u1 , . . . , uB from an importance sampling distribution, g(u), and

then computing

1 f (y|uj , )f (uj ; )

log L(; y)

.

B

g(uj )

B

j=1

function, and large values of B may be required.

McCulloch (1997) and Kuk and Cheng (1999), working in

the GLMM context, commented on the poor performance of

this method when used in isolation. I had a similar experience

in the MHMM setting. However, this method may be used successfully after executing a number of iterations of the MCEM

algorithm. One advantage of this approach is that standard errors are readily obtained as a function of the maximized loglikelihood.

4.4 Other Estimation Methods

Methods such as penalized quasi-likelihood (Breslow and

Clayton 1993) and h-likelihood (Lee and Nelder 1996), which

were developed for the estimation of GLMMs, rely on simple

forms for the derivatives of log f (y|u, ) with respect to . Unlike the GLMM case, in a MHMM, the Yit s are not independent

conditional on u. Thus, these derivatives are very complicated,

and neither method seems appropriate here.

5. SIMULATION STUDY

In this section I present the results of a small simulation study

designed to investigate how the uncertainty in my parameter

estimates depends on the model complexity and structure. Because my application of interest is the MS data described in Section 2, I conduct my study in this context. I consider a selection

of models that may be appropriate for such data, focusing my

study on models with one or no random effects. (Models with

more than one random effect take at least several hours to fit;

for this reason, I exclude them from my study.) For each model,

I execute the following two steps:

1. I generate a sequence of 20 counts for each of 30 independent patients divided equally into treatment and control groups (similar to the UBC data, but with only two

treatment groups).

2. I fit the model to the simulated data and record the parameter estimates.

Let Yit be the count observed at time t on patient i and let Zit

be the associated hidden state (remission = 1, relapse = 2). Let

xi = 1 if patient i is in the treatment group and xi = 0 otherwise.

I consider the following six models:

207

Yit |Zit Poisson(eaZit +xi ), logit{P(Zit = 1|Zi,t1 =

z)} = z

Cfr: Treatment and random effect in the conditional model;

Yit |Zit , ui Poisson(eaZit +xi +ui ), logit{P(Zit = 1|

Zi,t1 = z)} = z , ui N(0, e2 )

Cfr.Hf: Treatment effect in the conditional and hidden models,

random effect in the conditional model; Yit |Zit , ui

Poisson(eaZit +xi +ui ), logit{P(Zit = 1|Zi,t1 = z)} =

z + z xi , ui N(0, e2 )

Cf.Hf: Treatment effect in the conditional and hidden models;

Yit |Zit Poisson(eaZit +xi ), logit{P(Zit = 1|Zi,t1 =

z)} = z + z xi

Cf.Hr: Treatment effect in the conditional model, random effect in the hidden model; Yit |Zit Poisson(eaZit +xi ),

logit{P(Zit = 1|Zi,t1 = 1, ui )} = 1 + ui , logit{P(Zit =

1|Zi,t1 = 2)} = 2 , ui N(0, e2 )

Cf.Hfr: Treatment effect in the conditional and hidden models, random effect in the hidden model; Yit |Zit

Poisson(eaZit +xi ), logit{P(Zit = 1|Zi,t1 = 1, ui )} =

1 + 1 xi + ui , logit{P(Zit = 1|Zi,t1 = 2)} = 2 + 2 xi ,

ui N(0, e2 ).

The labeling of the models indicates whether fixed or random

effects are included and whether they appear in the conditional

or hidden models. For example, Cfr corresponds to the model

with a fixed (f) treatment effect and a random (r) effect in

the conditional (C) model. Similarly, Cf.Hf corresponds to the

model with fixed treatment effects in both the conditional and

the hidden (H) models.

I use logit{P(Zit = 1)} 1 in all cases. I do not include

P(Zit = 2) or P(Zit = 2|Zi,t1 = z) as unknown parameters in

the models because P(Zit = 2) = 1 P(Zit = 1) and P(Zit =

2|Zi,t1 = z) = 1 P(Zit = 1|Zi,t1 = z). The parameterizations of the models were chosen so as to avoid the need for

constrained optimization (e.g., the variance of the random effect must be nonnegative, but can take on any value on the

real line).

Models Cf and Cf.Hf assume that the same model applies

to each patient in the same treatment group. Models Cfr and

Cfr.Hf allow the mean lesion count, given the hidden state,

to vary according to a patient-specific random effect. It is assumed that this random effect impacts the two conditional

means equally. Models Cf.Hr and Cf.Hfr allow the probability of remaining in remission from one month to the next to

vary among patients, but treat the probability of moving from

remission to relapse as common to all patients. This assumption would be valid in practice if the distribution of the relapse

periods tended to be relatively homogeneous across patients.

Tables 2 and 3 give the sample mean, sample standard deviation, and average asymptotic standard error of the parameter

estimates obtained based on 200 simulations from each model.

I estimated the parameters by evaluating the likelihood (using

GaussHermite quadrature with 100 points in the case of Cfr,

Cfr.Hf, Cf.Hr, and Cf.Hfr) followed by the quasi-Newton maximization routine (using the true parameter values as the starting

values).

For models Cf, Cfr, Cfr.Hf, and Cf.Hf, the mean values in

these tables are close to the true parameter values. Furthermore,

the average asymptotic standard errors agree quite well with the

208

Table 2. Sample Mean (sample standard deviation, average asymptotic standard error) of the Parameter

Estimates Under Models Cf, Cfr, and Cfr.Hf

Model

Parameter

a1

a2

1

1

2

1

2

True value

Cf

Cfr

Cfr.Hf

1.00

1.50

2.00

.00

.85

.40

1.00

1.00

.50

1.500 (.050, .051)

2.011 (.149, .149)

.071 (.536, .543)

.848 (.201, .191)

.402 (.207, .211)

NA

NA

NA

1.485 (.185, .150)

1.975 (.281, .251)

.000 (.593, .551)

.851 (.213, .205)

.423 (.227, .216)

NA

NA

.560 (.208, .175)

1.516 (.160, .151)

1.975 (.365, .317)

.007 (.608, .553)

.863 (.202, .216)

.383 (.213, .227)

1.049 (.765, .689)

.956 (.782, .732)

.603 (.240, .188)

sample standard deviations. Histograms of the parameter estimates (not shown) do not deviate substantially from the normal

distribution. Therefore, it would seem that the usual asymptotic

properties apply reasonably well when there is no random effect or one random effect in the conditional modeleven for

such a modest sample size.

For models Cf.Hr and Cf.Hfr, with the exception of the estimates of , the sample means are close to the true values,

and the histograms are well behaved. However, the asymptotic

standard errors are, in general, larger than the sample standard

deviations of the parameter estimates. This suggests that more

data may be required in order to obtain accurate standard errors

when there is a random effect in the hidden model. To confirm this theory, I ran 200 additional simulations from Cf.Hr

and Cf.Hfr but with 60 patients instead of 30. The resulting asymptotic standard errors were very close to the sample standard

deviations (agreeing to two decimal places, in most cases).

In the case of both Cf.Hr and Cf.Hfr, the histograms of the

estimates of are distinctly bimodal, with the bulk (78%) of the

estimates clustering around the true value of .5, and the rest

clustering around 9 (implying that the variance of the random

effect, e2 , is approximately 0). This behavior suggests that the

estimated variance of the random effect will be close to 0 unless

the data provide strong evidence otherwise. The simulations using 60 patients support this claim: These histograms are indeed

much closer to unimodal, with less than 5% of the estimates

clustered around 9. Clearly, a considerable amount of data is

necessary for estimating the variance of the random effect in the

hidden model. Not surprisingly, the asymptotic standard errors

for this reason, I have excluded their averages from Table 3.

In terms of the standard deviations associated with the parameter estimates, I consider first the parameters of the conditional model, a1 , a2 , and . In particular, in Cf, one sees that

these parameters are quite precisely estimated, especially the

largest of these, a2 . The inclusion of a random effect in the conditional model (Cfr and Cfr.Hf) leads to increased variability in

these estimates. Moreover, incorporating a fixed treatment effect in the hidden model results in greater variability in the estimate of (Cf.Hf and Cf.Hfr); estimating treatment effects in

both parts of the model is more difficult than estimating a treatment effect in the conditional model alone. In contrast, adding a

random effect to the hidden model (Cf.Hr) appears to have little

effect on the precision of the estimates of a1 , a2 , and .

Turning now to the parameters of the hidden model, one sees

from Cf.Hr and Cf.Hfr that the estimate of 1 tends to be more

variable when P(Zit = 1|Zi,t1 = 1) includes a random effect.

The estimates of 1 and 2 are not precise in any model and

do not seem to be affected by the addition of a random effect

in either the conditional (Cfr.Hf) or hidden models (Cf.Hfr).

The results are similar when one considers 60 patients instead

of 30. However, when one increases the magnitude of (e.g.,

from .5 to 0), the variability in the estimate of 1 in Cf.Hfr is

substantially greater than that in Cf.Hf (simulations not shown).

Therefore, as was the case with the conditional model, one sees

that adding a random effect to the hidden model results in less

precision in the estimates of the fixed effects in this part of the

model.

Table 3. Sample Mean (sample standard deviation, average asymptotic standard error) of the Parameter

Estimates Under Models Cf.Hf, Cf.Hr, and Cf.Hfr

Model

Parameter

a1

a2

1

1

2

1

2

True value

Cf.Hf

Cf.Hr

Cf.Hfr

1.00

1.50

2.00

.00

.85

.40

1.00

1.00

.50

1.503 (.050, .052)

1.985 (.228, .227)

.004 (.581, .526)

.843 (.202, .200)

.356 (.217, .219)

1.019 (.608, .592)

.977 (.824, .758)

NA

1.508 (.050, .103)

2.011 (.160, .207)

.032 (.546, .564)

.854 (.229, .299)

.397 (.207, .266)

NA

NA

2.508 (3.778, )

1.496 (.050, .126)

1.970 (.248, .311)

.043 (.545, .656)

.823 (.261, .385)

.402 (.208, .306)

1.137 (.693, .754)

.988 (.965, .887)

2.633 (3.814, )

209

any case.

In summary, this study serves to illustrate two major points:

1. At least in the context of simple MHMMs, MLEs and

their asymptotic standard errors are typically well behaved.

2. There is far more information about the parameters associated with the conditional model than those associated

with the hidden model. Therefore, in practice, one would

likely confine any complex modeling to the conditional

model.

6. APPLICATION

I now use a MHMM to analyze the PRISMS data described

in Section 2. Let Yhit be the lesion count for patient i in treatment group h at time t and let Zhit {1, . . . , K} be the associated

hidden state. I assume that h {P, L, H}, where P = placebo,

L = low dose, and H = high dose. The results of the analysis in

MacKay (2002) suggested that two hidden states are appropriate for this type of data, and thus I use K = 2 here.

MacKay (2003) provided indication that a stationary Poisson HMM may be an appropriate model for an individual patients data. Incorporating these results in the MHMM framework, I assume that, given Zhit = k and uhi , Yhit is distributed as

Poisson(hik ) with

log hik = k + kh + ui ,

where {ui } are iid, each with an N(0, e2 ) distribution. I take

1P 2P 0 so that k determines the placebo groups log

mean lesion count while in state k. Thus, kL and kH are the

effects of the low- and high-dose treatments, respectively, on

this log mean. Furthermore, I model the transition probabilities

as

logit{P(Zhit = 1|Zhi,t1 = s)} = s + sh .

Again, I take 1P 2P 0, so that s determines the placebo

groups probability of making a transition from state s to a state

of remission. Consequently, sL and sH are the effects of the

low- and high-dose treatments, respectively, on this probability.

nih

In addition, I assume that {Zhit }t=1

is stationary for all h, i with

hik P(Zhit = k).

Under this model, the marginal moments discussed in Section 3.3 have a closed form. One clear advantage of this model

over that of Albert et al. (1994) is the relative interpretability of

the marginal mean structure:

2 2

2

e

E[Yhit ] = exp

hik ek +kh .

2

k=1

cov[Yis , Yit ] = C1 + C2 |ts| ,

where C1 and C2 are constant with respect to i, s, and t, and

0 < < 1. Therefore, this model assumes that the covariance

between two observations on a patient has two components:

a base level, C1 , common to all pairs of observations on this

patient; and a secondary level, C2 |ts| , that declines to 0 as the

time between the observations increases.

model with no random effect. In other words, I let Yhit |Zhit = k

be distributed as Poisson(hk ) with log hk = k + kh . I refer

to this as the fixed model.

I fit both models using the quasi-Newton method, trying

multiple starting values. In the case of the MHMM, I used

the GaussHermite quadrature formula (with 175 quadrature

points) to approximate the likelihood. The missing data in this

study were easily accommodated; for example, if r sequential

observations were missing on a given patient, I simply used the

r-step (rather than 1-step) transition probability in the appropriate piece of the likelihood. Table 4 gives the parameter estimates, approximate standard errors, and maximized likelihoods

for both models.

The estimates of 2L , 1H , and 2H suggest that the treatment

has a beneficial effect on the mean number of lesions (the high

dose more so than the low dose). However, because of the large

standard errors associated with the estimates of sh , I am unable

to detect an effect of treatment on the transition probabilities.

(These results are consistent with those in Sec. 5 regarding the

varying precision of the parameter estimates.) To validate these

conclusions, I fit the MHMM with a treatment effect in the conditional model only and obtained a maximum value of 506.4

for the log-likelihood. In contrast, when I included a treatment

effect in the hidden model only, I obtained a maximum value

of 512.9 for the log-likelihood. When compared to the maximum value of the likelihood of the original MHMM (505.2),

these values further suggest that the treatment effect in the conditional model is significant, whereas the treatment effect in the

hidden model is not.

Table 4 also shows that, by including the random effect, one

observes quite a large increase in the log-likelihood. In addition, one sees substantial changes in some of the estimates of

the fixed effects. More formally, one can use the variance component test described in MacKay (2003) to test the hypothesis that = 0. This test is equivalent to a comparison of the

fixed model and the MHMM and involves the bootstrap. I obtain a p value of <.001, implying strong evidence in favor of

the MHMM.

In addition, in contrast to the analysis of Albert et al. (1994)

discussed in Section 2, the MHMM (and my larger sample size,

Table 4. Parameter Estimates, Standard Errors, and Maximized

Likelihoods for the UBC PRISMS Data

MHMM

Parameter

1

2

1L

2L

1H

2H

1

2

1L

2L

1H

2H

log L

Estimate

2.01

.15

1.00

1.35

2.44

3.03

1.68

1.78

1.81

.33

1.37

.13

.54

505.2

Fixed model

SE

.44

.30

.61

.57

.54

.42

.51

.50

1.15

1.25

1.27

1.21

.14

Estimate

.62

2.01

2.35

2.28

3.06

.96

3.74

1.49

1.08

2.31

1.86

1.48

NA

547.0

SE

.10

.11

.37

.19

.44

.18

.44

.62

1.16

1.19

1.09

1.14

NA

210

presumably) allow one to detect the autocorrelation among observations on the same patient. In particular, if one assumes

nih

are iid rather than Markovian, then, conditional on

that {Zhit }t=1

uhi , Yhit has a mixture distribution but is uncorrelated with Yhis ,

s = t. The log-likelihood based on this model has a maximum

value of 528.3. The associated AIC value is thus 1,076.6,

compared with 1,036.5 in the case of the MHMM.

Note that, in my preliminary analysis of these data, I attempted to fit models with a patient-specific random effect in

the hidden as well as conditional model. Under these models,

the likelihoods were very flat, and convergence was difficult to

achieve. In both the model where I allowed P(Zhit = 1|Zhi,t1 =

1) to vary according to a random effect and that where I allowed P(Zhit = 1|Zhi,t1 = 2) to vary, the maximum value of

the likelihood was 505.2 (essentially identical to the maximum achieved by the model with a random effect in the conditional model only). In addition, the estimates of the variance of

the random effects in the hidden model were approximately 0,

and the estimates of the other parameters were very similar (relative to their standard errors) across all models. Thus, for these

data, it appears that the inclusion of random effects in the hidden model is unnecessary.

To conclude, by borrowing strength across patients using a

MHMM with a random effect in the conditional model, I am

able to detect a treatment effect as well as important features

of the data such as autocorrelation. Moreover, this model provides a significantly improved fit over the fixed model with the

addition of only one extra unknown parameter ( ).

7. DISCUSSION

In summary, I have shown that the addition of one or two

random effects to the conditional model for the observed data

results in a model that can be readily interpreted and estimated.

I have applied such a model to the PRISMS data and have

demonstrated the advantages of such an approach in this context.

It is also possible to include random effects in the model for

the hidden process, but such models are more difficult to interpret and may involve high-dimensional integrals. In general,

one has less information about the parameters of the hidden

process than about the parameters of the conditional model. In

this case, extending the model to allow patient-to-patient differences on this level may explain very little additional variation

in the observed data and, hence, may not be worthwhile from

a statistical standpoint. Moreover, capturing interpatient heterogeneity in the hidden processes is still possible by incorporating

covariates in this part of the model. Therefore, in practice, one

would be more likely to use Model I than Model IIunless the

data provided strong indication that the complexity of Model II

was warranted.

Estimation of models with more than three or four random

effects is an area of ongoing research. The MCEM algorithm

can be used, in theory, to estimate any MHMM, but the computational burden required for models with large numbers of correlated random effects can be prohibitive. However, as computing power increases, so will the feasibility of using the MCEM

method to estimate increasingly complex models.

[Received September 2005. Revised August 2006.]

REFERENCES

Albert, P. S., McFarland, H. F., Smith, M. E., and Frank, J. A. (1994), Time Series for Modelling Counts From a RelapsingRemitting Disease: Application

to Modelling Disease Activity in Multiple Sclerosis, Statistics in Medicine,

13, 453466.

Altman, R. M., and Petkau, A. J. (2005), Application of Hidden Markov Models to Multiple Sclerosis Lesion Count Data, Statistics in Medicine, 24,

23352344.

Breslow, N. E., and Clayton, D. G. (1993), Approximate Inference in Generalized Linear Mixed Models, Journal of the American Statistical Association,

88, 925.

Chan, J. S. K., and Kuk, A. Y. C. (1997), Maximum Likelihood Estimation for

Probit-Linear Mixed Models With Correlated Random Effects, Biometrics,

53, 8697.

Hughes, J. P., and Guttorp, P. (1994), A Class of Stochastic Models for Relating Synoptic Atmospheric Patterns to Regional Hydrologic Phenomena,

Water Resources Research, 30, 15351546.

Humphreys, K. (1997), Classification Error Adjustments for Female Labour

Force Transitions Using a Latent Markov Chain With Random Effects, in

Applications of Latent Trait and Latent Class Models in the Social Sciences,

eds. J. Rost and R. Langeheine, New York: Waxmann Munster, pp. 370380.

(1998), The Latent Markov Chain With Multivariate Random Effects, Sociological Methods & Research, 26, 269299.

Krogh, A. (1998), An Introduction to Hidden Markov Models for Biological Sequences, in Computational Methods in Molecular Biology, eds.

S. L. Salzberg, D. B. Searls, and S. Kasif, Amsterdam: Elsevier, pp. 4563.

Kuk, A. Y. C., and Cheng, Y. W. (1999), Pointwise and Functional Approximations in Monte Carlo Maximum Likelihood Estimation, Statistics and

Computing, 9, 9199.

Lee, Y., and Nelder, J. A. (1996), Hierarchical Generalized Linear Models,

Journal of the Royal Statistical Society, Ser. B, 58, 619678.

Leroux, B. G., and Puterman, M. L. (1992), Maximum-Penalized Likelihood

Estimation for Independent and Markov-Dependent Mixture Models, Biometrics, 48, 545558.

Levinson, S. E., Rabiner, L. R., and Sondhi, M. M. (1983), An Introduction to

the Application of the Theory of Probabilistic Functions of a Markov Process

to Automatic Speech Recognition, The Bell System Technical Journal, 62,

10351074.

Li, D. K., and Paty, D. W. (1999), Magnetic Resonance Imaging Results of

the PRISMS Trial: A Randomized, Double-Blind, Placebo-Controlled Study

of Interferon-Beta1a in RelapsingRemitting Multiple Sclerosis, Annals of

Neurology, 46, 197206.

MacDonald, I. L., and Zucchini, W. (1997), Hidden Markov Models and Other

Models for Discrete-Valued Time Series, London: Chapman & Hall.

MacKay, R. J. (2002), Estimating the Order of a Hidden Markov Model, The

Canadian Journal of Statistics, 30, 573589.

(2003), Hidden Markov Models: Multiple Processes and Model Selection, unpublished Ph.D. dissertation, University of British Columbia,

Dept. of Statistics.

McCulloch, C. E. (1997), Maximum Likelihood Algorithms for Generalized

Linear Mixed Models, Journal of the American Statistical Association, 92,

162170.

McCulloch, C. E., and Searle, S. R. (2001), Generalized, Linear, and Mixed

Models, New York: Wiley.

Nash, J. C. (1979), Compact Numerical Methods for Computers: Linear Algebra and Function Minimisation, New York: Wiley.

PRISMS Study Group (1998), Randomised Double-Blind Placebo-Controlled

Study of Interferon Beta-1a in Relapsing/Remitting Multiple Sclerosis, The

Lancet, 352, 14981504.

PRISMS Study Group and the University of British Columbia MS/MRI Analysis Group (2001), PRISMS-4: Long-Term Efficacy of Interferon-Beta-1a in

Relapsing MS, Neurology, 56, 16281636.

Seltman, H. J. (2002), Hidden Markov Models for Analysis of Biological

Rhythm Data, in Case Studies in Bayesian Statistics, Vol. 5, Springer-Verlag,

pp. 397405.

Sormani, M. P., Bruzzi, P., Comi, G., and Filippi, M. (2002), MRI Metrics

as Surrogate Markers for Clinical Relapse Rate in RelapsingRemitting MS

Patients, Neurology, 58, 417421.

Turner, T. R., Cameron, M. A., and Thomson, P. J. (1998), Hidden Markov

Chains in Generalized Linear Models, The Canadian Journal of Statistics,

26, 107125.

Wang, P., and Puterman, M. L. (1999), Markov Poisson Regression Models

for Discrete Time Series, Journal of Applied Statistics, 26, 855882.

- Jim Dai TextbookUploaded byParth
- A Location PredictorUploaded byFelipe Rocha
- Hmms (Hidden Markov Models) Based on Anomaly Intrusion Detection Method+Uploaded byscientia8
- HMM.pdfUploaded byRubina
- EM AlgorithmUploaded bynaumz
- markov chainUploaded byerika
- SSRN-id2472768.pdfUploaded byAnonymous pb0lJ4n5j
- Intrusion Detection Using Time-Inhomogeneous Hidden Bernoulli ModelUploaded byJournal of Computing
- Sun.expert01Uploaded byjprakash0205
- Example Likelihood-Newton-RaphsonUploaded byEdgar Mauricio Loza Ramírez
- 10.1.1.85Uploaded bySiddharth Nair
- 1-s2.0-S0165168405000733-mainUploaded byMarkAGregory
- Modified Modified Modified Page Page Rank Rank Model Model in Investigation InvestigationInvestigationInvestigation Investigation of Criminal Criminal Criminal GangUploaded byATS
- Hindi Speech Recognition and Online Speaker AdaptationUploaded byKrutarth Patel
- Voice PortalsUploaded byAfrah Ramsheed
- Stochastic Processes With ApplicationsUploaded byAyar Yuman Paco Sanizo
- Curve Fitting Approximation in Internet Traffic Distribution in Computer Network in Two Market EnvironmentUploaded byijcsis
- Stochastic Solutions ManualUploaded byAftab Uddin
- 3mafUploaded bymodario
- UntitledUploaded byapi-256504985
- 1-s2.0-S089982561300136X-mainUploaded byshaunasweeney5144
- Stochastic Models QuizUploaded byTümay Topçu
- 10.1.1.67Uploaded byretnogunawan
- Pontius y Malason (2005).pdfUploaded byValeria Vidal
- acUploaded byveenadivyakish
- Illusion and Genuine Control_FinalUploaded byJeffrey Satinover
- Foss Lecture4Uploaded byJarsen21
- Intro to MarkocChain Chutes-And-LaddersUploaded byMahewish Jamali
- profit or pauper shott piner stem cafe feb17Uploaded byapi-264326218
- art%3A10.1186%2F1471-2105-13-135Uploaded byoloyede_wole3741

- ec01Uploaded byLakshmi Seth
- Mitchell H. Katz-Evaluating Clinical and Public Health Interventions_ a Practical Guide to Study Design and Statistics (2010)Uploaded byLakshmi Seth
- 2161.pptUploaded byLakshmi Seth
- On Slums 2011FinalUploaded byLakshmi Seth
- md02Uploaded byLakshmi Seth
- md04Uploaded byLakshmi Seth
- epm_201_2015Uploaded byLakshmi Seth
- md08.pdfUploaded byLakshmi Seth
- md07Uploaded byLakshmi Seth
- md06Uploaded byLakshmi Seth
- md05Uploaded byLakshmi Seth
- md03Uploaded byLakshmi Seth
- md01Uploaded byLakshmi Seth
- ec09Uploaded byLakshmi Seth
- ec07Uploaded byLakshmi Seth
- ec08Uploaded byLakshmi Seth
- ec06Uploaded byLakshmi Seth
- ec03Uploaded byLakshmi Seth
- ec05Uploaded byLakshmi Seth
- ec04Uploaded byLakshmi Seth
- ec02Uploaded byLakshmi Seth
- as13Uploaded byLakshmi Seth
- as12Uploaded byLakshmi Seth
- as11Uploaded byLakshmi Seth
- as10Uploaded byLakshmi Seth
- as09Uploaded byLakshmi Seth
- as08Uploaded byLakshmi Seth

- The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting SecretsUploaded byTyphonBH
- The Effect of Exercises on the Heart RateUploaded bychanlego123
- hw5-rajKorpally_rithvikgambhir.odtUploaded byKoushik Reddy
- 00 Artificial IntelligenceUploaded byGul Sahiba
- Naved_Resume 5.5yr ExpUploaded byHamid Khan
- IT6502 NotesUploaded byMarba Beny
- Comparison of Noise Removal and Echo Cancellation for Audio SignalsUploaded byamiglani
- Tutorial 1 Power Spectra for Line CodesUploaded byasifmagsi415
- Image SteganographyUploaded bynoeryati
- Module FIRUploaded byashik_na
- Vlsi Perspectiveon Security Apporoaches Related to 5g SystemsUploaded byVarun Chauhan
- PERT ND CPMUploaded bybhaskarganesh
- Splay TreesUploaded bymanishbhardwaj8131
- courseCategories2010_2011_IIUploaded byFuadMuzaki09
- Remote internship post experiment feedback form.docx.docxUploaded byKapa Vivek
- Liquid Tank Control (CCPeng)Uploaded byBolWol
- ENG2314 - Practice 3bUploaded byRafiHunJian
- Lab Manual Network securityUploaded byGauravDuhoon
- DesignOfMagneticLevitationControllersUsingJacobiLinearizationFeedbackLinearizationAndSlidingMode.pdfUploaded byAshik Ahmed
- A Nonlinear Fuzzy PID Controller via Algebraic Product AndUploaded byhieuhuech
- Ant Colony Based OptimizationUploaded bySharad Kumar Dutta
- Formal Languages and Automata TheoryUploaded byNathan Samuel Obinwafor Uzoma
- Calculus 06 Applications of the DerivativeUploaded byAkash Roy
- NP-Complete Graph BokUploaded byCmpt Cmpt
- Indian Sign Language Gesture classificationUploaded byVkg Gpta
- 3Uploaded bytheanhdtvt
- transportation problemUploaded byPritam Mukherjee
- InterpolationUploaded byMuddys007
- Numerical Methods Formula MaterialUploaded byDhilip Prabakaran
- rravi-fsttcs09-talkUploaded byRavi Infotagger

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.