- Jihad N. Y. Xiao menor Present@MF+
- Continuous time Markov chains
- 02_stationarity
- Ref
- dfre
- Chapter 1 Financial statistics
- Time Averages and Ergodicity
- Stochastic_Processes_for_Modeling_Bridge.pdf
- 10-12-28-04-39-13-850-maris
- 3rde Chapter
- easterly_lostdecades
- Stable Random Fields Phd Thesis
- 62. a Bulk Quorum Queueing System With a Random Setup Time Under N-policy and With Bernoulli Vacation Schedule 2006
- midterm_27march2017 ECONOMETRICS II.pdf
- Lecture 3 Stationary Processes
- DDP-BBA-ModuleClassification_updated Nov 2010
- Parker
- analysis of brand and category sales
- Chapter_2.pdf
- Markov Chains and Stochastic Stability
- M Tech Syllabus ECE
- Statistics in Analytical Chemistry Part 9
- Hirerachical Decision Making Under Uncertainty in Relation to Telecommunication Industries
- Lecture 14 correlation autocorrelation
- UT Dallas Syllabus for se3341.501.09f taught by Michael Baron (mbaron)
- Poisson Process.
- The Impact of Exchange Rate Fluctuation on Trade Balance in Short and Long Run_2
- Roy D. Yates, David J. Goodman-Probability and Stochastic Processes_ a Friendly Introduction for Electrical and Computer Engineers-Wiley (2014) (1)
- Table 6_9
- Queuing
- 02. RC.pdf
- 01.Capacitor and Inductor.pdf
- radar.docx
- Voltage Stabilizer
- 36th-bcs-written-syllabus.pdf
- DSP.pdf
- TV.docx
- 36th-bcs-written-syllabus (1).pdf
- 36th Bcs Written Syllabus
- Typing mouse
- Human Eye L2
- Transcript of BSC
- Doc
- Exp-2-DC
- Ms Syllabus Apece Du
- Class Routine
- Atik
- USNews Electrical Ranking 2014

Lesson 4

Umberto Triacca

Facolt`a di Economia

Universit`a dell’Aquila

umberto.triacca@gmail.com

Umberto Triacca Stationary Stochastic Processes

One-step prediction of a stochastic process

Let {x

t

} be a stochastic process.

Suppose that the variable x

t

is observed for t = 1, 2, ..., T.

We consider the problem of predicting the value x

T+1

in terms

of the values

{x

T

, ..., x

1

}

up time T

Umberto Triacca Stationary Stochastic Processes

One-step prediction of a stochastic process

More precisely, our goal is to ﬁnd the linear combination of

x

T

, ..., x

1

, which forecasts x

T+1

with minimum mean square

error. This predictor, denoted by P

T

x

T+1

, has the form

P

T

x

T+1

= α

1

x

T

+ α

2

x

T−1

+ ... + α

T

x

1

where the coeﬃcients α

1

, α

2

, ..., α

T

are determined by ﬁnding

the values that minimize the mean square error

S(α

1

, α

2

, ..., α

T

) = E(x

T+1

−α

1

x

T

−α

2

x

T−1

−... −α

T

x

1

)

2

Umberto Triacca Stationary Stochastic Processes

One-step prediction of a stochastic process

A necessary and suﬃcient set conditions for α

1

, α

2

, ..., α

T

to

minimize the mean square error is

E [(x

T+1

−α

1

x

T

−α

2

x

T−1

−... −α

T

x

1

)x

T+1−j

] = 0 j = 1, ..., T

Umberto Triacca Stationary Stochastic Processes

One-step prediction of a stochastic process

In vector notation we have

ΓΓΓααα = γγγ

where

ααα = (α

1

, α

2

, ..., α

T

)

Γ =

E(x

2

T

) E(x

T−1

x

T

) · · · E(x

1

x

T

)

E(x

T

x

T−1

) E(x

2

T−1

) · · · E(x

1

x

T−1

)

.

.

.

.

.

.

.

.

.

.

.

.

E(x

T

x

1

) E(x

T−1

x

1

) · · · E(x

2

1

)

and

γγγ = (E(x

T+1

x

T

), E(x

T+1

x

T−1

), ..., E(x

T+1

x

1

))

**Umberto Triacca Stationary Stochastic Processes
**

One-step prediction of a stochastic process

Hence, the best linear predictor of x

T+1

in terms of the values

{x

T

, x

T−1

, ..., x

1

}

up time T, is given by

P

T

x

T+1

= α

1

x

T

+ α

2

x

T−1

+ ... + α

T

x

1

where the coeﬃcients ααα = (α

1

, α

2

, ..., α

T

)

satisﬁes the

condition

ΓΓΓααα = γγγ

Umberto Triacca Stationary Stochastic Processes

One-step prediction of a stochastic process

Of course, since

E(x

t

x

s

) = Cov(x

t

, x

s

) + E(x

t

)E(x

s

) ∀t, s,

the predictor

P

T

x

T+1

= α

1

x

T

+ α

2

x

T−1

+ ... + α

T

x

1

can be determined only if the T + 1 means

E(x

T+1

), E(x

T

), ..., E(x

1

)

the T variances

Var(x

T

), Var(x

T−1

), ..., Var(x

1

),

and the T +

T(T−1)

2

covariances are known.

Umberto Triacca Stationary Stochastic Processes

The problem

In the practice, these parameters are unknown.

Thus they must be estimated.

T + 1 + 2T +

T(T −1)

2

TOO MANY PARAMETERS TO ESTIMATE!!!

Umberto Triacca Stationary Stochastic Processes

The problem

If T=100, we have to estimate 1376 parameters!!!

This is impossible because we only have a single realization of

the process.

Umberto Triacca Stationary Stochastic Processes

The solution

Restrictions which enable us to reduce the number of unknown

parameters involved in order to be able to deduce their values

from a single realization:

1

restrictions on the time-heterogeneity of the process;

2

restrictions on the memory of the process.

Umberto Triacca Stationary Stochastic Processes

Restricting the time-heterogeneity

Now we introduce an important class of stochastic processes

which exhibit time-homogeneity.

This is the class of stationary stochastic processes.

Umberto Triacca Stationary Stochastic Processes

Stationary Stochastic Process

Deﬁnition

Consider a stochastic process {X

t

; t ∈ Z}. Let {t

1

, t

2

, · · · , t

s

}

be a ﬁnite set of integers. The process {X

t

; t ∈ Z} is strongly

stationary if the joint distibution function of the vector

(X

t

1

+k

, X

t

2

+k

, ..., X

t

s

+k

) is equal with the one of

(X

t

1

, X

t

2

, ..., X

t

s

) for all integers s ≥ 1 and for all

t

1

, t

2

, · · · , t

s

, k ∈ Z.

Umberto Triacca Stationary Stochastic Processes

The meaning of Stationary

The meaning of stationarity is that the distribution of a

number of random variables of the stochastic process is

the same as we shift them along the time index axis.

Umberto Triacca Stationary Stochastic Processes

The meaning of Stationary

If {X

t

; t ∈ Z}is a strongly stationary process, then, for

example,

X

1

, X

2

, X

3

, ...

have the same distribution function.

(X

1

, X

3

), (X

5

, X

7

), (X

9

, X

11

), .....

have the same joint distribution function and further

(X

1

, X

3

, X

5

), (X

7

, X

9

, X

11

), (X

13

, X

15

, X

17

), ...

must have the same joint distribution function, and so on

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Stationary Stochastic

Process

Consider the discrete stochastic process

{X

t

; t ∈ N}

where X

t

= A, with A ∼ U (3, 7)

This process is of course strongly stationary.

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

Consider the discrete stochastic process

{X

t

; t ∈ N}

where X

t

= tA, with A ∼ U (3, 7)

This process is not strongly stationary

Umberto Triacca Stationary Stochastic Processes

Weakly Stationary Stochastic Process

In many applications this is too strict a deﬁnition and hence

we deﬁne now stationary for a wider class of processes.

Umberto Triacca Stationary Stochastic Processes

Weakly Stationary Stochastic Process

Deﬁnition

A stochastic process {X

t

; t ∈ Z} is said to be

covariance-stationary, or weakly stationary if

1

E(X

t

) = µ ∀t

2

Var(X

t

) = γ

0

< ∞ ∀t

3

Cov(X

t

, X

t−k

) = γ

k

∀t, ∀k

Umberto Triacca Stationary Stochastic Processes

Weakly Stationary processes

A stochastic process is covariance-stationary if

1

it has the same mean value, µ, at all time points;

2

it has the same variance, γ

0

, at all time points; and

3

the covariance between the values at any two time points,

t, t −k, depend only on k, the diﬀerence between the

two times, and not on the location of the points along the

time axis.

Umberto Triacca Stationary Stochastic Processes

Weakly Stationary processes

Deﬁnition

A stochastic process {u

t

; t ∈ Z} in which the random variables

u

t

, t = 0 ±1, ±2... are such that

1

E(u

t

) = 0 ∀t

2

Var(u

t

) = σ

2

u

< ∞ ∀t

3

Cov(u

t

, u

t−k

) = 0 ∀t, ∀k

is called white noise with mean 0 and variance σ

2

u

, written

u

t

∼ WN(0, σ

2

u

)

.

If the random variables u

t

are independently and identically

distributed with mean 0 and variance σ

2

u

then we shal write

u

t

∼ IID(0, σ

2

u

)

Umberto Triacca Stationary Stochastic Processes

Weakly Stationary processes

A white noise process is weakly stationary

Umberto Triacca Stationary Stochastic Processes

White noise process that is strongly stationary

A process u

t

∼ IID(0, σ

2

u

) is a strongly stationary

This follows almost immediate from the deﬁnition.

1) Use the independence property to factor the joint

distribution into the product of the marginal distributions.

2) Then use the identical distribution property that

P(u

t

≤ b) = P(u

s

≤ b) for all t, s

Umberto Triacca Stationary Stochastic Processes

White noise process that is not strongly stationary

Let w be a random variable uniformly distributed in the

interval (0; 2π). We consider the process {Z

t

; t = 1, 2, ...}

deﬁned by

Z

t

= cos(tw) t = 1, 2, ...

We have that

1

E(Z

t

) = 0 ∀t

2

Var(Z

t

) =

1

2

∀t

3

Cov(Z

t

, Z

t−k

) = 0 ∀t, ∀k

Thus Z

t

∼ WN(0, .5). However, it can be shown that is not

strongly stationary.

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

Consider the discrete stochastic process

{Y

t

; t ∈ Z}

where Y

t

= α + βt + u

t

, with u

t

∼ WN(0, σ

2

u

)

This process is not weakly stationary

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

A possible realization of the stochastic process

Y

t

= 3 + 7t + u

t

, with u

t

∼ WN(0, σ

2

u

)

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

Consider the discrete stochastic process

{y

t

; t = 0, 1, 2, ...}

where y

0

= δ < ∞ and y

t

= y

t−1

+ u

t

for t = 1,2,..., with

u

t

∼ WN(0, σ

2

u

)

This process is called random walk.

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

The mean and the variance of y

t

are be given by

E(y

t

) = δ and Var(y

t

) = tσ

2

u

Thus a random walk is not weakly stationary process.

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

A possible realization of a random walk

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Covariance-Stationary

Stochastic Process

Consider the discrete stochastic process

{y

t

; t ∈ Z}

where and y

t

= u

t

+ 3u

t−1

, with u

t

∼ WN(0, σ

2

u

)

This process is covariance-stationary.

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

A possible realization of a this process is given by

Umberto Triacca Stationary Stochastic Processes

Relation between strong and weak Stationarity

If the process {X

t

; t ∈ Z} is strongly stationary and has ﬁnite

second moments, then {X

t

; t ∈ Z} is weakly stationary.

Proof: Consider a strongly stationary process, {X

t

; t ∈ Z},

with ﬁnite second moments. By deﬁnition of strongly

stationary process follows that X

t

has the same distribution for

each t ∈ Z. Since the process {X

t

; t ∈ Z} has ﬁnite second

moment, this implies that E(X

t

) and Var(X

t

) are both

constant. Further X

t

and X

t−k

has the same joint distribution

and hence the same covariance for all k and for all t. Thus a

strongly stationary stochastic process with ﬁnite second

moments is weakly stationary.

Umberto Triacca Stationary Stochastic Processes

Relation between strong and weak Stationarity

Of course a weakly stationary process is not necessarily

strongly stationary.

There is one important case however in which weak

stationarity does imply strong stationarity.

Umberto Triacca Stationary Stochastic Processes

Relation between strong and weak Stationarity

If {X

t

; t ∈ Z} is a weakly stationary Gaussian stochastic

process, then {X

t

; t ∈ Z} is strongly stationary

Umberto Triacca Stationary Stochastic Processes

- Jihad N. Y. Xiao menor Present@MF+Uploaded bynasserji8139
- Continuous time Markov chainsUploaded bypouty567
- 02_stationarityUploaded bychrischov
- RefUploaded byY.K. Chen
- dfreUploaded bysaudade96
- Chapter 1 Financial statisticsUploaded byDomagoj Drešar
- Time Averages and ErgodicityUploaded byShubham
- Stochastic_Processes_for_Modeling_Bridge.pdfUploaded byM. Ade Surya Pratama
- 10-12-28-04-39-13-850-marisUploaded byJagadeesh Kumar Tentu
- 3rde ChapterUploaded byGowtham Raj
- easterly_lostdecadesUploaded bySanaIkram84
- Stable Random Fields Phd ThesisUploaded byLuis Horacio Martínez Martínez
- 62. a Bulk Quorum Queueing System With a Random Setup Time Under N-policy and With Bernoulli Vacation Schedule 2006Uploaded byebenesarb
- midterm_27march2017 ECONOMETRICS II.pdfUploaded byAdrian Corredor
- Lecture 3 Stationary ProcessesUploaded byronalduck
- DDP-BBA-ModuleClassification_updated Nov 2010Uploaded byHe Qing
- ParkerUploaded byandres57042
- analysis of brand and category salesUploaded byWabisa Asif
- Chapter_2.pdfUploaded byHerbertTorres
- Markov Chains and Stochastic StabilityUploaded byandre_mate
- M Tech Syllabus ECEUploaded byAnamika Pancholi
- Statistics in Analytical Chemistry Part 9Uploaded byHelder Durao
- Hirerachical Decision Making Under Uncertainty in Relation to Telecommunication IndustriesUploaded byIJSTR Research Publication
- Lecture 14 correlation autocorrelationUploaded byAmitoj Singh
- UT Dallas Syllabus for se3341.501.09f taught by Michael Baron (mbaron)Uploaded byUT Dallas Provost's Technology Group
- Poisson Process.Uploaded bygopinath12331892
- The Impact of Exchange Rate Fluctuation on Trade Balance in Short and Long Run_2Uploaded byPhú Phạm
- Roy D. Yates, David J. Goodman-Probability and Stochastic Processes_ a Friendly Introduction for Electrical and Computer Engineers-Wiley (2014) (1)Uploaded byken
- Table 6_9Uploaded byMichelle Cosico
- QueuingUploaded byShubham

- 02. RC.pdfUploaded byShaikh Hassan Atik
- 01.Capacitor and Inductor.pdfUploaded byShaikh Hassan Atik
- radar.docxUploaded byShaikh Hassan Atik
- Voltage StabilizerUploaded byShaikh Hassan Atik
- 36th-bcs-written-syllabus.pdfUploaded byShaikh Hassan Atik
- DSP.pdfUploaded byShaikh Hassan Atik
- TV.docxUploaded byShaikh Hassan Atik
- 36th-bcs-written-syllabus (1).pdfUploaded byShaikh Hassan Atik
- 36th Bcs Written SyllabusUploaded byShaikh Hassan Atik
- Typing mouseUploaded byShaikh Hassan Atik
- Human Eye L2Uploaded byShaikh Hassan Atik
- Transcript of BSCUploaded byShaikh Hassan Atik
- DocUploaded byShaikh Hassan Atik
- Exp-2-DCUploaded byShaikh Hassan Atik
- Ms Syllabus Apece DuUploaded byShaikh Hassan Atik
- Class RoutineUploaded byShaikh Hassan Atik
- AtikUploaded byShaikh Hassan Atik
- USNews Electrical Ranking 2014Uploaded byShaikh Hassan Atik