5 views

Original Title: 2917LEZIONE4

Uploaded by Shaikh Hassan Atik

- 123444332-c-programs-class-11
- [Chris Chatfield] the Analysis of Time Series
- Information Theory Entropy Relative Entropy
- Gakhov Time Series Forecasting With Python
- mathunit-2
- math lesson
- Lecture 4
- SSRN-id3274354
- calendar
- Lesson 2
- Probability Random Process R08 NovDec 10
- The Origins of Truncated Levy Flights
- User Satisfaction Web Browsing
- Ebachillerato Planilla Ing Comp
- DCMT
- 2003RainDisag.pdf
- marcpv
- 6sA03
- 25
- Alternative Decomposition Methods

You are on page 1of 33

Lesson 4

Umberto Triacca

Facolt`a di Economia

Universit`a dellAquila

umberto.triacca@gmail.com

Umberto Triacca Stationary Stochastic Processes

One-step prediction of a stochastic process

Let {x

t

} be a stochastic process.

Suppose that the variable x

t

is observed for t = 1, 2, ..., T.

We consider the problem of predicting the value x

T+1

in terms

of the values

{x

T

, ..., x

1

}

up time T

Umberto Triacca Stationary Stochastic Processes

One-step prediction of a stochastic process

More precisely, our goal is to nd the linear combination of

x

T

, ..., x

1

, which forecasts x

T+1

with minimum mean square

error. This predictor, denoted by P

T

x

T+1

, has the form

P

T

x

T+1

=

1

x

T

+

2

x

T1

+ ... +

T

x

1

where the coecients

1

,

2

, ...,

T

are determined by nding

the values that minimize the mean square error

S(

1

,

2

, ...,

T

) = E(x

T+1

1

x

T

2

x

T1

...

T

x

1

)

2

Umberto Triacca Stationary Stochastic Processes

One-step prediction of a stochastic process

A necessary and sucient set conditions for

1

,

2

, ...,

T

to

minimize the mean square error is

E [(x

T+1

1

x

T

2

x

T1

...

T

x

1

)x

T+1j

] = 0 j = 1, ..., T

Umberto Triacca Stationary Stochastic Processes

One-step prediction of a stochastic process

In vector notation we have

=

where

= (

1

,

2

, ...,

T

)

E(x

2

T

) E(x

T1

x

T

) E(x

1

x

T

)

E(x

T

x

T1

) E(x

2

T1

) E(x

1

x

T1

)

.

.

.

.

.

.

.

.

.

.

.

.

E(x

T

x

1

) E(x

T1

x

1

) E(x

2

1

)

and

= (E(x

T+1

x

T

), E(x

T+1

x

T1

), ..., E(x

T+1

x

1

))

One-step prediction of a stochastic process

Hence, the best linear predictor of x

T+1

in terms of the values

{x

T

, x

T1

, ..., x

1

}

up time T, is given by

P

T

x

T+1

=

1

x

T

+

2

x

T1

+ ... +

T

x

1

where the coecients = (

1

,

2

, ...,

T

)

satises the

condition

=

Umberto Triacca Stationary Stochastic Processes

One-step prediction of a stochastic process

Of course, since

E(x

t

x

s

) = Cov(x

t

, x

s

) + E(x

t

)E(x

s

) t, s,

the predictor

P

T

x

T+1

=

1

x

T

+

2

x

T1

+ ... +

T

x

1

can be determined only if the T + 1 means

E(x

T+1

), E(x

T

), ..., E(x

1

)

the T variances

Var(x

T

), Var(x

T1

), ..., Var(x

1

),

and the T +

T(T1)

2

covariances are known.

Umberto Triacca Stationary Stochastic Processes

The problem

In the practice, these parameters are unknown.

Thus they must be estimated.

T + 1 + 2T +

T(T 1)

2

TOO MANY PARAMETERS TO ESTIMATE!!!

Umberto Triacca Stationary Stochastic Processes

The problem

If T=100, we have to estimate 1376 parameters!!!

This is impossible because we only have a single realization of

the process.

Umberto Triacca Stationary Stochastic Processes

The solution

Restrictions which enable us to reduce the number of unknown

parameters involved in order to be able to deduce their values

from a single realization:

1

restrictions on the time-heterogeneity of the process;

2

restrictions on the memory of the process.

Umberto Triacca Stationary Stochastic Processes

Restricting the time-heterogeneity

Now we introduce an important class of stochastic processes

which exhibit time-homogeneity.

This is the class of stationary stochastic processes.

Umberto Triacca Stationary Stochastic Processes

Stationary Stochastic Process

Denition

Consider a stochastic process {X

t

; t Z}. Let {t

1

, t

2

, , t

s

}

be a nite set of integers. The process {X

t

; t Z} is strongly

stationary if the joint distibution function of the vector

(X

t

1

+k

, X

t

2

+k

, ..., X

t

s

+k

) is equal with the one of

(X

t

1

, X

t

2

, ..., X

t

s

) for all integers s 1 and for all

t

1

, t

2

, , t

s

, k Z.

Umberto Triacca Stationary Stochastic Processes

The meaning of Stationary

The meaning of stationarity is that the distribution of a

number of random variables of the stochastic process is

the same as we shift them along the time index axis.

Umberto Triacca Stationary Stochastic Processes

The meaning of Stationary

If {X

t

; t Z}is a strongly stationary process, then, for

example,

X

1

, X

2

, X

3

, ...

have the same distribution function.

(X

1

, X

3

), (X

5

, X

7

), (X

9

, X

11

), .....

have the same joint distribution function and further

(X

1

, X

3

, X

5

), (X

7

, X

9

, X

11

), (X

13

, X

15

, X

17

), ...

must have the same joint distribution function, and so on

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Stationary Stochastic

Process

Consider the discrete stochastic process

{X

t

; t N}

where X

t

= A, with A U (3, 7)

This process is of course strongly stationary.

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

Consider the discrete stochastic process

{X

t

; t N}

where X

t

= tA, with A U (3, 7)

This process is not strongly stationary

Umberto Triacca Stationary Stochastic Processes

Weakly Stationary Stochastic Process

In many applications this is too strict a denition and hence

we dene now stationary for a wider class of processes.

Umberto Triacca Stationary Stochastic Processes

Weakly Stationary Stochastic Process

Denition

A stochastic process {X

t

; t Z} is said to be

covariance-stationary, or weakly stationary if

1

E(X

t

) = t

2

Var(X

t

) =

0

< t

3

Cov(X

t

, X

tk

) =

k

t, k

Umberto Triacca Stationary Stochastic Processes

Weakly Stationary processes

A stochastic process is covariance-stationary if

1

it has the same mean value, , at all time points;

2

it has the same variance,

0

, at all time points; and

3

the covariance between the values at any two time points,

t, t k, depend only on k, the dierence between the

two times, and not on the location of the points along the

time axis.

Umberto Triacca Stationary Stochastic Processes

Weakly Stationary processes

Denition

A stochastic process {u

t

; t Z} in which the random variables

u

t

, t = 0 1, 2... are such that

1

E(u

t

) = 0 t

2

Var(u

t

) =

2

u

< t

3

Cov(u

t

, u

tk

) = 0 t, k

is called white noise with mean 0 and variance

2

u

, written

u

t

WN(0,

2

u

)

.

If the random variables u

t

are independently and identically

distributed with mean 0 and variance

2

u

then we shal write

u

t

IID(0,

2

u

)

Umberto Triacca Stationary Stochastic Processes

Weakly Stationary processes

A white noise process is weakly stationary

Umberto Triacca Stationary Stochastic Processes

White noise process that is strongly stationary

A process u

t

IID(0,

2

u

) is a strongly stationary

This follows almost immediate from the denition.

1) Use the independence property to factor the joint

distribution into the product of the marginal distributions.

2) Then use the identical distribution property that

P(u

t

b) = P(u

s

b) for all t, s

Umberto Triacca Stationary Stochastic Processes

White noise process that is not strongly stationary

Let w be a random variable uniformly distributed in the

interval (0; 2). We consider the process {Z

t

; t = 1, 2, ...}

dened by

Z

t

= cos(tw) t = 1, 2, ...

We have that

1

E(Z

t

) = 0 t

2

Var(Z

t

) =

1

2

t

3

Cov(Z

t

, Z

tk

) = 0 t, k

Thus Z

t

WN(0, .5). However, it can be shown that is not

strongly stationary.

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

Consider the discrete stochastic process

{Y

t

; t Z}

where Y

t

= + t + u

t

, with u

t

WN(0,

2

u

)

This process is not weakly stationary

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

A possible realization of the stochastic process

Y

t

= 3 + 7t + u

t

, with u

t

WN(0,

2

u

)

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

Consider the discrete stochastic process

{y

t

; t = 0, 1, 2, ...}

where y

0

= < and y

t

= y

t1

+ u

t

for t = 1,2,..., with

u

t

WN(0,

2

u

)

This process is called random walk.

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

The mean and the variance of y

t

are be given by

E(y

t

) = and Var(y

t

) = t

2

u

Thus a random walk is not weakly stationary process.

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

A possible realization of a random walk

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Covariance-Stationary

Stochastic Process

Consider the discrete stochastic process

{y

t

; t Z}

where and y

t

= u

t

+ 3u

t1

, with u

t

WN(0,

2

u

)

This process is covariance-stationary.

Umberto Triacca Stationary Stochastic Processes

Example of a Discrete Non-Stationary Stochastic

Process

A possible realization of a this process is given by

Umberto Triacca Stationary Stochastic Processes

Relation between strong and weak Stationarity

If the process {X

t

; t Z} is strongly stationary and has nite

second moments, then {X

t

; t Z} is weakly stationary.

Proof: Consider a strongly stationary process, {X

t

; t Z},

with nite second moments. By denition of strongly

stationary process follows that X

t

has the same distribution for

each t Z. Since the process {X

t

; t Z} has nite second

moment, this implies that E(X

t

) and Var(X

t

) are both

constant. Further X

t

and X

tk

has the same joint distribution

and hence the same covariance for all k and for all t. Thus a

strongly stationary stochastic process with nite second

moments is weakly stationary.

Umberto Triacca Stationary Stochastic Processes

Relation between strong and weak Stationarity

Of course a weakly stationary process is not necessarily

strongly stationary.

There is one important case however in which weak

stationarity does imply strong stationarity.

Umberto Triacca Stationary Stochastic Processes

Relation between strong and weak Stationarity

If {X

t

; t Z} is a weakly stationary Gaussian stochastic

process, then {X

t

; t Z} is strongly stationary

Umberto Triacca Stationary Stochastic Processes

- 123444332-c-programs-class-11Uploaded byArmaan Chauhan
- [Chris Chatfield] the Analysis of Time SeriesUploaded byGabo ZS
- Information Theory Entropy Relative EntropyUploaded byJames
- Gakhov Time Series Forecasting With PythonUploaded byTariqul Islam Shuvo
- mathunit-2Uploaded byapi-205944031
- math lessonUploaded byjdcrisologo6749
- Lecture 4Uploaded byRajesh Bathija
- SSRN-id3274354Uploaded byAlejandro De Jesús Romo Rosales
- calendarUploaded byapi-301966268
- Lesson 2Uploaded byArun Choudhary
- Probability Random Process R08 NovDec 10Uploaded byRomeobenet
- The Origins of Truncated Levy FlightsUploaded bydiegoblrd
- User Satisfaction Web BrowsingUploaded byAnirbanab
- Ebachillerato Planilla Ing CompUploaded byZdr Onlyme
- DCMTUploaded byMari Sadr
- 2003RainDisag.pdfUploaded byVladimir Jerson Alferez Vargas
- marcpvUploaded bydurga03
- 6sA03Uploaded byMuhammad Bilal Junaid
- 25Uploaded byakita_1610
- Alternative Decomposition MethodsUploaded byleoazzola
- CAT Quant Myths & Preparation StrategiesUploaded bysakshij91
- 016_vgGray R. - Probability, Random Processes, And Ergodic PropertiesUploaded bydavid
- diffeqnUploaded byOctavio Martínez Baltodano
- 00847872.pdfUploaded byHiso Jokey
- Lamp IranUploaded byjessy
- Shorrocks Measuring Mobility Ect 1978Uploaded bySudhanshu Jaiswal
- 2010 IJC MA H2 P2 Prelim SolnUploaded byKelvin Dai Jibo
- 24 ProbabilityUploaded byJerickEmbanecidoMontilla
- Catatan Logit and Probit NotesUploaded byMuhammad Syah
- Emf 2015 Exercises Computer1 Including AssignmentUploaded byOzbridge

- AtikUploaded byShaikh Hassan Atik
- 36th-bcs-written-syllabus.pdfUploaded byShaikh Hassan Atik
- 01.Capacitor and Inductor.pdfUploaded byShaikh Hassan Atik
- TV.docxUploaded byShaikh Hassan Atik
- 02. RC.pdfUploaded byShaikh Hassan Atik
- radar.docxUploaded byShaikh Hassan Atik
- Voltage StabilizerUploaded byShaikh Hassan Atik
- Human Eye L2Uploaded byShaikh Hassan Atik
- Exp-2-DCUploaded byShaikh Hassan Atik
- USNews Electrical Ranking 2014Uploaded byShaikh Hassan Atik
- Ms Syllabus Apece DuUploaded byShaikh Hassan Atik
- Transcript of BSCUploaded byShaikh Hassan Atik
- Class RoutineUploaded byShaikh Hassan Atik

- Charging ArgumentUploaded bygeorge4012
- Lecture 08 Application of Linked ListsUploaded byZulfiqar Ali
- 2013 KCSE MARANDA MATHEMATICS P2 MS-1.pdfUploaded byKevin Okech
- CBOTUploaded bysangeet83
- textbook 2 6Uploaded byapi-293865618
- Ricci Tensor of Diagonal MetricUploaded byjspacemen01
- IEEE paper problem statementsUploaded byKeerthiPratheek Reddy
- notes5aUploaded byasukamonteiro
- Evaluation of Datalog Program PDFUploaded byHenry
- Matlab ExamUploaded byAliceAlormenu
- rsh_qam11_tif_05.docUploaded byJay Brock
- Machine Life- CalculusUploaded byShem Padilla
- Theory of Financial Decision Making IngersollUploaded byJorge Bejarano
- Notes on Partial Differential EquationsUploaded byJamie
- Important Instructions During PreparationUploaded byRahulsinghoooo
- Module 1 - Linear FunctionsUploaded bystepharry08
- 1st Periodical TestUploaded bymisseastsea
- Algorithm & Data Structures Lec1(BET)Uploaded byXafran Khan
- Ch5Uploaded byGREGORY LEONBERG
- 141_1_MidtermUploaded bySanjay Rai
- 600 - Pr 10 - The Korteweg-DeVries Field and Its Sustained DynamicsUploaded byBradley Nartowt
- Rational FunctionUploaded bycircleteam123
- Survival Models Solution Chapter 2Uploaded byNayaz Maudarbucus
- Digital Electronics Module 01Uploaded bylancenglot
- Solution of Boundary-Value Problems by Orthogonal CollocationUploaded byThirumalaimuthukumaranMohan
- GeometryUploaded byKent Kevin Nacor Nieves
- CMES_L01_L02_ModelCheking_a.pdfUploaded byAndreea Vescan
- Precalculus MATH 1730 Final Exam ReviewUploaded byvelisa815986
- Pseudo Code LibraryUploaded bymoshetalkar
- Define Utran INTRA Relation on RBDTM1EUploaded byRoy Adhiwibowo