Time averages and Ergodicity

Often we are interested in finding the various ensemble averages of a random process
{ X (t )} by means of the corresponding time averages determined from single realization
of the random process. For example we can compute the time-mean of a single
realization of the random process by the formula

1 T
mx
T �� 2T �
T
= lim x(t ) dt
-T

which is constant for the selected realization. m x T represents the dc value of x(t ).
Another important average used in electrical engineering is the rms value given by
1 T 2
T �� 2T �
xrms T = lim x (t )dt
-T

Can m x T and xrms T represent m X and EX 2 (t ) respectively ?

To answer such a question we have to understand various time averages and their
properties.

Time averages of a random process

The time-average of a function g ( X (t )) of a continuous random process { X (t )} is
defined by

1 T
2T �
g ( X (t )) T
= g ( X (t ))dt
-T

where the integral is defined in the mean-square sense.

Similarly, the time-average of a function g ( X n ) of a continuous random process { X n } is

defined by
1 N
g( X n ) = � g(X i )
N
2 N + 1 i =- N

The above definitions are in contrast to the corresponding ensemble average defined by
�
Eg ( X (t )) = �g ( x) f X ( t ) ( x) dx for continuous case
-�

= � g(x ) p
i�RX ( t )
i X (t ) ( xi ) for discrete case

The following time averages are of particular interest:

(a) Time-averaged mean

 1 T
mX T
= �X (t )dt (continuous case)
2T -T

1 N
mX = � Xi (discrete case)
N
2 N + 1 i =- N

(b) Time-averaged autocorrelation function

1 T
RX (t ) T
= �X (t ) X (t + t )dt (continuous case)
2T -T

1 N
RX [ m] = � X i X i +m (discrete case)
N 2 N + 1 i =- N

Note that, g ( X (t )) T and g ( X n ) N are functions of random variables and are governed
by respective probability distributions. However, determination of these distribution
functions is difficult and we shall discuss the behaviour of these averages in terms of their
mean and variances. We shall further assume that the random processes { X (t )} and
{ X n } are WSS.

Mean and Variance of the time averages

Let us consider the simplest case of the time averaged mean of a discrete-time WSS
random process { X n } given by
1 N
mX = � Xi
N
2 N + 1 i =- N
The mean of m X N
1 N
E mX =E � Xi
N
2 N + 1 i =- N
1 N
= � EX i
2 N + 1 i =- N
=m X

and the variance

2
� 1 �
( )
2 N
E mX - mX = E� � Xi - mX �
�2 N + 1 i =- N
N
�
2
� 1 N �
= E� � (Xi - mX )�
�2 N + 1 i =- N �
1 � E ( X - m )2 + 2 N
N N �
= 2 �� � � E ( X i - m X )( X j - m X ) �
( 2 N + 1) �i =- N
i X
i =- N ,i �j j =- N �

If the samples X - N , X - N +1 ,...., X 1 , X 2 ,...., X N are uncorrelated,

 2
� 1 �
( )
2 N
E mX - mX = E� � Xi - mX �
�2 N + 1 i =- N
N
�
1 � E ( X - m )2 �
N
= 2 ��
( 2 N + 1) i =- N
i X �

s X2
=
2N +1

( )
2
We also observe that lim E m X N
- mX =0
N ��

From the above result, we conclude that m X �mX

M .S .
N
���

Let us consider the time-averaged mean for the continuous case. We have
1 T
mX T =
2T �
X (t )dt
-T

1 T
\ E mX T =
2T �
EX (t ) dt
-T

1 T
m X dt = m X
2T �
=
-T

and the variance

2
�1 T �
E ( mX T - mX ) = E � �
2
-T X (t ) dt - m X �
�2T �
2
�1 T �
= E� � -T ( X (t ) - m X )dt �
�2T �
1 T T
= 2� -T �-T E ( X (t1 ) - m X )( X (t 2 ) - m X ) dt1dt 2
4T
1 T T
= 2� -T �-T C X (t1 - t2 )dt1dt2
4T
The above double integral is evaluated on the square area bounded by t1 = �T and
t2 = � T . We divide this square region into sum of trapezoidal strips parallel to
t1 - t2 = 0. Putting t1 - t2 = t and noting that the differential area between t1 - t2 = t and
t1 - t2 = t + dt is (2T - t )dt , the above double integral is converted to a single integral
as follows:

1 T T
E ( mX - mX ) =
2
�
-T �-T C X (t1 - t2 )dt1dt2
T
4T 2
1 2T
= 2� -2T (2T - t )C X (t )dt
4T
1 2T � t �
= �-2T �1- � C X (t )dt
2T � 2 T �
 t1

t1 - t2 = t + dt
t1 - t2 = 2T
T t1 - t2 = t

dt

-T T t2

-T t1 - t2 = -2T

Ergodicity Principle

If the time averages converge to the corresponding ensemble averages in the probabilistic
sense, then a time-average computed from a large realization can be used as the value for
the corresponding ensemble average. Such a principle is the ergodicity principle to be
discussed below:

Mean ergodic process

A WSS process { X (t )} is said to be ergodic in mean, if mX T
M .S .
��� � m X as T � �.
Thus for a mean ergodic process { X (t )} ,
lim E m X T = m X
T ��

and
lim var m X T
=0
T ��
We have earlier shown that
E mX T = mX
and
1
2T
� t �
var m X t dt
2T -�
T
= C X ( ) 1- �
�
2T � 2T �
Therefore, the condition for ergodicity in mean is

1
2T
� t �
lim �X �C (t ) 1- � dt = 0
T �� 2T
-2 T � 2T �

If C X (t ) decreases to 0 for t > t 0 , then the above condition is satisfied.

Further,
1
2T
� t � 1
2T

� C X (t ) 1-
� dt �
� �C X (t ) dt
2T -2T � 2T � 2T -2T

Therefore, a sufficient condition for mean ergodicity is

2T

�C X (t ) dt < �
-2 T
Example
Consider the random binary waveform { X (t )} discussed in Example .The
process has the auto-covariance function for t �Tp given by
� t
1-
� t �Tp
C X (t ) = � Tp
�
0
� otherwise

Here
2T 2T

�C X (t ) dt = 2 �C X (t ) dt
-2 T 0
Tp
� t �
= 2� 1-
� dt
� Tp ��
0 � �
� T3 T2 �
= 2�Tp + p 2 - p �
� 3Tp Tp �
� �
2T
= p
3
 �
\�C X (t ) dt < �
-�

Hence { X (t )} is not mean ergodic.

Autocorrelation ergodicity
T
1
RX (t ) T = X (t ) X (t + t )dt
2T -� T

If we consider Z (t ) = X (t ) X (t + t ) so that, m Z = RX (t )

Then { X (t )} will be autocorrelation ergodic if {Z (t )} is mean ergodic.

Thus { X (t )} will be autocorrelation ergodic if

1
2T
� t1 �
lim ��
1- C
� Z (t 1 )dt 1 = 0
T �� 2T
� 2T �
-2 T

where

CZ (t 1 ) = EZ (t ) Z (t - t 1 ) - EZ (t ) EZ (t - t 1 )
= EX (t ) X (t - t ) X (t - t ) X (t - t - t 1 ) - R X2 (t )

Involves fourth order moment.

Hence the condition for autocorrelation ergodicity of a jointly Gaussian process is found.

Thus X (t ) will be autocorrelation ergodic if

1
2T
� t �
lim ��
1- C (t )dt � 0
� z
T �� 2T
� 2T �
-2 T

Now CZ (t ) = EZ (t ) Z (t + t ) - RX (t )
2

Hence, X (t) will be autocorrelation ergodic

1
2T
� t �
If Tlim
�� 2T � 1-
� (
�
2T �
Ez (t ) z (t + a ) - RX2 (t ) ) da � 0
-2T �

Example
Consider the random–phased sinusoid given by
X (t ) = A cos( w0 t + f ) where A and w0 are constants and f ~ U [0, 2p ] is a random
variable. We have earlier proved that this process is WSS with m X = 0 and
A2
RX (t ) = cos w0t
2
For any particular realization x(t ) = A cos( w0t + f1 ),

1 T
mx A cos( w0t + f1 ) dt
2T �
T
=
-T

1
= A sin( w0T )
Tw0
and
T
1
Rx (t ) T
= �A cos( w t + f ) A cos( w (t + t ) + f )dt
0 1 0 1
2T -T
T
A2
= [cos w t + A cos( w (2t + t ) + 2f )]dt
� 0 0 1
4T -T

A2 cos w0t A2 sin( w0 (2T + t ))

= +
2 4 w0T
A2 cos w0t
We see that as T � �, both T
mx T
�� 0 and Rx (t )
2
For each realization, both the time-averaged mean and the time-averaged autocorrelation
function converge to the corresponding ensemble averages. Thus the random-phased
sinusoid is ergodic in both mean and autocorrelation.

Remark
A random process { X (t )} is ergodic if its ensemble averages converge in the M.S. sense
to the corresponding time averages. This is a stronger requirement than stationarity- the
ensemble averages of all orders of such a process are independent of time. This implies
that an ergodic process is necessarily stationary in the strict sense. The converse is not
true- there are stationary random processes which are not ergodic.

Following Fig. shows a hierarchical classification of random processes.

 Random processes

WSS random process

WSS Process

Ergodic Processes

Example
Suppose X (t ) = C where C ~ U [0 a ]. { X (t )} is a family of straight line as illustrated in
Fig. below.
 X (t ) = a
3
X (t ) = a
4
1
X (t ) = a
2

1
X ()t = a
4

X (t ) = 0
t
a
Here m X = and
2
1 T
m X T = lim �Cdt is a different constant for different realizations. Hence { X (t )} is
T �� 2T -T

not mean ergodic.