A brief note on the Karhunen-Love expansion

Alen Alexanderian

Abstract
We provide a detailed derivation of the Karhunen-Love expansion of a stochastic
process. We also discuss briey Gaussian processes, and provide a simple numerical
study for the purpose of illustration.
Contents
1 Introduction 1
2 Compact operators 2
2.1 Hilbert-Schmidt operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Linear operators on a Hilbert space . . . . . . . . . . . . . . . . . . . . . . 4
2.3 Spectral theorem for compact self-adjoint operators . . . . . . . . . . . . . 4
3 Mercers Theorem 5
4 Stochastic processes 5
4.1 Autocorrelation function of a stochastic process . . . . . . . . . . . . . . . 6
5 Karhunen-Love expansion 7
6 A classical example 11
6.1 Spectral decomposition of the autocorrelation function . . . . . . . . . . . 11
6.2 Simulating the random eld . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
References 12
1 Introduction
The purpose of this brief note is to provide a self-contained coverage of the idea
of the Karhunen-Love (KL) expansion of a stochastic process. Writing of this note was
motivated by being exposed to the many applications of the KL expansion in uncertainty
propagation through dynamical systems with random parameters (see e.g. in [3, 1]).
Since a clear and at the same time rigorous coverage the KL exapnsion is not so simple
to nd in the literature, here we provide a simple exposition of the theoretical basis
for the KL expansion, including a detailed proof of convergence. We will see that the
KL expansion is obtained through an interesting application of the Spectral Theorem
for compact normal operators, in conjunction with Mercers theorem which connects

The University of Texas at Austin, USA. E-mail: alen@ices.utexas.edu

Last revised: July 12, 2013
A brief note on the Karhunen-Love expansion
the spectral representation of a Hilbert-Schmidt integral operator to the corresponding
Hilbert-Schmidt kernel.
We begin by recalling some functional analytic basics on compact operators in Sec-
tion 2. Next, Mercers Theorem is recalled in Section 3. Then, we recall some basics
regarding stochastic processes in Section 4. In that section, a basic result stating the
equivalence of mean-square continuity of a stochastic process and the continuity of the
corresponding autocorrelation function is mentioned also. We cover the KL expansion
of a centered mean-square continuous stochastic process in Section 5 with a proof of
convergence. Finally, in Section 6, we provide a numerical example where the KL ex-
pansion of a Gaussian random eld is simulated.
2 Compact operators
Let us begin by recalling the notion of precompact and relatively compact sets.
Denition 2.1. (Relatively Compact)
Let X be a metric space; A X is relatively compact in X, if

A is compact in X.
Denition 2.2. (Precompact)
Let X be a metric space; A X is precompact (also called totally bounded) if for every
> 0, there exist nitely many points x
1
, . . . , x
N
in A such that
N
1
B(x
i
, ) covers A.
The following Theorem shows that when we are working in a complete metric space,
precompactness and relative compactness are equivalent.
Theorem 2.3. Let X be a metric space. If A X is relatively compact then it is
precompact. Moreover, if X is complete then the converse holds also.
Then, we dene a compact operator as below.
Denition 2.4. Let X and Y be two normed linear spaces and T : X Y a linear map
between X and Y . T is called a compact operator if for all bounded sets E X, T(E)
is relatively compact in Y .
By the above denition 2.4, if E X is a bounded set, then T(E) is compact in
Y . The following basic results shows a couple of different ways of looking at compact
operators.
Theorem 2.5. Let X and Y be two normed linear spaces; suppose T : X Y , is a
linear operator. Then the following are equivalent.
1. T is compact.
2. The image of the open unit ball under T is relatively compact in Y .
3. For any bounded sequence {x
n
} in X, there exist a subsequence {Tx
n
k
} of {Tx
n
}
that converges in Y .
Let us denote by B[X] the set of all bounded linear operators on a normed linear
space space X:
B[X] = {T : X X| T is a bounded linear transformation.}.
Note that equipped by the operator norm B[X] is a normed linear space. It is simple to
show that compact operators form a subspace of B[X]. The following result (c.f. [4] for
a proof) shows that the set of compact normal operators is in fact a closed subspace of
B[X].
Theorem 2.6. Let {T
n
} be a sequence of compact operators on a normed linear space
X. Suppose T
n
T in B[X]. Then, T is also a compact operator.
2
A brief note on the Karhunen-Love expansion
Another interesting fact regarding compact linear operators is that they form an
ideal of the ring of bounded linear mappings B[X]; this follows fromthe following result.
Lemma 2.7. Let X be a normed linear space, and let T and S be in B[X]. If T is
compact, then so are ST and TS.
Proof. Consider the mapping ST. Let {x
n
} be a bounded sequence in X. Then, by
Theorem 2.5(3), there exists a subsequence {Tx
n
k
} of {Tx
n
} that converges in X:
Tx
n
k
y

X.
Now, since S is continuous, it follows that STx
n
k
S(y

); that is, {STx

n
k
} converges
in X also, and so ST is compact. To show TS is compact, take a bounded sequence
{x
n
} in X and note that {Sx
n
} is bounded also (since S is continuous). Thus, again by
Theorem 2.5(3), there exists a subsequence {TSx
n
k
} which converges in X, and thus,
TS is also compact.
Remark 2.8. A compact linear operator of an innite dimensional normed linear space
is not invertible in B[X]. To see this, suppose that T has an inverse S in B[X]. Now,
applying the previous Lemma, we get that I = TS = ST is also compact. However,
this implies that the closed unit ball in X is compact, which is not possible since we
assumed X is innite dimensional
1
.
2.1 Hilbert-Schmidt operators
Let D R
n
be a bounded domain. We call a function k : D D R a Hilbert-
Schmidt kernel if
_
D
_
D
|k(x, y)|
2
dxdy < ,
that is, k L
2
(D D) (note that one special case is when k is a continuous function on
D D). Dene the integral operator K on L
2
(D), K : u Ku for u L
2
(D), by
[Ku](x) =
_
D
k(x, y)u(y) dy. (2.1)
Clearly, K is linear; moreover, it is simple to show that K : L
2
(D) L
2
(D):
Let u L
2
(D), then
_
D

(Ku)(x)

2
dx =
_
D

_
D
k(x, y)u(y) dy

2
dx

_
D
_
_
D
|k(x, y)|
2
dy
__
_
D
|u(y)|
2
dy
_
dx (Cauchy-Schwarz)
= ||k||
L
2
(DD)
||u||
L
2
(D)
< ,
so that Ku L
2
(D). The mapping K is what we call a Hilbert-Schmidt operator. The
following result which is usually proved using Theorem 2.6 is very useful.
Lemma 2.9. Let D be a bounded domain in R
n
and let k L
2
(D D) be a Hilbert-
Schmidt kernel. Then, the integral operator K : L
2
(D) L
2
(D) given by [Ku](x) =
_
D
k(x, y)u(y) dy is a compact operator.
1
Recall that the closed unit ball in a normed linear space X is compact if and only if X is nite dimensional.
3
A brief note on the Karhunen-Love expansion
Remark 2.10. One can think of Hilbert-Schmidt operators as generalizations of the
idea of matrices in innite-dimensional spaces. Note that if A is an n n matrix (a
linear mapping on R
n
), then, the action of Ax of X on a vector x R
n
is given by
[Ax]
i
=
n

j=1
A
ij
x
j
(2.2)
Now note that,
[Ku](x) =
_
D
k(x, y)u(y) dy
is an analog of (2.2) with the summation replaced with an integral.
2.2 Linear operators on a Hilbert space
Here we collect some basic denitions fromlinear operators theory in Hilbert spaces.
Let H be a real Hilbert space with inner product , : H H R. A linear operator
T : H H is called self adjoint if
Tx, y = x, Ty , x, y H.
Example 2.11. Let us consider a Hilbert-Schmidt operator K on L
2
([a, b]) as in (2.1)
(where for simplicity we have taken D = [a, b] R). Then, it is simple to show that K is
self-adjoint if and only if k(x, y) = k(y, x) on [a, b] [a, b].
A linear operator T : H H, is called positive if Tx, x 0 for all x in H. Recall
that a scalar R is called an eigenvalue of T if there exists a non-zero x H such
that Tx = x. Now, note that the eigenvalues of a positive operator are necessarily
non-negative which can be seen easily as follows; let be an eigenvalue of T with
eigenvector x H, then
0
Tx, x
x, x
=
x, x
x, x
= .
2.3 Spectral theorem for compact self-adjoint operators
Compact self-adjoint operators on innite dimensioal Hilbert spaces resemble many
properties of the symmetric matrices. Of particular interest is the spectral decomposi-
tion of a compact self-adjoint operator as given by the following:
Theorem 2.12. Let H be a (real or complex) Hilbert space and let T : H H be a
compact self-adjoint operator. Then, H has an orthonormal basis {e
i
} of eigenvectors
of T corresponding to eigenvalues
i
. In addition, the following holds:
1. The eigenvalues
i
are real having zero as the only possible point of accumulation.
2. The eigenspaces corresponding to distinct eigenvalues are mutually orthogonal.
3. The eigenspaces corresponding to non-zero eigenvalues are nite-dimensional.
In the case of a positive compact self-adjoint operator, we know that the eigenvalues
are non-negative. Hence, we may order the eigenvalues as follows

1

2
... 0.
Remark 2.13. Recall that for a linear operator A on a nite dimensional linear space,
we dene its spectrum (A) as the set of its eigenvalues. On the other hand, for a linear
4
A brief note on the Karhunen-Love expansion
operator T on an innite dimensional (real) normed linear space the spectrum (T) of
T is dened by,
(T) = { R : T I is not invertible in B[X]},
and (T) is the disjoint union of the point spectrum (set of eigenvalues), contiuous
spectrum, and residual spectrum (see [4] for details). As we saw in Remark 2.8, a
compact operator T on an innite dimensional space X cannot be invertible in B[X];
therefore, we always have 0 (T). However, not much can be said on whether = 0
is in point spectrum (i.e. an eigenvalue) or the other parts of the spectrum.
3 Mercers Theorem
Let D = [a, b] R. We have seen that given a continuous kernel k : D D R,
we can dene a Hilbert-Schmidt operator through (2.1) which is compact and has a
complete set of eigenvectors in L
2
(D). The following result by Mercer provides a series
representation for the kernel k based on spectral representation of the corresponding
Hilbert-Schmidt operator K. A proof of this result can be found for example in [2].
Theorem 3.1 (Mercer). Let k : DD R be a continuous function, where D = [a, b]
R. Suppose further that the corresponding Hilbert-Schmidt operator K : L
2
(D)
L
2
(D) given by (2.1) is postive. If {
i
} and {e
i
} are the eigenvalues and eigenvectors of
K, then for all s, t D,
k(s, t) =

i
e
i
(s)e
i
(t), (3.1)
where convergence is absolute and uniform on D D.
4 Stochastic processes
In what follows we consider a probability space (, F, P), where is a sample space,
F is an appropriate -algebra on and P is a probability measure. A real valued random
variable X on (, F, P) is an F/B(R)-measurable mapping X : (, F, P) (R, B(R)).
The expectation and variance of a random variable X is denoted by,
E[X] :=
_

X() dP(), Var [X] := E

_
(X E[X])
2

.
L
2
(, F, P) denotes the Hilbert space of (equivalence classes) of real valued square
integrable random variables on :
L
2
(, F, P) = {X : R :
_

|X()|
2
dP() < }.
with inner product, X, Y = E[XY ] =
_

XY dP and norm ||X|| = X, X

1/2
.
Let D R, a stochastic prcess is a mapping X : D R, such that X(t, )
is measurable for every t D; alternatively, we may dene a stochastic process as a
family of random variables, X
t
: R with t D, and refer to X as {X
t
}
xD
. Both of
these points of view of a stochastic process are useful and hence we will be switching
between them as appropriate.
A stochastic process is called centered if E[X
t
] = 0 for all t D. Let {Y
t
}
tD
be an
arbitrary stochastic process. We note that
Y
t
= E[Y
t
] +X
t
,
5
A brief note on the Karhunen-Love expansion
where X
t
= Y
t
E[Y
t
] and {X
t
}
tD
is a centered stochastic process. Therefore, without
loss of generality, we will focus our attention to centered stochastic processes.
We say a stochastic process is mean-square continuous if
lim
0
E
_
(X
t+
X
t
)
2

= 0.
The following denition is also useful.
Denition 4.1 (Realization of a stochastic process). Let X : D R be a stochastic
process. For a xed , we dene

X : D R by

X(t) = X
t
(). We call

X a
realization of the stochastic process.
For more details on theory of stochastic processes please consult [7, 5, 6].
4.1 Autocorrelation function of a stochastic process
The autocorrelation function of a stochastic process {X
t
}
tD
is given by R
X
: D
D R dened through
R
X
(s, t) = E[X
s
X
t
] , s, t D.
Lemma 4.2. A stochastic process {X
t
}
t[a,b]
is mean-square continuous if and only if
its autocorrelation function R
X
is continuous on [a, b] [a, b].
Proof. Suppose R
X
is continuous, and note that
E
_
(X
t+
X
t
)
2

= E
_
X
2
t+

2E[X
t+
X
t
]+E
_
X
2
t

= R
X
(t+, t+)2R
X
(t+, t)+R
X
(t, t).
Therefore, since R
X
is continuous,
lim
0
E
_
(X
t+
X
t
)
2

= lim
0
R
X
(t +, t +) 2R
X
(t +, t) +R
X
(t, t) = 0.
That is X
t
is mean-square continuous. Conversely, if X
t
is mean-square continous we
proceed as follows:
|R
X
(t +, s +) R
X
(t, s)| = |E[X
t+
X
s+
] E[X
t
X
s
] |
=

E[(X
t+
X
t
)(X
s+
X
s
)] + E[(X
t+
X
t
)X
s
] + E[(X
s+
X
s
)X
t
]

E[(X
t+
X
t
)(X
s+
X
s
)]

E[(X
t+
X
t
)X
s
]

E[(X
s+
X
s
)X
t
]

E
_
(X
t+
X
t
)
2

1/2
E
_
(X
s+
X
s
)
2

1/2
+ E[(X
t+
X
t
)]
1/2
E
_
X
2
s

1/2
+ E
_
(X
s+
X
s
)
2

1/2
E
_
X
2
t

1/2
,
where the last inequality follows from Cauchy-Schwarz inequality. Thus, we have,
|R
X
(t + , s + ) R
X
(t, s)| E

(X
t+
X
t
)
2

1/2
E

(X
s+
X
s
)
2

1/2
+ E [(X
t+
X
t
)]
1/2
E

X
2
s

1/2
+ E

(X
s+
X
s
)
2

1/2
E

X
2
t

1/2
,
and therefore, by mean-square continuity of X
t
we have that
lim
(,)(0,0)
|R
X
(t +, s +) R
X
(t, s)| = 0.
6
A brief note on the Karhunen-Love expansion
5 Karhunen-Love expansion
Let D R. In this section, we assume that X : D R is a centered mean-square
continuous stochastic process such that X L
2
(D ). With the technical tools from
the previous sections, we are now ready to derive the KL expansion of X.
Dene the integral K : L
2
(D) L
2
(D) by
[Ku](s) =
_
D
k(s, t)u(t) dt, k(s, t) = R
X
(s, t), (5.1)
The following lemma summarizes the properties of the operator K.
Lemma 5.1. Let K : L
2
(D) L
2
(D) be as in (5.1). Then the following hold:
1. K is compact.
2. K is positive
3. K is self-adjoint.
Proof. (1) Since the process X is mean-square continuous, Lemma 4.2 implies that
k(s, t) = R
X
(s, t) is continuous. Therefore, by Lemma 2.9, K is compact.
(2) We need to show Ku, u 0 for every u L
2
(D), where , denotes the L
2
(D)
inner product.
Ku, u =
_
D
Ku(s)u(s) ds =
_
D
_
_
D
k(s, t)u(t) dt
_
u(s) ds
=
_
D
_
_
D
E[X
s
X
t
] u(t) dt
_
u(s) ds
= E
__
D
_
D
X
s
X
t
u(t)u(s) dt ds
_
= E
_
_
_
D
X
s
u(s) ds
__
_
D
X
t
u(t) dt
_
_
= E
_
_
_
D
X
t
u(t) dt
_
2
_
0,
where we used Fubinis Theorem to interchange integrals.
(3) This follows trivially from R
X
(s, t) = R
X
(t, s) and Fubinis theorem:
Ku, v =
_
D
Ku(s)v(s) ds =
_
D
_
_
D
k(t, s)v(s) ds
_
u(t) dt = u, Kv .
Now, let K be dened as in (5.1) the previous lemma allows us to invoke the spectral
theorem for compact self-adjoint operators to conclude that K has a complete set of
eigenvectors {e
i
} in L
2
(D) and real eigenvalues {
i
}:
Ke
i
=
i
e
i
. (5.2)
Moreover, since K is positive, the eigenvalues
i
are non-negative (and have zero as
the only possible accumulation point). Now, the stochastic process X which we xed in
the beginning of this section is assumed to be square integrable on D and thus, we
may use the basis {e
i
} of L
2
(D) to expand X
t
as follows,
X
t
=

i
x
i
e
i
(t), x
i
=
_
D
X
t
e
i
(t) dt (5.3)
7
A brief note on the Karhunen-Love expansion
The above equality is to be understood in mean square sense. To be most specic, at this
point we have that the realizations

X of the stochastic process X admit the expansion

X =

i
x
i
e
i
where the convergence is in L
2
(D ). We will see shortly that the result is in fact
stronger, and we have
lim
N
E
_
_
X
t

N

i=1
x
i
e
i
(t)
_
2
_
= 0,
uniformly in D, and thus, as a consequence, we have that (5.3) holds for all t D.
Before proving this, we examine the coefcients x
i
in (5.3). Note that x
i
are random
variables on . The following lemma summarizes the properties of the coefcients x
i
.
Lemma 5.2. The coefcients x
i
in (5.3) satisfy the following:
1. E[x
i
] = 0
2. E[x
i
x
j
] =
ij

j
.
3. Var [x
i
] =
i
.
Proof. To see the rst assertion note that
E[x
i
] = E
__
D
X
t
e
i
(t) dt
_
=
_

_
D
X
t
()e
i
(t) dt dP()
=
_
D
_

X
t
()e
i
(t) dP() dt (Fubini)
=
_
D
E[X
t
] e
i
(t) dt = 0,
where the last conclusion follows from E[X
t
] = 0 (X is a centered process). To see the
second assertion, we proceed as follows
E[x
i
x
j
] = E
_
_
_
D
X
s
e
i
(s) ds
__
_
D
X
t
e
j
(t) dt
_
_
= E
__
D
_
D
X
s
e
i
(s)X
t
e
j
(t) ds dt
_
=
_
D
_
D
E[X
s
X
t
] e
i
(s)e
j
(t) ds dt
=
_
D
_
_
D
k(s, t)e
j
(t) dt
_
e
i
(s) ds
=
_
D
[Ke
j
](s)e
i
(s) ds from (5.1)
= Ke
j
, e
i

=
j
e
j
, e
i

=
j

ij
,
where again we have used Fubinis Theorem to interchange integrals and the last con-
clusion follows fromorthonormality of eigenvectors of K. The assertion (3) of the lemma
follows easily from (1) and (2):
Var [x
i
] = E
_
(x
i
E[x
i
])
2

= E
_
x
2
i

=
i
.
8
A brief note on the Karhunen-Love expansion
Now, we have the technical tools to prove the following:
Theorem 5.3 (Karhunen-Loeve). Let X : D R be a centered mean-square con-
tinuous stochastic process with X L
2
( D). There exist a basis {e
i
} of L
2
(D) such
that for all t D,
X
t
=

i=1
x
i
e
i
(t), in L
2
(),
where coefcients x
i
are given by x
i
() =
_
D
X
t
()e
i
(t) dt and satisfy the following.
1. E[x
i
] = 0
2. E[x
i
x
j
] =
ij

j
.
3. Var [x
j
] =
j
.
Proof. Let K be the Hilbert-Schmidt operator dened as in (5.1). We know that K has a
complete set of eigenvectors {e
i
} in L
2
(D) and non-negative eigenvalues {
i
}. Note that
x
i
() =
_
D
X
t
()e
i
(t) dt satisfy the the properties (1)-(3) by Lemma 5.2. Next, consider

n
(t) := E
_
_
X
t

n

i=1
x
i
e
i
(t)
_
2
_
.
The rest of the proof amounts to showing lim
n

n
(t) = 0 uniformly (and hence point-
wise) in D.

n
(t) = E
_
_
X
t

n

i=1
x
i
e
i
(t)
_
2
_
= E
_
X
2
t

2E
_
X
t
n

i=1
x
i
e
i
(t)
_
+ E
_
_
n

i,j=1
x
i
x
j
e
i
(t)e
j
(t)
_
_
(5.4)
Now, E
_
X
2
t

= k(t, t) with k as in (5.1),

E
_
X
t
n

i=1
x
i
e
i
(t)
_
= E
_
X
t
n

i=1
_
_
D
X
s
e
i
(s) ds
_
e
i
(t)
_
=
n

i=1
_
_
D
E[X
t
X
s
] e
i
(s) ds
_
e
i
(t)
=
n

i=1
_
_
D
k(t, s)e
i
(s) ds
_
e
i
(t)
=
n

i=1
[Ke
i
](t)e
i
(t)
=
n

i=1

i
e
i
(t)
2
(5.5)
Through a similar argument, we can show that
E
_
_
n

i,j=1
x
i
x
j
e
i
(t)e
j
(t)
_
_
=
n

i=1

i
e
i
(t)
2
(5.6)
Therefore, by (5.4), (5.5), and (5.6) we have

n
(t) = k(t, t)
n

i=1

i
e
i
(t)e
i
(t),
9
A brief note on the Karhunen-Love expansion
invoking Theorem 3.1 (Mercers Theorem) we have
lim
n

n
(t) = 0,
uniformly; this completes the proof.
Remark 5.4. Suppose
k
= 0 for some k, and consider the coefcient x
k
in the expan-
sion (5.3). Then, we have by the above Theorem E[x
k
] = 0 and Var [x
k
] =
k
= 0, and
therefore, x
k
= 0. That is, the coefcient x
k
corresponding to a zero eigenvalue is zero.
Therefore, only x
i
corresponding to postive eigenvalues
i
appear in KL expansion of a
square integrable, centered, and mean-square continous stochastic process.
In the view of the above remark, we can normalize the coefcients x
i
in a KL ex-
pansion and dene
i
=
1

i
x
i
. This leads to the following, more familiar, version of
Theorem 5.3.
Corollary 5.5. Let X : D R be a centered mean-square continuous stochastic
process with X L
2
( D). There exist a basis {e
i
} of L
2
(D) such that for all t D,
X(t, ) =

i=1
_

i
()e
i
(t) in L
2
().
where
i
are centered mutually uncorrelated random variables with unit variance and
are given by,

i
() =
1

i
_
D
X
t
()e
i
(t) dt.
The KL expansion of a Gaussian process has the further property that
i
are indepen-
dent standard normal random variables (see e.g. [3, 1]). The latter is a useful property
in practical applications, used extensively in the method of stochastic nite element [1].
10
A brief note on the Karhunen-Love expansion
6 A classical example
Here we consider the KL decomposition of a Gaussian random eld X, which is
characterized by its variance
2
and an autocorrelation function R
X
(s, t) given by,
R
X
(s, t) =
2
exp
_

|s t|
L
c
_
. (6.1)
We show in Figure 1 a plot of R
X
(s, t) over [0, 1] [0, 1].
Figure 1: The autocorrelation function.
6.1 Spectral decomposition of the autocorrelation function
For this particular example, the eigenfunctions e
i
(t) and eigenvalues
i
(t) can be
computed analytically. The analytic expression for eigenvalues and eigenvectors can be
found for example in [1, 3]. We consider the case of
2
= 1 and L
c
= 1 in (6.1). In
Figure 2, we show the rst few eigenfunctions and eigenvalues of the autocorrelation
function dened in (6.1).
0 0.2 0.4 0.6 0.8 1
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
0 2 4 6 8 10
10
3
10
2
10
1
10
0
(a) (b)
Figure 2: The rst few eigenfunctions (a) and eigenvalues (b) of the autocorrelation
function.
To get an idea of how fast the approximation,
R
N
X
(s, t) =
N

i=1

i
e
i
(s)e
i
(t)
converges to R
X
(s, t) we show in Figure 3 the plots of R
N
X
(s, t) for N = 2, 4, 6, 8. In
Figure 4, we see that with N = 6, absolute error is bounded by 8 10
2
.
11
A brief note on the Karhunen-Love expansion
Figure 3: Improvements of the approximations to R
X
(s, t) as the expansion order is
increased.
(a) (b) (c)
Figure 4: (a) The autocorrelation function R
X
(s, t), (b) the approximation R
N
X
(s, t) with
N = 6, and (c) pointwise difference between R
X
(s, t) and R
N
X
(s, t) with N = 6.
6.2 Simulating the random eld
Having the eigenvalues and eigenfunctions of R
X
(t, ) at hand, we can simulate the
random eld X(t, ) with a truncated KL expansion,
X(t, )
.
=
N

i=1
_

i
()e
i
(t).
As discussed before, in this case,
i
are independent standard normal variables. In
Figure 5(a), we plot a few realizations of X(t, ), t [0, 1] and in Figure 5(b), we show
the distribution of X(t, ) at t = 1/2 versus standard normal distribution.
References
[1] Roger G. Ghanem and Pol D. Spanos. Stochastic nite elements: a spectral approach.
Springer-Verlag New York, Inc., New York, NY, USA, 1991.
12
A brief note on the Karhunen-Love expansion
0 0.2 0.4 0.6 0.8 1
3
2
1
0
1
2
3
4
6 4 2 0 2 4 6
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
Figure 5: (a) A few realizations of the random eld X(t, ), and (b) distribution of X(t, )
at t = 1/2 approximated by simulating the truncated KL expansion (blue) versus a
standard normal distribution (red).
[2] Israel Gohberg, Seymour Goldberg, and M. A. Kaashoek. Basic classes of linear operators.
2004.
[3] O. P. Le Maitre and Omar M. Knio. Spectral Methods for Uncertainty Quantication With
Applications to Computational Fluid Dynamics. Scientic Computation. Springer, 2010.
[4] Arch W. Naylor and George Sell. Linear operator theory in engineering and science. Springer-
Verlag, New York, 1982.
[5] L. C. G. Rogers and David Williams. Diffusions, Markov processes, and martingales. Vol. 1.
Cambridge Mathematical Library. Cambridge University Press, Cambridge, 2000. Founda-
tions, Reprint of the second (1994) edition.
[6] L. C. G. Rogers and David Williams. Diffusions, Markov processes, and martingales. Vol. 2.
Cambridge Mathematical Library. Cambridge University Press, Cambridge, 2000. It calcu-
lus, Reprint of the second (1994) edition.
[7] David Williams. Probability with martingales. Cambridge Mathematical Textbooks. Cam-
bridge University Press, Cambridge, 1991.
13