Notesc

Stochastic Processes in Continuous Time
Joseph C. Watkins
December 14, 2007
Contents
1 Basic Concepts 3
1.1 Notions of equivalence of stochastic processes . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Sample path properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Properties of ltrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 Stopping times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5 Examples of stopping times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 Levy Processes 12
3 Martingales 16
3.1 Regularity of Sample Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2 Sample Path Regularity and Levy Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.3 Maximal Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.4 Localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.5 Law of the Iterated Logarithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4 Markov Processes 33
4.1 Denitions and Transition Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.2 Operator Semigroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.2.1 The Generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.2.2 The Resolvent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.3 The Hille-Yosida Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.3.1 Dissipative Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.3.2 Yosida Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.3.3 Positive Operators and Feller Semigroups . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.3.4 The Maximum Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.4 Strong Markov Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.5 Connections to Martingales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.6 Jump Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.6.1 The Structure Theorem for Pure Jump Markov Processes . . . . . . . . . . . . . . . . 53
4.6.2 Construction in the Case of Bounded Jump Rates . . . . . . . . . . . . . . . . . . . . 55
4.6.3 Birth and Death Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.6.4 Examples of Interacting Particle Systems . . . . . . . . . . . . . . . . . . . . . . . . . 59
1
CONTENTS 2
4.7 Sample Path Regularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.7.1 Compactifying the State Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.8 Transformations of Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.8.1 Random Time Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.8.2 Killing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.8.3 Change of Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.9 Stationary Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.10 One Dimensional Diusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.10.1 The Scale Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.10.2 The Speed Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.10.3 The Characteristic Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.10.4 Boundary Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5 Stochastic Integrals 84
5.1 Quadratic Variation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.2 Denition of the Stochastic Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.3 The Ito Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.4 Stochastic Dierential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.5 Ito Diusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
1 BASIC CONCEPTS 3
1 Basic Concepts
We now consider stochastic processes with index set = [0, ). Thus, the process
X : [0, ) S
can be considered as a random function of time via its sample paths or realizations
t X
t
(), for each .
Here S is a metric space with metric d.
1.1 Notions of equivalence of stochastic processes
As before, for m 1, 0 t
1
< < t
m
, B B(S
m
), the Borel -algebra, call
t1,...,tm
(B) = P(X
t1
, . . . , X
tm
) B
the nite dimensional distributions of X.
We also have a variety of notions of equivalence for two stochastic processes, X and Y .
Denition 1.1. 1. Y is a version of X if X and Y have the same nite dimensional distributions.
2. Y is a modication of X if for all t 0, (X
t
, Y
t
) is an S S-random variable and
PY
t
= X
t
= 1.
3. Y is indistinguishable from X if
P; Y
t
() ,= X
t
() for some t 0 = 0.
Recall that the Daniell-Kolmogorov extension theorem guarantees that the nite dimensional distribu-
tions uniquely determine a probability measure on S
[0,)
.
Exercise 1.2. If A X
t
; t 0, then there exists a countable set C [0, ) so that A X
t
; t C.
Thus, many sets of interest, e.g., x; sup
0sT
f(x
s
) > 0 or x :
_
T
0
f(x
s
) ds < are not measurable
subsets of S
[0,)
. Thus, we will need to nd methods to nd versions of a stochastic process so that these
sets are measurable.
Exercise 1.3. If X and Y are indistinguishable, then X is a modication of Y . If X is a modication of
Y , then X and Y have the same nite dimensional distributions.
1 BASIC CONCEPTS 4
1.2 Sample path properties
Denition 1.4. We call X
1. measurable if X is B[0, ) T-measurable,
2. (almost surely) continuous, (left continuous, right continuous) if for (almost) all the sample path
is continuous, (left continuous, right continuous).
Focusing on the regularity of sample paths, we have
Lemma 1.5. Let x : [0, ) S and suppose
x
t+
= lim
st+
x
s
exists for all t 0,
and
x
t
= lim
st
x
s
exists for all t > 0.
Then there exists a countable set C such that for all t [0, )C,
x
t
= x
t
= x
t+
.
Proof. Set
C
n
= t [0, ); maxd(x
t
, x
t
), d(x
t
, x
t+
), d(x
t
, x
t+
) >
1
n
If C
n
[0, m] is innite, then by the Bolzano-Weierstrass theorem, it must have a limit point t [0, m]. In
this case, either x
t
or x
t+
would fail to exist. Now, write
C =
_
m=1
_
n=1
(C
n
[0, m]),
the countable union of nite sets, and, hence, countable.
Lemma 1.6. Let D be a dense subset of [0, ) and let x : D S. If for each t 0,
x
+
t
= lim
st+,sD
x
s
exists, then x
+
is right continuous. If for each t > 0,
x
t
= lim
st,sD
x
s
exists, then x
is left continuous.
Proof. Fix t
0
> 0. Given > 0, there exists > 0 so that
d(x
+
t0
, x
s
) whenever s D (t
0
, t
0
+).
Consequently, for all s (t
0
, t
0
+)
d(x
+
t0
, x
+
s
) = lim
us+,uD
d(x
+
t0
, x
u
) .
and x
+
is right continuous.
The left continuity is proved similarly.
1 BASIC CONCEPTS 5
Exercise 1.7. If both limits exist in the previous lemma for all t, then x
t
= x
+
t
for all t > 0.
Denition 1.8. Let C
S
[0, ) denote the space of continuous S-valued functions on [0, ). Let D
S
[0, )
denote the space of right continuous S-valued functions having left limits on [0, ).
Even though we will not need to use the metric and topological issue associated with the spaces C
S
[0, )
and D
S
[0, ), we proved a brief overview of the issues.
If we endow the space C
S
[0, ) with the supremum metric
(x, y) = sup
0s
d(x
s
, y
s
), then the
metric will not in general be separable. In analogy with the use of seminorms in a Frechet to dene a metric,
we set, for each t > 0,
t
(x, y) = sup
s
d(x
maxs,t]
, y
maxs,t]
).
Then,
T
satises the symmetric and triangle inequality properties of a metric. However, if x and y
agree on [0, T], but x
s
,= y
s
for some s > T, then x ,= y but
t
(x, y) = 0. However, if
t
(x, y) = 0 for all t,
then x = y. Consider a bounded metric

d(x
0
, y
0
) = maxd(x
0
, y
0
), 1 and set
t
(x, y) = sup
0tt

d(x
s
, y
s
),
then we can create a metric on C
S
[0, ) which is separable by giving increasingly small importance to large
values of t. For example, we can use
(x, y) =
_

0
e
t

t
(x, y) dt.
Then, (C
S
[0, ), ) is separable whenever (S, d) is separable and complete whenever (S, d) is complete.
For the space D
S
[0, ), then unless the jumps match up exactly then the distance from x to y might be
large. To match up the jumps, we introduce a continuous strictly increasing function : [0, ) [, ) that
is one to one and onto and dene
t
(x, y) = sup
s
d(x
maxs,t]
, y
max(s),t]
).
and
(
x, y) = inf
maxess sup
t
[ log
t
t
[,
_

0
e
t
t
(x, y) dt.
As before, (D
S
[0, ), ) is separable whenever (S, d) is separable and complete whenever (S, d) is complete.
Exercise 1.9. C
R
[0, ) with the
metric is not separable. Hint: Find an uncountable collection of

real-valued continuous functions so that the distance between each of them is 1.
With a stochastic process X with sample paths in D
S
[0, ), we have the following moment condition
that guarantee that X has a C
S
[0, ) version.
Proposition 1.10. If (S, d) be a separable metric space and set d
1
(x, y) = mind(x, y), 1. Let X be a
process with sample paths in D
S
[0, ). Suppose for each T > 0, there exist > 1, > 0, and C > 0 such
that
E[d
1
(X
t
, X
s
)
] C(t s)
0 s t T. (1.1)
Then almost all sample paths of X belong to C
S
[0, ).
1 BASIC CONCEPTS 6
Proof. Let T be a positive integer.
Claim.
0<tT
d
1
(X
t
, X
t
)
liminf
N
2
N
T
k=1
d
1
(X
k2
N, X
(k1)2
N)
.
First note that by Lemma 1.5, the left side is the sum of a countable number of terms. In addition, note
that for each n 1, t [0, T]; d
1
(X
t
, X
t
) > 1/n is a nite set. Thus, for N suciently large, these points
are isolated by the 2
N
partition of [0, T]. Thus, in the limit, these jumps are captured. Consequently,
0<tT
d
1
(X
t
, X
t
)
I
d1(Xt,Xt)>1/n]
liminf
N
2
N
T
k=1
d
1
(X
k2
N, X
(k1)2
N)
. (1.2)
Note that the left side of expression (1.2) increases as n increases. Thus, let n to establish the
claim.
By Fatous lemma, and the moment inequality (1.1),
E[
0<tT
d
1
(X
t
, X
t
)
] liminf
n
2
n
T
k=1
E[d
1
(X
k2
n, X
(k1)2
n)
]
liminf
n
2
n
T
k=1
C2
n
= liminf
n
CT2
n(1)
= 0.
Thus, for each T, with probability 1, X has no discontinuities on the interval [0, T] and consequently
almost all sample paths of X belong to C
S
[0, ).
1.3 Properties of ltrations
Denition 1.11. A collection of sub--algebras T
t
; t 0 of the -algebra T is called a ltration if s t
implies that T
s
T
t
.
For a given stochastic process X, write T
X
t
for the ltration X
s
; 0 s t. Call T
X
t
; t 0 is called
the natural ltration associated to the process X.
The ltration T
t
; t 0 in continuous time may have some additional structure
Denition 1.12. 1. T
t
; t 0 is right continuous if for each t 0,
T
t
= T
t+
=
>0
T
t+
.
2. T
t
; t 0 is complete if (, T, P) is complete and T
0
contains all events having probability zero.
Combining the process X and the ltration T
t
; t 0, we have, as before:
1 BASIC CONCEPTS 7
Denition 1.13. X is adapted if X
t
is T
t
-measurable. In addition, a process X is T
t
-progressive if, for
every t 0,
X[
[0,t]
is B[0, t] T
t
-measurable.
Exercise 1.14. 1. The ltration T
t+
; t 0 is right continuous.
2. If X is T
t
-progressive, then X is T
t
-adapted and measurable. Hint. Approximate X on [0, t]
by X
n
s
() = X
mint,([ns]+1)/n)]
().
3. If X is T
t
-adapted and right continuous, then is T
t
-progressive.
4. Let X be T
t
-progressive and let f : S R be bounded and measurable. Show that f X is T
t
-
progressive and A
t
=
_
t
0
f(X
s
) ds is T
t
-adapted.
1.4 Stopping times
A stopping time, , and the corresponding stopped -algebra T
are dened for continuous time processes

in a manner analogous to the discrete time case.
Denition 1.15. A non-negative (possibly innite) random variable is an T
t
-stopping time if for every
t 0,
t T
t
.
The stopped -algebra
T
= A T; A t T
t
, for all t 0.
Many of the same properties hold with the same proof as for discrete time processes.
Proposition 1.16. 1. The supremum of a countable number of stopping times is a stopping time.
2. The minimum of a nite number of stopping times is a stopping time.
3. T
is a -algebra
4. For a second stopping time , min, is T
-measurable.
5. If , then T
The following proposition holds in discrete time. However the proof in continuous is more involved.
Proposition 1.17. Let X be an T
t
-progressive process and a nite stopping time. Then X
is T
measurable.
Proof. Pick B(S). We must show that X
B T
. Consequently, for each t 0, we must show

that.
X
B t T
t
.
We shall accomplish this using a compositions of maps. Note that by the previous proposition, min, t is
T
t
-measurable. Thus
(min(), t, )
is a measurable mapping from (, T
t
) to ([0, t] , B([0, t]) T
t
).
1 BASIC CONCEPTS 8
Next, because X is an T
t
-progressive process,
(s, ) X
s
()
is a measurable mapping from ([0, t] , B([0, t]) T
t
) to (S, B(S)).
Thus, the composition
X
min(),t]
()
is a measurable mapping from (, T
t
) to (S, B(S)) and X
min,t]
T
t
.
Finally, X
B t = X
min,t]
B t. Because both of these events are elements of
the -algebra T
t
, then so is their intersection.
Some additional properties are connected to the continuity of time.
Lemma 1.18. A [0, ]-valued random variable is an T
t+
-stopping time if and only if t < T
t
for
every t 0.
Proof. (necessity)
< t =
_
n=1
t
1
n
T
t
T
t+
.
(suciency) For n m,
< t +
1
n
T
t+1/m
.
Therefore,
t =
n=1
< t +
1
n

m=1
T
t+1/m
T
t+
.
Corollary 1.19. For an T
t+
-stopping time ,
= t T
t
.
Proof. = t = t < t T
t
.
Proposition 1.20. Let
n
; n 1. If T
t
is right continuous, then inf
n
n
, liminf
n
n
, and
limsup
n
n
are T
t
-stopping times.
Proof. Use the lemma above and note that
inf
n
n
< t =
n=1
n
< t.
Now use
liminf
n
n
= sup
m
inf
nm
n
and limsup
n
n
= inf
m
sup
nm
n
.
1 BASIC CONCEPTS 9
Proposition 1.21. Every T
t
-stopping time is the nonincreasing limit of bounded stopping times.
Proof. Take the stopping times
n
= min, n for n = 1, 2, . . . .
Denition 1.22. Call a stopping time discrete if its range is almost surely countable.
Proposition 1.23. Every T
t
-stopping time is the nonincreasing limit of discrete stopping times.
Proof. For n = 1, 2, . . ., choose the nested collection of sequences 0 = t
n
0
< t
n
1
< with lim
k
t
n
k
= ,
and lim
n
sup
k
(t
n
k+1
t
n
k
) = 0. For an T
t+
-stopping time , dene
n
=
_
t
n
k
if t
n
k1
< t
n
k
if = .
Then clearly
n
is discrete and = lim
n
n
. To see that
n
is a stopping time, set
n
(t) = maxt
n
k
; t
n
k

t. Then
n
t =
n

n
(t) =
n
(t) T
n(t)
T
t
.
Exercise 1.24. Let
n
; n 1 be a sequence of T
t
-stopping times and let = lim
n
n
.
1. If
n
; n 1 is nondecreasing, then is an T
t
-stopping time.
2. If
n
; n 1 is nonincreasing, then is an T
t+
-stopping time.
Denition 1.25. For a stochastic process X and a nite stopping time Dene the stopped process
X
t
= X
min,t]
and the shifted process
X
t
= X
+t
.
Proposition 1.26. Let X be an T
t
-progressive process and let be a nite stopping time. Dene the
collection of -algebras
(
t
= T
min,t]
and H
t
= T
+t
.
Then
1. (
t
; t 0 is a ltration and X
is both (
t
-progressive and T
t
-progressive.
2. H
t
; t 0 is a ltration and
X is H
t
-progressive.
Proof. For the rst statement, because min, t; t 0 is an increasing set of stopping times, (
t
; t 0
is a ltration. Because (
t
T
t
, then if X
is (
t
-progressive, then it is automatically T
t
-progressive. With
this in mind, choose B(S) and t 0. We show that
(s, ) [0, t] ; X
min(),s]
() B[0, t] (
t
.
Now, dene
(
s,t
= C ([0, t] min, t s); C B[0, t] T
s
.
1 BASIC CONCEPTS 10
Claim: (
s,t
B([0, t]) (
t
.
Clearly, (
s,t
is a -algebra. Thus, it suces to choose C of the form B A where B B([0, t]) and
A T
s
noting that the collection of such B A generates B[0, t] T
s
.
A min, t s (
t
. Thus, for B B[0, t],
(B A) ([0, t] min, t s) = B (A min, t s) B[0, t] (
t
.
Thus, B A (
s,t
.
Now, write
(s, ) [0, t] ; X
min(),s]
()
= (s, ) [0, t] ; X
min(),s]
() , min(), t s t (s, ); X
s
() , s < min, t.
To see that the rst term is in B[0, t] (
t
, write it as
(s, ); min(), t s t ([0, t] X
min,s]
() )
and note that
(s, ); min(), t s t =
n=1
_
k=1
([
k
n
, t]
k
n
min, t <
k + 1
n
).
Similarly, the second term equals
_
n=1
_
k=1
(s, ); X
s
() , s <
k
n
(s, );
k
n
min(), t
and use the claim.
For the second statement, rst note that by the same argument as before, H
t
; t 0 is a ltration. By
the rst part, the mapping
(s, ) X
min()+t,s]
from ([0, ] , B[0, ] T
+t
) to (S, B(S)) is measurable. We also have the measurable mapping
(u, ) (() +u, )
from ([0, t] , B[0, t] T
+t
) to ([0, ] , B[0, ] T
+t
). The mapping
(u, ) X
+u
()
is the composition of these two mappings, so it is measurable. Consequently,
X is H
t
-progressive.
1.5 Examples of stopping times
Denition 1.27. Let x : [0, ) S and let s 0.
1. The rst entrance time into A after time s is dened by
e
(A, s) = inft s; x
t
A.
1 BASIC CONCEPTS 11
2. The rst contact time into A after time s is dened by
c
(A, s) = inft s; clx
u
; s u t A ,= .
Here cl(B) is the closure of the set B.
The rst exit from B after time s is the rst entrance to B
c
. If s = 0, then it is suppressed in the
notation.
Proposition 1.28. Suppose that X is a right continuous T
t
-adapted process and that is an T
t
-stopping
time.
1. If A is closed and X has left limits at each t > 0 or if A is compact then
c
(A, ) is an T
t
-stopping
time.
2. If A is open then
e
(A, ) is an T
t+
-stopping time.
Proof. If A is open, we have by the right continuuity of X,
e
(A, ) < t =
_
sQ[0,t]
X
s
A < s T
t
proving 2.
Under the conditions of part 1,
c
(A, ) = lim
n
e
(A
1/n
, )
where A
= x; d(x, A) < . Thus
c
(A, ) t = ( s X
t
A
n=1
e
(A
1/n
, ) < t T
t
proving 1.
2 L
EVY PROCESSES 12
2 Levy Processes
We begin our study of continuous time stochastic process with the continuous time analog of random walks.
Denition 2.1. An R
d
-valued process X is call a Levy process or a process with stationary independent
increments if
1. (stationary increments) The distribution of X
t+s
X
t
does not depend on t.
2. (independent increments) Let 0 = t
0
< t
1
< < t
n
. Then the random variables
X
tj
X
tj1
; j = 1, . . . , n
are independent.
3. (stochastic continuity)
X
t
X
0

P
0 as t 0.
Exercise 2.2. 1. Any linear combination of independent Levy processes is a Levy process.
2. If X has nite mean, X
0
= 0 and = EX
1
, then EX
t
= t.
3. If, in addition, X has nite variance and
2
= Var(X
1
), then Var(X
t
) =
2
t.
Denition 2.3. A Poisson process N with parameter is a Levy process with N
0
= 0 and
PN
t
= n =
(t)
n
n!
e
t
.
Exercise 2.4. Show that the denition of a Poisson process satises the compatibility condition in the
Daniell-Kolmogorov extension theorem.
Denition 2.5. Let Y
k
, k 1 be independent and identically distributed R
d
-valued random variables and
let N be a Poisson process, then
X
t
=
Nt
k=1
Y
k
is called a compound Poisson process.
Exercise 2.6. Show that a compound Poisson process is a Levy process.
Exercise 2.7. Let Y
k
take on the values 1 each with probability 1/2 and let X be the associated compound
Poisson process. Assume that N has parameter . Then
PX
t
= m = e
t
I
m
(t)
where I
m
is the modied Bessel function of order m. Hint. The modied Bessel functions of integer order
I
m
(x) have generating function exp(x(z +z
1
)/2).
2 L
EVY PROCESSES 13
If Y
k
is not integer valued, then we use the characteristic function
s
for X
s
s
(u) = Ee
iu,Xs)
= E[E[e
iu,Xs)
[N
s
]]
=
n=0
E[e
iu,(Y1++Yn))
]PN
t
= n
=
n=0
E[e
iu,Y1)
]
n
(t)
n
n!
e
t
= exp(t(
Y
(u) 1)) = exp(t
_
(e
iu,y)
1) (dy))
where and
Y
are, respectively, the distribution and the characteristic function for the Y
k
.
Exercise 2.8. For a Levy process X let
s
and
s
(u) denote, respectively, the distribution and the charac-
teristic function for the X
s
. Then,
s+t
=
s
t
and consequently
s+t
=
s

t
.
Use the dominated convergence theorem to show that the stochastic continuity of a Levy process implies
that
s
is right continuous and so, by the functional relationship in the exercise above,
s
(u) = exp s(u),
for some .
Denition 2.9. The function is called the characteristic exponent for the Levy process.
Denition 2.10. A one dimensional Brownian motion B with parameters and
2
is a Levy process with
B
0
= 0 and
PB
t
A =
1
2t
_
A
exp
_
(x t)
2
2
2
t
_
dx.
If = 0 and
2
= 1, the B is called a standard Brownian motion.
For d-dimensional Brownian motion B, we require R
d
and a symmetric, positive semi-denite matrix
, Then, B is a Levy process and if B
0
= 0, then
PB
t
A =
1
_
det()(2t)
n
_
A
exp
_
(x t)
T
1
(x t)
2t
_
dx.
Exercise 2.11. 1. Show that Brownian motion as dened above satises the consistency condition in the
extension theorem.
2. Find the characteristic exponent for Brownian motion.
Exercise 2.12. Let B be a d-dimensional Brownian motion. Show that there exists an invertible matrix C ,
a vector , and a d-dimensional process

B
t
= (

B
1
t
, . . . ,

B
d
t
) in which the component are independent standard
Brownian motions so that B
t
= C

B
t
+t.
As a consequence, we sometimes call B a Brownian motion if B
t
= C

B
t
for some d d-matrix C.
Exercise 2.13. Let B be a standard Brownian motion. Then the following are also standard Brownian
motions.
2 L
EVY PROCESSES 14
1. B,
2. B
t+t0
B
t0
; t 0 for t
0
> 0.
3. aB
t/a
2; t 0 for a > 0.
Proposition 2.14. Let B be a standard Brownian motion. Then,
Psup
t
B
t
= +, inf
t
B
t
= = 1.
Proof. Set B
= supB
t
; t Q [0, ). Then, for any a Q
+
, B
and aB
have the same distribution.

Consequently, B
is concentrated on the set 0, . Set p
= PB
= 0. Then
p
PB
1
0, B
u
0, u Q [1, )
PB
1
0, supB
1+t
B
1
; t Q
+
<
= PB
1
0PsupB
1+t
B
1
; t Q
+
<
= PB
1
0PsupB
t
; t Q
+
< =
1
2
p
Thus, p
= 0 and Psup
t
B
t
= + = 1. Because B is a standard Brownian motion, Pinf
t
B
t
= =
1. Because the intersection of a pair of probability 1 events has probability 1, the proposition follows.
Take X
0
= 0. Then, we can write
X
t
= (X
t
X
(n1)t/n
) + (X
(n1)t/n
X
(n1)t/n
) + + (X
t/n
X
0
).
Thus, for any n, X
t
can be written as the sum of n independent identically distributed random variables.
This motivates the following:
Denition 2.15. A random variable Y is said to have an innitely divisible distribution if for every n, there
exists n independent identically distributed random variables, Y
1,n
, . . . , Y
n,n
such that
Y = Y
1,n
+ +Y
n,n
.
The Levy-Khintchine formula gives the characteristic function for any innitely divisible distribution on
R
d
,
exp (u) = exp
_
i, u)
1
2
u, u) +
_ _
e
iu,y)
1
iu, y)
1 +[y[
2
_
(dy)
_
.
For this formula, there is some exibility in the last function (y) = y/(1 +[y[
2
). We need to be anti-
symmetric, bounded and have derivative 1 at the origin. We can choose to be any d-dimensional vector
and to be any symmetric non-negative denite d d-matrix. The measure , called the Levy measure can
be any Radon measure satisfying
1. 0 = 0, and
2.
_
]y]
2
1+]y]
2
(dy) < .
2 L
EVY PROCESSES 15
Now, clearly exp((u)/n) is the characteristic function of a random variable. The sum of n such inde-
pendent random varibles has characteristic function exp (u). Thus, we have the converse that any innitely
divisible distribution can be realized by X
1
for some Levy process X.
If
_
]y]
1+]y]
2
(dy) < , then we can write
exp (u) = exp(i
, u)
1
2
u, u) +
_
(e
iu,y)
1) (dy)).
where
= +
_
y
1 +y
2
(dy)
This is the characteric function of a random variable that is the independent sum of a normal random
variable and a compound Poisson process evaluated at time 1.
If = (R
d
0) < , and if CC
T
= , then the process
X
t
= t +C

B
t
+
Nt
n=1
Y
n
t
the X is a Levy process with characteristic exponent where

B
t
= (

B
1
t
, . . . ,

B
d
t
) with components that are
independent standard Brownian motions, N is a Poisson process with parameter and Y
n
; n 1 is an
independent sequence of random variables with common distrobution /.
3 MARTINGALES 16
F
3 Martingales
The denition of martingale in continuous time is exactly the analog of the discete time denition.
Denition 3.1. A real valued process X with E[X
t
[ < for all t 0 and adapted to a ltration T
t
; t 0
is an T
t
-martingale if
E[X
t
[T
s
] = X
s
for all t > s,
an T
t
-submartingale if
E[X
t
[T
s
] X
s
for all t > s,
and an T
t
-;gale if the inequality above is reversed.
Remark 3.2. Many previous facts about discrete martingales continue to hold for continuous time martin-
gales.
1. X is an T
t
martingale if and only if for every s < t, E[X
t
; A] = E[X
s
; A], for all A T
s
.
2. Suppose X is an T
t
-martingale, is convex and E[(X
t
)[ < for all t 1. Then X is an
T
t
-submartingale.
3. Suppose X is an T
t
-submartingale, is convex and non-decreasing, E[(X
t
)[ < for all t 0. Then
X is an T
t
-submartingale.
Exercise 3.3. An adapted stochastic process M is a martingale if and only if
EM
= EM
0
for every bounded stopping time .
Example 3.4. Let X be a Levy process, X
0
= 0.
1. If X has nite mean, = EX
1
. Then X
t
t is a martingale.
2. If X has mean zero and nite variance,
2
= Var(X
1
). Then X
2
t

2
t is a martingale.
3. Let X have characteristic exponent . Then exp(iuX
t
t(u)) is a complex-valued martingale.
Exercise 3.5. Compute the martingales above for the Poisson process and one-dimensional Brownian mo-
tion.
The optional sampling theorem continues to hold. In this context, let
t
; t 0 be an increasing
collection of stopping times and set
Y
t
= X
t
, and (
t
= T
t
.
Assume that each
t
satsies the sampling integrability conditions for X. If X is an T
t
-submartingale, then
Y is a (
t
-submartingale.
We have similar statements in the discrete time case with 0 =
0

1
and Y
n
= X
n
. One
important special case of this is the choice of
n
= t
n
where t
n
; n 1 is a set of non-random times.
3 MARTINGALES 17
Proposition 3.6. Let and be two T
t
-stopping times taking values in a nite set F = t
1
< < t
m
.
If X is an T
t
-submartingale, then
E[X
[T
] X
min,]
.
Proof. We show that for every A T
,
E[X
; A] E[X
min,]
; A].
Write A =
m
i=1
(A = t
i
). to see that it is sucient to show that for each i,
E[X
; A = t
i
] E[X
min,]
; A = t
i
] E[X
min,ti]
; A = t
i
].
Because A = t
i
T
ti
, the inequality above holds provided that
E[X
[T
ti
] X
min,ti]
.
Claim. E[X
min,t
k+1
]
[T
t
k
] X
min,t
k
]
.
E[X
min,t
k+1
]
[T
t
k
] = E[X
min,t
k+1
]
I
>t
k
]
[T
t
k
] +E[X
min,t
k+1
]
I
t
k
]
[T
t
k
]
= E[X
t
k+1
[T
t
k
]I
>t
k
]
+X
min,t
k
]
I
t
k
]
X
t
k
I
>t
k
]
+X
min,t
k
]
I
t
k
]
= X
min,t
k
]
Thus, E[X
[T
ti
] = E[X
min,tm]
[T
ti
] X
min,tm1]
. Use this for the frist step and the tower property
of -algebras in the induction step to complete the proof.
Example 3.7. Let N be a Poisson process with parameter . Let
n
= inft 0; N
t
= n.
1. Fix a time t, then min
n
, tis bounded, so by the optional sampling theorem,
0 = EN
minn,t]
E min
n
, t.
Now, use the monotone convergence theorem on each of these terms and the fact that
n
< almost
surely to obtain
0 = EN
n
E
n
and, therefore, E
n
=
n
2. Consider
exp(iuN
t
t(e
iu
1)),
the exponential martingale for the Poisson process. Then
1 = E[exp(iuN
n

n
(e
iu
1)],
and thus
e
iun
= E[exp(
n
(e
iu
1)].
Set
v = (e
iu
1) or u = i log(
v +
)
3 MARTINGALES 18
yielding
Ee
v
= (

+v
)
n
This is the n-th power of the Laplace transform of an exponential random variable and, hence, the
Laplace transform of a (n, ) random variable.
Exercise 3.8. Using N and
n
as in the example above. Show that
n+1

n
is an exponential random
variable independent of T
N
n
t see that
n+1

n
; n 1 is an independent and identically distributed
sequence of exponential random variables.
Exercise 3.9. Note that N
t
< n =
n
> t. Use this to obtain the density of a (n, ) random variable.
3.1 Regularity of Sample Paths
Lemma 3.10. Let X be a submartingale, t > 0 and F a nite subset of [0, t]. Then for each x > 0,
Pmax
sF
X
s
x
1
x
EX
+
t
,
and
Pmin
sF
X
s
x
1
x
(EX
+
t
EX
0
).
Proof. The rst statement follows immediately from the corresponding result for discrete time submartin-
gales. For the second statement, dene
= minu F; X
u
x.
By the proposition above, E[X
t
[T
] X
min,t]
. In addition, if < , then min , t = .
EX
0
EX
min,t]
= E[X
min,t]
; < ] +E[X
min,t]
; = ]
E[X
; < ] +E[X
t
; = ] xP < +EX
+
t
.
Now simplify.
Corollary 3.11. Let X be a submartingale, t > 0 and C a countable subset of [0, t]. Then for each x > 0,
Pmax
sC
X
s
x
1
x
EX
+
t
,
and
Pmin
sC
X
s
x
1
x
(EX
+
t
EX
0
).
Proof. Choose nite subests F
1
F
2
so that C =
n=1
F
n
. Then, for 0 < x < x,
Pmax
sC
X
s
x lim
n
Pmax
sFn
X
s
x
1
x
EX
+
t
.
Now, let x x.
3 MARTINGALES 19
Corollary 3.12. Let D be a countable dense subset of [0, ). Then
P sup
sD[0,t]
X
s
< = P inf
sD[0,t]
X
s
= 1.
We can now make similar analogies to the discrete time case for upcrossings. For a stochastic process X,
dene U(a, b, F) to be the number of upcrossings of the interval (a, b) by X restricted to a nite set F. For
C countable, as before, write C =
n=1
F
n
and dene Then U(a, b, F
n
) is a monotone increasing sequence.
Call its limit U(a, b, C).
Exercise 3.13. Show that the denition above is independent of the choice of the nite sets F
n
.
Theorem 3.14 (Doobs upcrossing inequality). Let X be a submartingale and let t > 0. For a countable
subset C [0, t],
EU(a, b, C)
E(X
t
a)
+
b a
.
Proof. Choose F
n
as above, then by the discrete time version of the uncrossing inequality
EU(a, b, F
n
)
E(X
t
a)
+
b a
.
Now, use the monotone convergence theorem.
Corollary 3.15. Let D be a countable dense subset of [0, ). Then
PU(a, b, D [0, t]) < = 1.
Remark 3.16. Set
0
=
n=1
_
_
sup
sD[0,n]
X
t
< inf
sD[0,n]
X
t

a<b,a,bD
U(a, b, D [0, n]) <
_
_
.
Then, P(
0
) = 1.
Exercise 3.17. For
0
,
X
+
t
() = lim
st+,sD
X
s
()
exists for all t 0.
X
t
() = lim
st,sD
X
s
()
exists for all t > 0. Furthermore, X
+
() D
S
[0, ) and X
+
t
() = X
t
().
Set X
+
t
() = 0 for all /
0
.
Proposition 3.18. Let X be a submartingale and dene X
+
as above. Then = t 0; PX
+
t
,= X
+
t
>
0 is countable. PX
t
= X
+
t
= 1 for t / and
X
t
() =
_
X
t
() if t
X
+
t
() if t /
denes a modication of X almost all of whose sample paths have right and left limits at all t 0 and are
right continuous at all t / .
3 MARTINGALES 20
Proof. Consider the space L
() of real-valued random variables on and dene the metric

(
1
,
2
) = E[min[
1

2
[, 1].
We can consider the stochastic process X as a map
[0, ) (L
(), )
Now X
+
has right and left hand limits in this metric and so by the lemma is countable.
Choose a real number a, then
a
(x) = maxx, a is an increasing convex function and so
a
(X
t
) is a
submartingale. Consequently,
a
a
(X
s
) E[
a
(X
t
)[T
X
s
], 0 s t.
Because E[
a
(X
t
)[T
X
s
]; 0 s t is uniformly integrable, and
[
a
(X
s
)[ [a[ +[E[
a
(X
t
)[T
X
s
][
we see that
a
(X
s
); 0 s t is also uniformly integrable. Therefore, almost sure convergence implies
convergence in L
1
and
a
a
(X
s
) lim
us+,uD
E[
a
(X
u
)[T
X
s
] = E[
a
(X
+
s
)[T
X
s
].
For s / , X
+
s
= X
+
s
a.s and
E[E[
a
(X
+
s
)[T
X
s
]
a
(X
s
)] lim
us,uD
E[
a
(X
+
s
)
a
(X
u
)] = 0.
Thus, the non-negative random variable E[
a
(X
+
s
)[T
X
s
]
a
(X
s
) has zero expectation and is consequently
0 almost surely.
Using again that X
+
s
= X
+
s
a.s. and that X
+
s
is T
X
s
-measurable,
a
(X
s
) = E[
a
(X
+
s
)[T
X
s
] =
a
(X
+
s
) almost surely.
Thus, for almost all , if X
s
() > a, then X
s
() = X
+
s
(). Because this holds for all a R,
PX
s
= X
+
s
= 1 for all s /
i.e., X is right continuous for s / and

X is a modication of X
To see that

X has both left and right limits for all t 0, replace D with D in the construction of
0
and call this new set

0
. For
, and t 0,
X
+
t
() = lim
st+
X
+
s
() = lim
st+,sD
X
s
().
Thus,
X
+
t
() = lim
st+
X
s
()
and

X has right limits for every t.
Repeat the procedure with X
to show the existence of left limits.

3 MARTINGALES 21
Remark 3.19. In most of the situations that we will encounter, = and if the process is dened, as in
the case of Levy processes, by its nite dimensional distributions, then we can take the version with sample
paths in D
S
[0, ).
Exercise 3.20. The convergence under the metric is equivalent to convergence in probability.
Corollary 3.21. Let Z L
1
, then for any ltration T
t
; t 0, and for all t 0,
E[Z[T
s
]
L
1
E[Z[T
t+
] as s t +.
Proof. Let X
t
= E[Z[T
t+
]. Then X is a martingale. By the proposition above, X has right limits a.s. at
each t 0. X is uniformly integrable. Consequenty,
X
s
X
t+
a.s. and in L
1
as s t +.
Note that X
t+
is T
t+
-measurable. Take A T
t+
, then for s > t, T
t+
T
s
and E[X
s
; A] = E[Z; A].
Consequently,
E[X
t+
; A] = lim
st+
E[X
s
; A] = E[Z; A].
Thus, X
t+
= E[Z[T
t+
].
Theorem 3.22 (Doobs regularity theorem). If X is a T
t
-submartingale, then the process X
+
is also a
T
t
-submartingale. Moreover, X
+
is a modication of X if and only if the map
t X
+
t
is right continuous from [0, ) to L
1
(), that is for every t 0,
lim
st+
E[[X
s
X
t
[] = 0.
Proof.
Theorem 3.23 (Optional sampling formula). Let X be a right continuous T
t
-submartingale and let and
be T
t
-stopping tmes. Then for each t > 0,
E[X
min,t]
[T
] X
min,,t]
.
If, in addition, satises the sampling integrability conditions for X, then
E[X
[T
] X
min,]
.
Proof. We have the theorem in the case that and are discrete stopping times. Now, let
n
,
n
; n 1
be a sequence of nonincreasing discrete stopping times that, converge, respectively to and .
As before for a, set
a
(x) = maxx, a and note that
a
(X
t
) is a submartingale. Then,
E[
a
(X
minn,t]
)[T
n
]
a
(X
minn,n,t]
.
Use the fact that T
T
n
and the tower property to conclude that
E[
a
(X
minn,t]
)[T
] E[
a
(X
minn,n,t]
[T
].
3 MARTINGALES 22
Note that
a
a
(X
minn,t]
) E[
a
(X
t
)[T
n
] and a
a
(X
minn,n,t]
) E[
a
(X
t
)[T
minn,n]
].
Consequently, as before,
a
(X
minn,t]
); n 1 and
a
(X
minn,n,t]
) are uniformly integrable. Now
let n and use the right continuity of X to obtain
E[
a
(X
min,t]
)[T
] E[
a
(X
min,,t]
[T
]. =
a
(X
min,,t]
)
Now let, a . The second part follows are in the discrete time case.
We have similar theorems to the discrete time case on convergence as t .
Theorem 3.24. (Submartingale convergence theorem) Let X be a right continuous submartingale. If
sup
t
EX
+
t
< ,
then
lim
t
X
t
= X
exists almost surely with E[X
[ < .
Theorem 3.25. Let X be a right continuous T
t
-submartingale. Then X is uniformly integrable if and only
if there exists a random variable X
such that
X
t

L
1
X
.
Furthermore, when this holds
X
t
X
a.s.
Corollary 3.26. A nonnegative supermartingale converges almost surely.
Proposition 3.27. Let X be a right continuous non-negative T
t
-;gale. Let
c
(0) be the rst contact time
with 0. Then, with probability 1, X
t
= 0 for all t
c
(0).
Proof. For n = 1, 2, . . ., let
n
=
e
([0, n
1
)), the rst entrance time into [0, n
1
). Then
n
is an T
t+
-stopping
time and
c
(0) = lim
n
n
. If
n
< , X
n
n
1
. Consequently, for every t 0,
E[X
t
[T
n+
] X
mint,n]
and hence
E[X
t
[T
n+
]I
nt]
,
1
n
,
and, taking n , we have upon taking expected values that
E[X
t
I
c(0)t]
] 0.
Now, use the non-negativity and right continuity of X.
3 MARTINGALES 23
3.2 Sample Path Regularity and Levy Processes
From Doobs regularilty theorem, we have that any Levy process with nite mean has a version in D
R
d[0, ).
For Brownian motion, we have the following.
Proposition 3.28. Brownian motion has a version in C
R
d[0, ).
Proof. B
t
= C

B
t
+t where C is a d d matrix, is a d-dimensional vector and

B is a vector consisting of
d independent standard Brownian motions. Thus, if standard Brownian motion has a continuous version, so
does B Thus, let

B be standard Brownian motion on R. Then

B
t
is a normal random variable, mean zero,
variance t. Thus, E

B
4
t
= 3t
2
or
E[[
B
t

B
s
]
4
] = 3(t s)
2
.
Because B has a version in D
R
d[0, ), we can apply the moment theorem on continuous versions with C = 3,
= 4 and = 2.
Example 3.29. Let B be a continuous version of standard Brownian motion. Recall that for this process
Psup
t
B
t
= , inf
t
B
t
= = 1. In particular, because B is continuous, for each a R, the stopping
time
a
= inft 0; B
t
= a is nite with probability 1.
1. Choose a, b > 0 and dene = inft 0; B
t
/ (a, b) = min
a
,
b
. Because each point in R is
recurrent a.s., is almost surely nite. Thus, by the optional sampling theorem,
0 = EB
min,n]
= bPB
= b, n aPB
= a, n +E[B
n
; > n].
Now, let n . For the third term, use the bounded convergence theorem to obtain
0 = bPB
= b aPB
= a
or
E = PB
= b =
a
b +a
.
Now, use the martingale B
2
t
t to obtain
0 = EB
2
min,n]
E min, n.
For the rst term use the bounded convergence theorem and for the second use the monotone convergence
theorem as n to see that
EB
2
= b
2
a
b +a
+a
2
b
b +a
= ab.
For the second term use the monotone convergence theorem as n to see that
E = ab.
2. Set X
t
= B
t
+t. For x > 0, dene the stopping times
x
= inft 0; X
t
= x
Now
exp(uX
t
t) = exp(uB
t
( u)t)
3 MARTINGALES 24
is a martingale provided that
u =
1
2
u
2
,
that is,
u
=
_
2
+ 2.
Note that if > 0, u
< 0 < u
+
. Thus the martingale exp(u
+
X
t
t) is bounded on [0,
x
]. The
optional sampling theorem applies and
1 = E[exp(u
+
X
x

x
)] = e
u+x
E[e
x
].
Consequently,
E[e
x
] = exp(x(
_
2
+ 2 )).
Take 0, then exp(
n
) I
n<]
. Therefore,
P
x
< =
_
1 0
e
2x
< 0
In addition, the Laplace transform can be inverted to see that
x
has density
f
x
(t) =
x
2t
3
exp(
(x t)
2
2t
).
Exercise 3.30. Let B be standard Brownian motion, a > 0, and = inft 0; [B
t
[ = a.
1. E[exp()] = sech(a
2).
2. E
2
= 5a
4
/3. Hint: Prove that B
4
t
6tB
2
t
+ 3t
2
is a martingale.
Proposition 3.31. Let X be a right continuous Levy process. Then for each s 0, the process

X
t
=
X
s+t
X
s
is a Levy process with the same nite dimensional distribution as X that is independent of T
X
s+
.
Proof. For each > 0, T
X
s+
T
X
s+
, X
s+t+
X
s+
is a Levy process independent of T
X
s+
. Let A T
X
s+
Choose a right continuous modication of X, times 0 = t
0
< t
1
< < t
n
, and bounded continuous functions
f
j
.
E[
n
j=1
f
j
(X
s+tj
X
s+tj1
); A] = lim
0+
E[
n
j=1
f(X
s+tj+
X
s+tj1+
); A]
= lim
0+
E[
n
j=1
f(X
s+tj+
X
s+tj1+
)]P(A)
= lim
0+
E[
n
j=1
f(X
tj
X
tj1
)]P(A)
= E[
n
j=1
f(X
tj
X
tj1
)]P(A)
3 MARTINGALES 25
Proposition 3.32. Let X be a right continuous Levy process and an almost surely nite stopping time.
Then the process

X
s
= X
s+
X
is a Levy process with respect to the ltration T

X
+s
; s 0 having the
same nite dimensional distributions as X
s
X
0
and is independent of T
X
+
.
Proof. Consider times 0 = t
0
< t
1
< < t
n
, bounded continuous functions f
1
, f
2
, . . . , f
n
, and A T
X
+
.
For the case that is a discrete stopping time with range s
k
; k 1, we have
E[
n
j=1
f
j
(X
+tj
X
+tj1
); A] =
k=1
E[
n
j=1
f
j
(X
+tj
X
+tj1
); A = s
k
]
=
k=1
E[
n
j=1
f
j
(X
s
k
+tj
X
s
k
+tj1
); A = s
k
]
=
k=1
E[
n
j=1
f
j
(X
tj
X
tj1
)]P(A = s
k
)
= E[
n
j=1
f
j
(X
tj
X
tj1
)]P(A)
Note that A = s
k
T
s
k
+
and thus the third line follows from the second by the previous proposition.
For the general case, let
m
; m 1 be a decreasing sequence of stopping times with limit . Pick
A T
T
m
. Now, apply the identity above to the
m
, use right continuity and the bounded convergence
theorem to obtain the result.
Corollary 3.33 (Blumenthal 0-1 law). For a Levy process X, with X
0
= 0 every event in T
X
0+
has probability
0 or 1.
Proof. Fix A T
X
0+
, then, in addition, A X
t
; t 0 = X
t
X
0
; t 0. Then T
X
0+
and X
t
X
0
; t
0 are independent -algebras and thus P(A) = P(A A) = P(A)P(A) and so P(A) = 0 or 1.
In particular, if is an T
X
t+
-stopping time then
P = 0 = 0 or 1.
Example 3.34. 1. Suppose that f(t) > 0 for all t > 0. Then
limsup
t0+
B
t
f(t)
= c a.s.
for some c [0, ]. If f(t) =

t, the B
t
/f(t) is a standard normal and so the limit above cannot be
nite. Consequently, c = . If f(t) = t
, > 1/2, then B

t
/f(t) is normal, mean 0, variance t
12
and so the limit above cannot be positive. Consequently, c = 0.
2. Let = inft > 0; B
t
> 0. Then P t PB
t
> 0 = 1/2. Thus,
P = 0 = lim
t0+
P t , = 0,
thus it must be 1.
3 MARTINGALES 26
Theorem 3.35 (Reection principle). Let a R and let B be a continuous standard Brownian motion.
Dene the stopping time
a
= inft > 0; B
t
> a. The process
B
t
=
_
B
t
if t <
a
,
2a B
t
if t
a
.
is a continuous standard Brownian motion.
Proof.
a
< with probability 1. Therefore, X
t
= B
a+t
a and X are standard Brownian motions
independent of B
a
. Consequently (B
a
, X) and (B
a
, X) have the same distribution. Dene the continuous
map
: C
R
[0, ) C
R
[0, ) C
R
[0, )
by
(b, x)
t
= b
t
I
at]
+ (x
a+t
a)I
a>t]
.
Then (B
a
, X) = B and (B
a
, X) =

B.
Exercise 3.36. State and prove a similar theorem for symmetric Levy processes and general nite stopping
times.
Corollary 3.37. Dene
B
t
= supB
s
; s t.
Then, for all a, x 0 and t 0,
PB
t
a, B
t
a x = PB
t
a +x.
Proof.
PB
t
a, B
t
a x = PB
t
a,

B
t
a x = PB
t
a, B
t
a +x = PB
t
a +x.
Use the corollary in the case a > 0 and x = 0 to obtain.
P
a
t = PB
t
a = PB
t
a, B
t
a +PB
t
a, B
t
a
= 2PB
t
a =
1
2t
_

a
e
x
2
/2t
dx
Exercise 3.38. 1. Show that
a
has density
f
n
(t) =
a
2t
3
e
a
2
/2t
.
2. Show that (B
t
,
a
) has joint density
f
(B
t
,a)
(x, t) =
2(2a x)
2t
3
exp(
(2a x)
2
2t
), a 0, x a.
3. Show that
a
; a 0 is a Levy process that does not have a nite stopping time.
3 MARTINGALES 27
3.3 Maximal Inequalities
Proposition 3.39 (Doobs submartingale maximal inequality). Let X be a right continuous submartingale.
1. Then for each x > 0 and t > 0,
P sup
0st
X
s
x
1
x
EX
+
t
,
and
P inf
0st
X
s
x
1
x
(EX
+
t
EX
0
).
2. If X is non-negative. Then for > 1 an d t > 0,
E[ sup
0st
X
s
]
_

1
_
EX
t
.
Proof. The rst statement follows form the corresponding result for a countable collection of times in [0, t]
and right continuity.
For the second statement. set = inft 0; X
s
> x. Then is an T
t+
-stopping time. The right
continuity of X implies that X
x whenever < . Consequently,

sup
0st
X
t
> x t sup
0st
X
t
x.
Moreover, these events have equal probability for all but countably many x > 0 and hence
xP t E[X
; t] E[X
t
; t].
By the optional sampling theorem,
EX
min,t]
EX
t
.
Now the balance of the proof follows the same line as in the discrete time case.
Proposition 3.40. Let B be a continuous version of standard Brownian motion and dene
B
t
=
_
tB
1/t
if t > 0
0 if t = 0
Then

B is also a continuous version of standard Brownian motion.
Proof. An easily calculation shows that

B has stationary independent increments, that

B
t+s

B
t
is normally
distributed with mean 0 and variance s and that

B is continouus on (0, ).
To establish continuity at 0, rst note that is equivalent to showing that
lim
t
B
t
t
= 0.
This subsequential limit along the integers is 0 by the strong law of large numbers. The following claim
proves the theorem.
3 MARTINGALES 28
Claim.
lim
n
max
ntn+1
[B
t
B
n
[
n
= 0 a.s.
Let > 0. Then
P max
ntn+1
[B
t
B
n
[ > n = P max
0t1
[B
t
[ > n
1
(n)
2
EB
2
1
.
Thus,
n=0
P max
ntn+1
[B
t
B
n
[ > n =
n=0
1
(n)
2
=

2
6
< .
Thus, by the rst Borel-Cantelli lemma,
P max
ntn+1
[B
t
B
n
[
n
> i.o. = 0
and the claim holds.
We will need the following integration result for the next theorem.
Exercise 3.41. Let be the cumulative distribution function for the standard normal. Then, for a > 0
(a) = 1 (a)
1
2
exp(
1
2
a
2
)a
1
(1 a
2
).
Hint. Use exp(z
2
/2) > (1 3z
4
) exp(z
2
/2) and nd the antiderivative.
3.4 Localization
To extend our results to a broader class of processes, we introduce the concept of localization.
Denition 3.42. Let Y be a T
t
-adapted D
R
[0, )-process,
1. A stopping time reduces Y if
Y
I
>0]
is uniformly integrable.
2. Y is called a local martingale if there exists a increasing sequence of stopping times
n
; n 1,
lim
n
n
= so that
n
reduces Y and Y
n
is a martingale. We shall call the sequence above a
reducing sequence for Y .
Exercise 3.43. 1. If reduces Y and , then reduces Y .
2. Local martingales form a vector space of processes.
3. Take
n
= n to see that martingales are local martingales.
4. Let
n
= inft 0 : [Y
t
[ n, then min
n
,
n
is a reducing sequence for Y .
3 MARTINGALES 29
This next proposition gives us our rst glimpse into the stochastic calculus.
Proposition 3.44. Suppose that Y is a right continuous T
t
-local martingale, and that V is a real-valued
T
t
-adapted, bounded continuous process having locally bounded variation. Then
M
t
= V
t
Y
t

_
t
0
Y
s
dV
s
is an T
t
-local martingale.
Remark 3.45. This uses the fact that we will later learn that integrals of martingales are martingales.
Then, M can be found from an integration by parts formula
_
t
0
V
s
dV
s
= V
t
Y
t
V
0
Y
0

_
t
0
Y
u
dV
u
.
Exercise 3.46 (summation by parts). Let a
k
; k 0 and b
k
; k 0 be two real valued sequences, then
n1
k=m
a
k
(b
k+1
b
k
) = a
n
b
n
a
m
b
m

n1
k=m
b
k+1
(a
k+1
a
k
).
Proof. (of the proposition) Let
n
; n 1 be a localizing sequence for Y so that
n
is less than the contact
time with the set (n, n)
c
.
Write [V [
t
for the variation of V on the interval [0, t]. Because V is continuous,
n
, the rst contact time
of [V [ into [n, ) is a stopping time. Because [V [ is continuous,
n
as n .
Write
n
= min
n
,
n
, we show that M
n
is a martingale, i. e,
E
_
V
n
t+s
Y
n
t+s
Y
n
s
V
n
s

_
mint+s,n]
mint,n]
Y
n
u
dV
n
u
[T
t
_
= 0. (3.1)
Let t = u
0
< u
1
< < u
m
= t +s be a partition of the interval [t, t +s]. Then, by the tower property,
and the fact that V is adapted and thatY
n
is a martingale, we see that
E[
m1
k=0
V
n
u
k
(Y
n
u
k+1
Y
n
u
k
)[T
t
] =
m1
k=0
E[V
n
u
k
E[(Y
n
u
k+1
Y
n
u
k
)[T
u
k
[T
t
] = 0.
Use the summation by parts formula to obtain that
E[V
n
t+s
Y
n
t+s
Y
n
s
V
n
s

m1
k=0
Y
n
u
k+1
(V
n
u
k+1
V
n
u
k
)[T
t
] = 0. (3.2)
The sum in the conditional expectation converges almost surely to the Riemann-Stieltjes integral
_
t+s
t
Y
n
u
dV
n
u
.
3 MARTINGALES 30
To complete the proof, we must show that the process is uniformly integrable. However,
n1
k=0
Y
n
u
k
(V
n
u
k+1
V
n
u
k
)
m1
k=0
[Y
n
u
k
[[V
n
u
k+1
V
n
u
k
[ maxn, Y
n
t+s
([V
n
[
t+s
[V
n
[
t
) maxn, Y
n
t+s
n
which is integrable.
Thus, the Riemann sums converge in L
1
and we have both that (3.1) holds and that M
n
is uniformly
integrable.
3.5 Law of the Iterated Logarithm
Theorem 3.47. (law of the iterated logarithm for Brownian motion) Let B be a standard Brownian motion.
Then,
P
_
limsup
t0+
B
t
_
2t log log(1/t)
= 1
_
= 1.
Proof. (McKean) Write h(t) =
_
2t log log(1/t).
Part 1.
P
_
limsup
t0+
B
t
h(t)
1
_
= 1.
Consider the exponential martingale Z
t
() = exp(B
t
1
2
2
t). Then by Doobs submartingale maximal
inequality,
P sup
0st
(B
s

1
2
s) > = P sup
0st
Z
s
() > e
EZ
t
() = e
.
Fix the and in (0, 1) and apply the inequality above to the sequences
t
n
=
n
,
n
=
n
(1 +)h(
n
),
n
=
1
2
h(
n
).
Then,
n
=
1
2
n
(1 +)h(
n
)
2
= (1 +) log log
n
= (1 +)(log n + log log
1
).
Consequently, for = (log(1/))
(1+)
,
P sup
0stn
(B
s

1
2
n
s) >
n
n
(1+)
.
Because > 0, these probabilities have a nite sum over n. Thus, by the rst Borel-Cantelli lemma
P sup
0stn
(B
s

1
2
n
s) >
n
i.o. = 0.
Thus, we have, for in an event having probability 1, there exists N() so that
sup
0stn
(B
s

1
2
n
s) >
n
3 MARTINGALES 31
for all n N(). For such n, consider t in the interval (
n+1
,
n
]. Then
B
t
() sup
0s
n
B
s
()
n
+
1
2
n
=
1
2
(2 +)h(
n
)
1
2
1/2
(2 +)h(t).
Consequently,
limsup
t0+
B
t
()
h(t)

1
2
1/2
(2 +).
Now, use the fact inf
,(0,1)
1
2
1/2
(2 +) = 1 to obtain
limsup
t0+
B
t
()
h(t)
1.
Part 2.
P
_
limsup
t0+
B
t
h(t)
1
_
= 1.
Choose (0, 1) and dene the independent events
A
n
= B
n B
n+1 > h(
n
)
1 .
Noting that B
n B
n+1 is a normal random variable with mean 0 and variance

n
(1 ), we have
P(A
n
) = 1 (
n/2
h(
n
))
1
2
exp
_
1
2
n
h(
n
)
2
_
n/2
h(
n
)
1
(1
n
h(
n
)
2
).
Because
1
2
n
h(
n
)
2
= log n + log log(
1
),
we have that
n=1
P(A
n
) =
by comparison to
n=1
(n
log n)
1
. Now, use the second Borel-Cantelli lemma to conclude that for innitely
many n
B
n B
n+1 > h(
n
)
1 .
Use the rst part of the theorem, applied to B to obtain that
B
n+1 2h(
n+1
) > 4
h(
n
)
for all suciently large n > 1/ log ,. Thus, for innitely many n,
B
n >
_
1 4
_
h(
n
).
This proves that, with probability 1,
limsup
t0
B
t
h(t)

1 4
for all (0, 1). Now, part 2 follows by noting that sup
(0,1)
1 4
= 1.
3 MARTINGALES 32
Corollary 3.48. Let B be a standard Brownian motion. Then,
P
_
limsup
t
B
t
2t log log t
= 1
_
= 1.
Proof. Use the fact that tB
1/t
is a Brownian motion.
4 MARKOV PROCESSES 33
4 Markov Processes
4.1 Denitions and Transition Functions
Denition 4.1. A stochastic process X with values in a metric space S is called a Markov process provided
that
PX
s+t
B[T
X
t
= PX
s+t
B[X
t
(4.1)
for every s, t 0 and B B(S).
If T
t
; t 0 is a ltration and T
X
t
T
t
, then we call X a Markov process with respect to the ltration
T
t
; t 0 if the (4.1) holds with T
X
t
replaced by T
t
.
The probability measure given by (B) = PX
0
B is called the initial distribution of X.
Exercise 4.2. 1. If X is Markov process with respect to the ltration T
t
; t 0 then it is a Markov
process.
2. Use the standard machine to show that the denition above is equivalent to
E[f(X
s+t
)[T
X
t
] = E[f(X
s+t
)[X
t
].
for all bounded measurable f.
Denition 4.3. A function
p : [0, ) S B(S) [0, 1]
is a called time homogeneous transition function if,
1. for every (t, x) [0, ) S, p(t, x, ) is a probability,
2. for every x S, p(0, x, ) =
x
,
3. for every B B(S), p(, , B) is measurable, and
4. (Chapman-Kolmogorov equation) for every s, t 0, x S, and B B(S),
p(t +s, x, B) =
_
S
p(s, y, B)p(t, x, dy).
The transition function P is a transition function for a time-homogeneous Markov process X, if, for
every s, t 0 and B B(S),
PX
t+s
B[T
X
t
] = p(s, X
t
, B).
Exercise 4.4. Show, using the standard machine, the the identity above is equivalent to
E[f(X
t+s
[T
X
t
] =
_
S
f(y)p(s, X
t
, dy)
for every s, t 0 and bounded measurable f.
To see how the Chapman-Kolmgorov equation arises, note that for all s, t, u 0 and B B(S),
p(t +s, X
u
, B) = PX
u+t+s
B[T
X
u
= E[PX
u+t+s
B[T
X
u+t
[T
X
u
]
= E[p(s, X
u+t
, B)[T
X
u
] =
_
S
p(s, y, B)p(t, X
u
, dy).
Exercise 4.5. Let X be a Levy process with
t
(B) = PX
t
B. Then the transition function
p(t, x, B) =
t
(B x).
In this case, the Chapman-Kolmogorov equations state that
s+t
=
s

t
.
The transition function and the initial distribution determine the nite dimensional distributions of X
by
PX
0
B
0
, X
t1
B
1
, . . . , X
tn
B
n
=
_
B0
_
B1

_
Bn1
p(t
n
t
n1
, x
n1
, B
n
)p(t
n1
t
n2
, x
n2
, dx
n1
) p(t
1
, x
0
, dx
1
)(dx
0
).
Exercise 4.6. Show that the consistency condition in the Daniell-Kolmogorov are satised via the Chapman-
Kolmogorov equation.
This gives us a unique measure on (S
[0,)
, B(S
[0,)
). Denote probabilities with respect to this measure
P
and P
x
for P
x
.
Rarely are we able to write this transition function explicitly. In two well known cases, Brownian motion
and the Poisson process we can.
Example 4.7. For Brownian motion,
P(t, x, B) =
1
2t
_
B
exp
_
(y x)
2
2t
_
dt, t 0, x R, B (R).
For the Poisson process with parameter ,
P(t, x, y) =
(t)
yx
(y x)!
e
t
, t 0, y x 0, x, y N.
The kernels P(s, , ) are naturally associated with the operator on the bounded measurable functions.
Denition 4.8. The transition operator for the transition function P is dened by
T(s)f(x) =
_
S
f(y)p(s, x, dy) = E
x
[f(X
s
)].
Consequently,
T(s)f(X
t
) =
_
S
f(y)p(s, X
t
, dy) = E[f(X
t+s
)[T
X
t
].
Exercise 4.9. The family of operators T(t); t 0 satises
1. T(0)f = f
2. T(s +t)f = T(s)T(t)f
for all bounded measurable f.
4.2 Operator Semigroups
Denition 4.10. A one-paramter family T(t); t 0 of bounded linear operators on a Banach space
(L, [[ [[) is call a semigroup if
1. T(0)f = f
2. T(s +t)f = T(s)T(t)f
for all f L. This semigroup is called strongly continuous if
lim
t0+
T(t)f = f.
for all f L. This semigroup is called a contraction semigroup if
[[T(t)[[ = sup[[T(t)f[[; [[f[[ = 1 1.
Exercise 4.11. Let A be a bounded linear operator and dene, for t 0, the linear operator
exp tA =
k=0
1
k!
t
k
A
k
, exp 0 = I.
Then, exp tA is a strongly continuous semigroup and [[ exp tA[[ exp t[[A[[.
Proposition 4.12. Let T be a strongly continuous contraction semigroup. Then, for each f L, the
mapping
t T(t)f
is a continuous function from [0, ) into L.
Proof. For t 0 and h > 0,
[[T(t +h)f T(t)f[[ = [[T(t)(T(h)f f)[[ [[T(t)[[ [[T(h)f f[[ [[T(h)f f[[.
For 0 h t,
[[T(t h)f T(t)f[[ = [[T(t h)(T(h)f f)[[ [[T(t h)[[ [[T(h)f f[[ [[T(h)f f[[.
Now, let h 0.
4.2.1 The Generator
Denition 4.13. Let A be a linear operator with domain T(A), a subspace of L and range 1(A).
1. The graph of A is given by
((A) = (f, Af) : f T(A) L L.
2. L L is a Banach space under the norm [[(f, g)[[ = [[f[[ + [[g[[. A is a closed operator if its graph is
a closed subspace of L L.
Note that if T(A) = L, then A is closed.
3. Call

A an extension of A if T(A) T(

A) and

Af = Af for all f T(A).
Denition 4.14. The (innitesimal) generator of a semigroup T is the linear operator G dened by
Gf = lim
t0
1
t
(T(t)f f).
The domain of G is the subspace of all f L for which the limit above exists.
Exercise 4.15. 1. Let L be the bounded real-valued functions on N. For the Poisson process with param-
eter , show that the generator is
Gf(x) = (f(x + 1) f(x))
and that T(G) = L.
2. Assume that A is a bounded operator. The semigroup exp tA has generator A. T(A) = L.
We will be examining continuous functions u : [a, b] L. Their Riemann integrals will be dened via
Riemann sums and limits. Thus, we will have analogous denitions for Riemann integrable and for the
improper Riemann integrals
_
a
u(s) ds and an analogous statement for the fundamental theorem
lim
h0
1
h
_
a+h
a
u(s) ds = u(a).
Exercise 4.16. 1. If u : [a, b] L is continuous and [[u[[ is Riemann integrable, then u is integrable and
[[
_
b
a
u(s) ds[[
_
b
a
[[u(s)[[ ds.
2. Let B be a closed linear operator on L and assume that u : [a, b] L is continuous, that u(s) T(B)
for all s [a, b], and that both u and Bu are Riemann integrable. Then
_
b
a
u(s) ds T(B) and B
_
b
a
u(s) ds =
_
b
a
Bu(s) ds.
3. If u is continuously dierentiable, then
_
b
a
d
dt
u(t) dt = u(b) u(a).
The example of the semigroup exp tA motivates the following identities.
Proposition 4.17. Let T be a strongly continuous semigroup on L with generator G.
1. If f L and t 0, then
_
t
0
T(s)f ds T(G) and T(t)f f = G
_
t
0
T(s)f ds. (4.2)
2. If f T(G) and t 0, then T(t)f T(G) and
d
dt
T(t)f = GT(t)f = T(t)Gf. (4.3)
3. If f T(G) and t 0, then
T(t)f f =
_
t
0
GT(s)f ds =
_
t
0
T(s)Gf ds. (4.4)
Proof. 1. Observe that (T(h) I)/h is closed for all h > 0. Thus,
1
h
(T(h) I)
_
t
0
T(s)f ds =
1
h
_
t
0
(T(s +h)f T(s)f) ds =
1
h
_
t+h
t
T(s)f ds
1
h
_
h
0
T(s)f ds.
Now, let h 0 to obtain (4.2).
2. Write G
h
= (T(h) I)/h. Then, for all h > 0
1
h
(T(t +h)f T(t)f) = G
h
T(t)f = T(t)G
h
f
Consequently, T(t)f T(G) and
d
+
dt
T(t)f = GT(t)f = T(t)Gf.
To check the left derivative, use the identity
1
h
(T(t h)f T(t)f) T(t)Gf = T(t h)(G
h
G)f + (T(t h) T(t))Gf,
valid for h (0, t].
3. This is a consequence of part 2 in this proposition and part 3 in the lemma above.
Exercise 4.18. Interpret the theorem above for the semigroup
T(s)f(x) = f(x +s)
for measurable functions f : R R.
Corollary 4.19. If G is the generator of a a strongly continuous contraction semigroup T on L, then G is
closed and T(G) is dense in L.
Proof. Because, for every f L,
lim
h0+
1
h
_
h
0
T(s)f ds = f,
T(G) is dense in L.
To see that G is closed, choose f
n
; n 1 so that
f
n
f and Gf
n
g as n .
Then, by (4.3),
T(t)f
n
f
n
=
_
t
0
T(s)Gf
n
ds for each t > 0.
Thus, by letting n , we obtain
T(t)f f =
_
t
0
T(s)g ds for each t > 0.
Now, divide by t and let t 0 to conclude that
f T(G) and Gf = g.
4.2.2 The Resolvent
We now give some necessary conditions for an operator to be the generator of a strongly continuous semi-
group. The Hille-Yosida will show that this conditions are sucient.
Denition 4.20. For a strongly continuous contraction semigroup T, dene the resolvent,
R()g =
_

0
e
s
T(s)g ds.
Exercise 4.21. If X is a time homogenuous Markov process with semigroup T and
is an independent
exponential random variable with parameter , then
R()g = Eg(X
).
Proposition 4.22. Let T be a strongly continuous contraction semigroup with generator G, then for all
> 0, (I G)
1
is a bounded linear operator on L and
R()g = (I G)
1
g
for all g L.
Proof. Fix > 0 and choose g L. Then
[[R()g[[ =
_

0
e
s
[[T(s)g[[ ds
1
[[g[[.
Also, for any h > 0,
1
h
(T(h) I)R()g =
1
h
_

0
e
s
(T(s+h)g T(s)g) dt =
e
h
1
h
_

0
e
s
T(s)g ds
e
h
h
_
h
0
e
s
T(s)g ds.
Let h 0 to see that R()g T(G) and GR()g = R()g g, or
(I G)R()g = g.
Thus, R() is a right inverse for I G and 1(I G) = L.
If g T(G), then use the fact that G is closed to see that
R()Gg =
_

0
e
s
T(s)Gg ds =
_

0
G(e
s
T(s))g ds = G
_

0
e
s
T(s)g ds = GR()g
or
R()(I G)g = g.
Thus, (I G) is one-to-one and its left inverse is R()
Denition 4.23. (G) = ; (I G) has a bounded inverse is called the resolvent set.
Thus, if G is the generator of a strongly continuous contraction semigroup, then its resolvent set (G)
contains (0, ).
Remark 4.24. Let
be an exponential random variable with parameter , then as ,
converges
in distribution to the degenerate random varibale that takes on the value 0 with probability 1. This motivates
the following.
Proposition 4.25. Let R(); 0 be the resolvent for a strongly continuous contraction semigroup with
generator G, then
lim
R()f = f.
Proof. Note that for each f T(G),
R()f f = R()Gf and that [[G
Af[[
1
[[Gf[[.
Thus, the formula holds for the dense set f T(G). Now, use the fact that [[R() I[[ 2 for all > 0
and approximate.
Exercise 4.26. 1. (resolvent identity) Let , > 0, then
R()R() = R()R() =
R() R()

.
2. If (G) and [ [ < [[R()[[
1
, then
R() =
n=0
( )
n
R()
n+1
.
Consequently, (G) is open.
3. If G is the generator of a semigroup then
[[f Gf[[ [[f[[ for every f T(G), and > 0.
4.3 The Hille-Yosida Theorem
The Hille-Yosida theorem gives a characterization for generators. We begin with a denition.
4.3.1 Dissipative Operators
Denition 4.27. A linear operator A on L is dissipative if
[[f Af[[ [[f[[ for every f T(A), and > 0. (4.5)
Remark 4.28. If A is dissipative, then divide (4.5) by to obtain that [[f Af/[[ [[f[[ or
[[f cA [[f[[ for all c > 0.
Exercise 4.29. 1. If A is dissipative, then I A is one-to-one.
2. If A is dissipative and (I A)
1
exists, then [[(I A)
1
[[
1
.
Lemma 4.30. Let A be a dissipative linear operator on L and let > 0. Then A is closed if and only if
1(I A) is closed.
Proof. Suppose that A is closed. If f
n
; n 0 T(A), and (I A)f
n
h, then
[[(I A)(f
n
f
m
)[[ [[f
n
f
m
[[,
and f
n
; n 0 is a Cauchy sequence. Thus, there exists f L such that f
n
f and hence
Af
n
f h.
Because A is closed, f T(A) and
h = (I A)f.
This shows that 1(I A) is closed.
Suppose 1(I A) is closed. If f
n
; n 0 T(A), f
n
f and Af
n
g. Consequently,
(I A)f
n
f g.
Because 1(I A) is closed,
f g = (I A)f
0
for some f
0
T(A).
Because A is dissipative,
[[(I A)(f
n
f
0
)[[ [[f
n
f
0
[[,
and f
n
f
0
. Hence, f = f
0
T(A) and Af = g. This shows that A is closed.
Lemma 4.31. Let A be a dissipative closed linear operator on L. Set
+
(A) = (A) (0, ). If
+
(A) ,= ,
then
+
(A) = (0, )
Proof. The condition is equivalent to the statement that
+
(A) is both open and closed in (0, ). We have
shown that (A) is open.
Suppose that
n
; n 1
+
(A) with
n
> 0. Given g L, dene, for each n,
g
n
= (I A)(
n
I A)
1
g.
Because A is dissipative
[[g
n
g[[ = [[((I A) (
n
I A))(
n
I A)
1
g[[ = [[(
n
)(
n
I A)
1
g[[
[
n
[
n
[[g[[.
Thus, g
n
g. Consequently, 1(I A) is dense in L. We have shown that this range is closed, thus
1(I A) = L. Because I A is one-to-one and [[(I A)
1
[[
1
, we have that
+
(A). Thus,
+
(A) is closed.
4.3.2 Yosida Approximation
The generator of a strongly continuous contraction semigroup is sometimes an unbounded operator. The
Yosida approximation gives us a method of approximating generators using bounded operators. Later, we
will give a probabilistics interpretation to this approximation.
Denition 4.32. Let G be a dissipative closed linear operator on L, and suppose that T(G) is dense in L
and that (0, ) (G). Then the Yosida approximation of G is
G
= G(I G)
1
, > 0.
Proposition 4.33. The Yosida approximation has the following properties:
1. For each > 0, G
is a bounded linear operator on L and T
(t) = exp(tG
); t 0 generates a
strongly continuous contraction semigroup on L.
2. G
= G
for all , > 0

3. For every f T(G),
lim
f = Gf.
Proof. Because
I = (I G)R() on L, A
=
2
R() I on L.
Because
R()(I A) = I on T(G), A
= R()G on T(G).
Note that
[[T
(t)[[ e
t
[[ exp(t
2
R())[[ e
t
e
t
2
]]R()]]
1.
proving part 1.
Part 2 follows from the rst identity above and the resolvent identity.
Part 3 follows from the second identity above and the fact that for f T(G), R()f f as as
.
Lemma 4.34. For a pair of commuting bounded operators B and C on L assume that [[ exp(tB)[[ 1 and
[[ exp(tC)[[ 1 for all t 0. Then
[[ exp(tB)f exp(tC)f[[ t[[Bf Cf[[.
Proof.
exp(tB)f exp(tC)f =
_
t
0
d
dt
(exp(sB) exp((t s)C))f ds
=
_
t
0
exp(sB)(B C) exp((t s)C)f ds
=
_
t
0
exp(sB) exp((t s)C)(B C)f ds.
The rest is easy.
Theorem 4.35 (Hille-Yosida). A linear operator G is the generator of a strongly continuous contraction
semigroup on L if and only if
1. T(G) is dense in L.
2. G is dissipative.
3. 1(
0
I G) = L for some
0
> 0
Proof. The necessity of these conditions has been established.
We have shown that conditions 2 and 3 implies that G is closed. By condition 3,
0
(G) and, therefore,
(0, ) (G).
As before, for each > 0, dene the Yosida approximation G
and its semigroup T
. Then by the lemma

above, we
[[T
(t)f T
(t)f[[ t[[G
f G
f[[.
for all f L, t 0 and , > 0.
Consequently, because the Yosida approximations converge to the generator G, we have, for all f T(G),
lim
(t)f
exists for all t 0 uniformly on bounded intervals. Call this limit T(t)f. Because T(G) is dense, this limit
holds for all f L.
Claim 1. T is a strongly continuous contraction semigroup.
The fact that T(0)f = f and that T(t) is a contraction is immediate. Use the identity
T(t)f f = (T(t) T
(t))f + (T
(t) I).
to check the strong continuity at 0 and
T(s +t)f T(s)T(t)f = (T(s +t) T
(s +t))f +T
(s)(T
(t) T(t))f + (T
(s) T(s))T(t)f
to check the semigroup property.
Claim 2. G is the generator of T.
For every f L, t 0, and > 0,
T
(t)f f =
_
t
0
T
(s)G
f ds.
For each f T(G) and s 0,
T
(s)G
f T(s)Gf = T
(s)(G
f Gf) + (T
(s) T(s))Gf.
Use the convergence of the Yosida approximations and the convergence of the semigroups to see that
T
(s)G
f T(s)Gf as
uniformly for s [0, t]. Thus the Riemann integrals converge and
T(t)f f =
_
t
0
T(s)Gf ds f T(G), t 0.
Consequently, the generator

G of T is an extension of G.
With this in mind, choose

f T(

G). Because I G is surjective, there exists f T(G) so that
(I

G)

f = (I G)f.
Now, evaluate this expression for two distinct values of and subtract to obtain that

f = f.
Lemma 4.36. Let A be a dissipative operator on L and suppose that u : [0, ) T(A) be continuous for
all s > 0 and Au : [0, ) L continuous. If
u(t) = u() +
_
t
Au(s) ds
for all t > > 0, then [[u(t)[[ [[u(0)[[ for all t 0.
Proof. Let 0 < = t
0
< t
1
< < t
n
= t. Then
[[u(t)[[ = [[u()[[ +
n
j=1
([[u(t
j
)[[ [[u(t
j1
)[[)
= [[u()[[ +
n
j=1
([[u(t
j
)[[ [[u(t
j
) (t
j
t
j1
)Au(t
j
)[[)
+
n
j=1
([[u(t
j
) (t
j
t
j1
)Au(t
j
)[[ [[u(t
j
) (u(t
j
) u(t
j1
))[[)
[[u()[[ +
n
j=1
([[u(t
j
)
_
tj1
tj
Au(t
j
) ds[[) [[u(t
j
)
_
tj
tj1
Au(s) ds[[)
[[u()[[ +
n
j=1
_
tj
tj1
[[Au(t
j
) Au(s)[[ ds.
The rst sum is negative by the dissipativity of A. Now use the continuity of Au and u and let max(t
j
t
j1
) 0. Then, let 0.
Proposition 4.37. Let T and S be strongly continuous contraction semigroups on L with generators G
T
and G
S
. If these two generators are equal, the T = S.
Proof. Use the lemma above with
u(t) = (T(t) S(t))f
and note that u(0) = 0 and Au(s) = 0 for all s.
Denition 4.38. Call a linear operator A on L closable if it has a closed linear extension. The closure

A
of A is the minimal closed linear extension of A.
Often the generator G is an unbounded operator. In these instances, establishing T(G) can be a tedious
process. Thus, we look for an alternative form for the Hille-Yosida theorem.
Lemma 4.39. Let G be a dissipative linear operator on L with T(G) dense in L. Then G is closable and
1(I G) = 1(I

G) for every > 0.
Proof. For the rst assertion, we show that
f
n
; n 0 T(G), f
n
0, Gf
n
g implies g = 0.
Choose g
m
; m 0 T(G) such that g
m
g. Then by the dissipativity of G,
[[(I G)g
m
g[[ = lim
n
[[(I G)(g
m
+f
n
)[[ lim
n
[[g
m
+f
n
[[ = [[g
m
[[
for every > 0. Now divide by and let to obtain, for each m, [[g
m
g[[ [[g
m
[[. Letting m
yields g = 0.
The inclusion 1(I

G) 1(I G) is straightforward. The reverse inclusion follows from the fact
that dissipative linear operator

G is closed if and only if 1(I

G) is closed.
Theorem 4.40. A linear operator G on L is closable and its closure

G is the generator of a strongly
continuous contraction semigroup on L if and only if
1. T(G) is dense in L.
2. G is dissipative.
3. 1(
0
I G) is dense in L for some
0
> 0
Proof. By the lemma above G satises 1-3 if and only if G is closable and

G satises 1-3 in the Hille-Yosida
theorem.
Denition 4.41. Let A be a closed linear operator on L. A subspace D of T(A) is called a core for A if A
is the closure of the restriction of A to D.
Proposition 4.42. Let G be the generator of a strongly continuous contraction semigroup on L. Then a
subspace D of T(G) is a core for G if and only if D is dense in L and 1(
0
I G) is dense in L for some
0
> 0.
4.3.3 Positive Operators and Feller Semigroups
We now return to the consideration of Markov processes.
Denition 4.43. Let S be locally compact and let C
0
(S) be the Banach space of continuous functions that
vanish at innity with norm
[[f[[ = sup[f(x)[ : x S.
Denition 4.44. Call an operator on a function space positive if it maps nonnegative functions to nonneg-
ative functions.
To establish positivity, we will use the following proposition.
Proposition 4.45. Let T be a strongly continuous contraction semigroup on L with generator G and resol-
vent R. For M L, let
M
= > 0; R() : M M.
If M is closed and convex and
M
is unbounded, then for each t 0,
T(t) : M M.
Proof. For , 0 satisfying [1 /[ < 1,
R() =
n=0
_
1

_
n
(R())
n+1
.
Because M is closed and convex,
M
implies (0, ]
M
. Thus,
M
= (0, ). Use the Yosida
approximation
T
(t) = e
t
exp(t(R())) = e
t
n=0
(t)
n
n!
(R())
n
to conclude that
T
(t) : M M.
Now use the fact that M is closed and that T
(t)f converges to to T(t)f as .

Denition 4.46. 1. Call a semigroup conservative if there exist a sequence f
n
C
0
(S) that is bounded
in norm and converges pointwise to 1, and
lim
n
T(t)f
n
= 1.
for all t 0
2. Call a conservative strongly continuous positive contraction semigroup T on C
0
(S) a Feller semigroup.
Exercise 4.47. 1. If T is conservative, then A1 = 0
2. If A1 = 0, then T is conservative.
4.3.4 The Maximum Principle
Denition 4.48. A linear operator A on C
0
(S) satises the positive maximum principle if whenever f
T(A), x S, then
f( x) = supf(x); x S 0 implies Af( x) 0.
Remark 4.49. To see why this ought to be true for a Markov process, note that
0 E
x
[f(X
s
)] f( x) = T(s)f( x) f( x).
now divide by s and let s 0.
Proposition 4.50. Let S be locally compact. A linear operator A on C
0
(S) satisfying the positive maximum
principle is dissipative.
Proof. Let f T(A) and > 0. There exists x S so that [f( x)[ = [[f[[. Suppose f( x) 0. (Otherwise
replace f with f.) Then by the positive maximum principle, Af( x) 0 and hence
[[f Af[[ f( x) Af( x) f( x) = [[f[[.
In this context, we have the following variant of the Hille-Yosida theorem.
Theorem 4.51. Let S be locally compact. A linear operator G on C
0
(S) is closable and its closure

G is the
generator of a positive strongly continuous contraction semigroup on C
0
(S) if and only if
1. T(G) is dense in C
0
(S).
2. G satises the positive maximum principle.
3. 1(
0
I G) is dense in C
0
(S) for some
0
> 0.
Proof. The necessity of conditions 1 and 3 follows from the theorem above. To check the neccessity of 2, x
f T(G) and x S so that supf(x) : x S = f( x) 0. Then, for each t 0,
T(t)f( x) T(t)f
+
( x) [[f
+
[[ = f( x)
and Gf( x) 0.
Conversely, suppose G satises conditions 1-3. Because condition 2 implies that G is dissipative,

G
generates a strongly continuous contraction semigroup T. To prove that T is positive, we note, by the
proposition above, it suces to prove that R() maps nonnegative functions to nonegative functions.
Because 1(
0
I G) is dense for some
0
> 0, and G is dissipative, 1(I

G) = C
0
(S) all > 0. Thus,
it is equivalent to show that for f T(G), and > 0,
(I

G)f 0 implies f 0.
To establish the contrapositive of this statement, choose f T(

G) so that inff(x) : x S < 0. Thus,
there exist f
n
; n 0 T(G) so that
lim
n
(I G)f
n
= (I

G)f.
Because

G is dissipative, we have that f
n
f as n . Let x
n
and x be the points in which f
n
and f,
respectively take on their minimum values. Then
inf(I

G)f; x S liminf
n
(I

G)f
n
( x
n
) liminf
n
f
n
( x
n
) = f( x) < 0.
Exercise 4.52. Let X
1
and X
2
be independent Markov processes on a common state space S. Is (X
1
, X
2
)
a Markov process? If so, determine its generator?
4.4 Strong Markov Property
Denition 4.53. Let X be a T
t
-progressive, T
t
Markov process with associated transition function P. Let
be an almost surely nite T
t
-stopping time. Then X is strong Markov at if for all t 0 and B B(S),
PX
+s
B[T
X
= p(s, X
, B),
or equivalently, for any t 0 and bounded measurable f,
E[f(X
+s
)[T
X
] =
_
S
f(y)p(s, X
, dy).
X is called strong Markov with respect to T
t
; t 0 if X is strong Markov for all almost nite T
t
-
stopping times.
In reviewing the results on Levy processes and its regularity properties, we obtain the following.
Proposition 4.54. A right continuous Levy process is strong Markov.
The following proposition has essentially the same proof as in the discrete time case.
Proposition 4.55. Let X be a T
t
-progressive, T
t
Markov process with associated transition function P.
The X is strong Markov at all discrete almost surely nite T
t
-stopping times.
Now, using the essentially the same proof we had for Levy processes, we have
Proposition 4.56. Let X be a right continuous T
t
-progressive, T
t
Markov process with associated transition
function P. Then X is strong Markov at all almost surely nite T
t+
-stopping times.
Theorem 4.57. Let X be a Markov process with time homogenous transition function P. Let Y be a bounded
measurable process and assume that and let be a T
X
t
-stopping time so that X is strong Markov at + t
for all t 0. Then
E
[Y
[T
] =
(X
) on <
where
s
(x) = E
x
Y
s
.
Proof. By the standard machine, we need only consider functions of the form
Y
s
= f
0
(s)
n
k=1
f
k
(X
t
k
), 0 < t
1
< < t
n
.
f
0
, f
1
. . . . , f
n
continuous and bounded. In this case
s
(x) = f
0
(s)
_ _

_
f
n
(x
n
)p(t
n
t
n1
, x
n1
, dx
n
)f
n1
(x
n1
)p(t
n1
t
n2
, x
n2
, dx
n1
)
f
1
(x
1
)p(t
1
, x, dx
1
) = f
0
(s)(x).
The proof will process by induction on n. The case n = 0 states that
E
[f
0
()[T
] = f
0
() on < .
This follows form the fact that is T
-measurable.
Let
(x) = E[
n
k=2
f
k
(X
t
k
)].
Then,
s
(x) = f
0
(s)
_
(x
1
)f
1
(x
1
)p(t
1
, x, dx
1
)
On < ,
E
[Y
[T
] = E
[f
0
()
n
k=1
f
k
(X
+t
k
)[T
]
= f
0
()E
[E
[
n
k=2
f
k
(X
+t
k
)[T
+t1
]f
1
(X
+t1
)[T
]
= f
0
()E
(X
+t1
)f
1
(X
+t1
)[T
]
= f
0
()E
[(
(X
t1
)f
1
(X
t1
))
[T
]
= f
0
()(X
) =
(X
)
Using essentially the same proof as in the case of Levy processes, we have the following.
Theorem 4.58 (Blumenthal 0-1 law). Let X be a right continuous T
t
-progressive, T
t
Markov process. Then
for each x S every event in T
X
0+
has probability 0 or 1.
Corollary 4.59. Let be an T
X
t+
-stopping time and x x S. Then either
P
x
= 0 = 1 or P
x
> 0 = 1.
4.5 Connections to Martingales
Proposition 4.60. Let X be an S-valued progressive Markov process and let T, the semigroup assocated to
X, have generator G. Then for f T(G),
M
f
t
= f(X
t
)
_
t
0
Gf(X
s
) ds
is an T
X
t
-martingale.
Proof. For each t, u 0
E[M
f
t+u
[T
X
t
] = T(u)f(X
t
)
_
t
0
Gf(X
s
) ds
_
t+u
t
T(s t)Gf(X
t
) ds
= T(u)f(X
t
)
_
u
0
T(s)Gf(X
t
) ds
_
t
0
Gf(X
s
) ds
= f(X
t
)
_
t
0
Gf(X
s
) ds = M
f
s
Exercise 4.61. Let X be a right continuous Markov process with generator G and assume that
f(X
t
)
_
t
0
g(X
s
) ds
is a martingale, then f T(G) and Gf = g.
The optional sampling formula gives us the following.
Theorem 4.62 (Dynkin formula). In addition to the conditions above, let be a stopping time with E
<
, then
E
f(X
) = E
f(X
0
) +E
[
_

0
Gf(X
s
) ds]
Analogous to the situation for discrete time Markov chains, we make the following denitions.
Denition 4.63. Let G be the generator for a time homogeneous Markov process X and let f T(G).
Then call
1. f harmonic if Gf = 0.
2. f superharmonic if Gf 0.
3. f subharmonic if Gf 0.
Remark 4.64. 1. If f harmonic, then f(X
t
) is a martingale.
2. If f superharmonic, then f(X
t
) is a supermartingale.
3. If f subharmonic, then f(X
t
) is a submartingale.
Example 4.65. 1. Let D
0
, D
1
are two disjoint subsets of the state space S. If h is harmonic with respect
to a Markov process X having generator G and
h(x) = 0 for x D
0
, h(x) = 1 for x D
1
.
Assume that the rst entrance times
Dj
, j = 0, 1 are stopping times and is their minimum. Then if
has nite mean,
h(x) = E
x
h(X
) = P
D0
>
D1
.
2. If we can solve the equation Gf = 1, then
M
f
t
= f(X
t
) +t
is martingale. Consider a domain D so that
D
is a stopping time. Gf = 1, f = 0 on D, and
D
satises the sampling integrability conditions for M
f
, then
f(x) = E
x
D
.
Lemma 4.66. Assume that g 0 and let R() be the resolvent for a positive continuous contraction
semigroup T associated to a Markov process X. Then
e
t
R()g(X
t
)
is a non-negative T
X
t+
-supermartingale.
Proof. For t > s, use the fact that the semigroup and the resolvent commute to obtain that
E[e
t
R()g(X
t
)[T
X
s+
] = e
t
T(t s)R()g(X
s
)
= e
t
_

0
e
u
T(t s +u)g(X
s
) du
= e
s
_

ts
e
u
T(u)g(X
s
) du
e
s
R()g(X
s
)
Lets continue this line analysis. Consider the bounded random variable,
Y =
_

0
e
s
g(X
s
) ds
Then
Y =
_
t
0
e
s
g(X
s
) ds +e
t
Y
t
and
E
x
Y = R()g(x).
We have Doobs martingale
Z
,g
t
= E[Y [T
X
t
] =
_
t
0
e
s
g(X
s
) ds +e
t
E[Y
t
[T
X
t
]
=
_
t
0
e
s
g(X
s
) ds +e
t
R()g(X
t
)
Now, because Doobs martingale is uniformly integrable, any stopping time satises the sampling
integrability conditions for Z
,g
and therefore,
R()g(x) = E
x
[
_

0
e
s
g(X
s
) ds] +E
x
[e
R()g(X
)].
Now, let f = (I G)g, then the Doobs martingale above becomes
C
,f
t
= e
t
f(X
t
) +
_
t
0
e
s
(I G)f(X
s
) ds
and the analog to Dynkins formula becomes
f(x) = E
x
[e
f(X
)] +E
x
[
_

0
e
s
(I G)f(X
s
) ds].
Let f
satisfy the eigenvalue problem Gf
= f
with f
= 1 on D, then
f
(x) = E
x
e
D
.
Theorem 4.67. Let X be a measurable T
t
-adapted process and let f and g be bounded, measurable functions,
inf
x
f(x) > 0. Assume that
Y
t
= f(X
t
)
_
t
0
g(X
s
) ds
is an T
t
martingale. Then
f(X
t
) exp
_
_
t
0
g(X
v
)
f(X
v
)
dv
_
is a martingale.
Proof. Let
V
t
= exp
_
_
t
0
g(X
v
)
f(X
v
)
dv
_
.
Then, we have the local martingale
Y
t
V
t

_
t
0
Y
s
dV
s
=
_
f(X
t
)
_
t
0
g(X
s
) ds
_
exp
_
_
t
0
g(X
v
)
f(X
v
)
dv
_
+
_
t
0
_
f(X
s
)
_
s
0
g(X
u
) du
_
g(X
s
)
f(X
s
)
exp
_
_
s
0
g(X
v
)
f(X
v
)
dv
_
= f(X
t
) exp
_
_
t
0
g(X
v
)
f(X
v
)
dv
_
_
t
0
g(X
s
) ds exp
_
_
t
0
g(X
v
)
f(X
v
)
dv
_
+
_
t
0
g(X
s
) exp
_
_
s
0
g(X
v
)
f(X
v
)
dv
_
ds
_
t
0
_
s
0
g(X
u
)
g(X
s
)
f(X
s
)
exp
_
_
s
0
g(X
v
)
f(X
v
)
dv
_
du ds
For the double integral, reverse the order of integration, then we have, upon integrating the s variable that
_
t
0
g(X
u
) exp
_
_
s
0
g(X
v
)
f(X
v
)
dv
_
t
u
du
=
_
t
0
g(X
u
) exp
_
_
t
0
g(X
v
)
f(X
v
)
dv
_
du +
_
t
0
g(X
u
) exp
_
_
u
0
g(X
v
)
f(X
v
)
dv
_
du
and the last three terms sum to zero.
Now use the fact that f and g are bounded and f is bounded away from 0 to see that the local martingale
above is indeed a martingale.
Exercise 4.68. Let X be a measurable T
t
-adapted process and let f and g be bounded, measurable functions,
1. Assume that inf
x
f(x) > 0 and that
f(X
t
) exp
_
_
t
0
g(X
v
)
f(X
v
)
dv
_
is an T
t
martingale. Then
f(X
t
)
_
t
0
g(X
s
) ds
is an T
t
martingale.
2. Assume that
f(X
t
)
_
t
0
g(X
s
) ds
is an T
t
martingale. Then
e
t
f(X
t
) +
_
t
0
e
s
(f(X
s
) g(X
s
)) ds
is an T
t
martingale.
4.6 Jump Processes
Denition 4.69. Call a time homogenuous Markov process X a pure jump process, if starting from any
point x S, the process is right continuous and has all its sample paths constant except for isolated jumps.
In the case of a countable state space, the stochastic nature of a pure jump process is captured by the
innitesimal transitions rates
P
x
X
h
= y = g(x, y)h +o(h).
The innitesimal generator G is the rate of change of averages for a function f : S R of the process.
Gf(x) = lim
h0
E
x
f(X
h
) f(x)
h
, f T(G).
To relate these two concepts, write
E
x
f(X
h
) =
y,=x
f(y)g(x, y)h +f(x)(1
y,=x
g(x, y)h) +o(h)
E
x
f(X
h
) f(x) =
y,=x
g(x, y)(f(y) f(x))h +o(h)
Gf(x) =
yS
g(x, y)(f(y) f(x))
Thus, G can be written as an innitesimal transition matrix. The xy-entry x ,= y is g(x, y). The diagonal
entry
g(x, x) =
y,=x
g(x, y).
Thus, the o-diagonal entries of the matrix G are non-negative and the row sum is 1.
4.6.1 The Structure Theorem for Pure Jump Markov Processes
For processes that move from one state to another by jumping the exponential distribution plays an important
role.
Proposition 4.70. Let X be a pure jump process, then
1
= inft 0; X
t
,= X
0
is an T
t+
-stopping time.
Proof.
1
= inf
n
: n 1 where
n
= infk2
n
; X
k/2
n ,= X
0
Now
n
t T
X
t
T
t
.
The
n
are T
t
-stopping times and, consequently,
1
is an T
t+
-stopping time.
Theorem 4.71 (structure theorem for pure jump Markov processes). Under P
x
,
1
and X
1
are independent
and there is a B(S) measurable function on S such that
P
x
1
> t = exp((x)t).
Proof. Set
e
x
(t+s) = P
x
1
> t+s = P
x
1
> t+s, X
t
= x,
1
> t = P
x
1
> t+s[X
t
= x,
1
> tP
x
X
t
= x,
1
> t
Note that
P
1
> t +s[X
t
= x,
1
> t = P
x
1
> t +s[X
t
= x = P
x
1
> s = e
x
(s).
and that P
x
X
t
= x,
1
> t = P
x
1
> t = e
x
(t). Thus,
e
x
(t +s) = e
x
(s)e
x
(t)
or for some [0, ], e
x
(t) = exp((x)t). The function is measurable because P
x
1
> t is measurable.
Let
1
(t) be the rst exit from state x after time t. Then, for B B(S), x / B,
P
x
X
1
B,
1
> t = P
x
X
1(t)
B,
1
> t, X
t
= x = P
x
X
1(t)
B[
1
> t, X
t
= xP
x
1
> t, X
t
= x.
For the rst term
P
x
X
1(t)
B[
1
> t, X
t
= x = P
x
X
1(t)
B[X
t
= x = P
x
X
1
B.
For the second term,
P
x
1
> t, X
t
= x = P
x
1
> t.
Thus, the time of the rst jump and the place of the rst jump are independent.
Let : S B(S) [0, 1] be a transition function and dened by (x, B) = P
x
X
1
B, then
E
x
f(X
h
) = E
x
[f(X
h
)[
1
> h]P
x
1
> h +E
x
[f(X
h
)[
1
h]P
x
1
h
= f(x)e
(x)h
+
_
f(y)(x, dy)(1 e
(x)h
) +o(h)
Thus,
E
x
f(X
h
) f(x) = (1 e
(x)h
)
__
f(y)(x, dy) f(x)
_
+o(h)
and
Gf(x) = (x)
_
(f(y) f(x))(x, dy).
In the case of a countable set of states, set (x, y) = T(x, y).
Gf(x) = (x)
yS
T(x, y)(f(y) f(x))
Equating the two expressions for the generator, we nd that
g(x, x) = (x)
and for y ,= x
g(x, y) = (x)T(x, y) or T(x, y) =
g(x, y)
(x)
For example, let S = 1, 2, 3, 4,
G =
_
_
_
_
5 2 3 0
4 10 3 3
2 2 5 1
5 0 0 5
_
_
_
_
, T =
_
_
_
_
0 .4 .6 0
.4 0 .3 .3
.4 .4 0 .2
1 0 0 0
_
_
_
_
, =
_
_
_
_
5
10
5
5
_
_
_
_
Exercise 4.72. 1. Let T
1
, . . . , T
n
be independent, exponential random variables, having respectively, pa-
rameters
1
, . . . ,
n
, then
minT
1
, . . . , T
n
is an exponential random variable with parameter

1
+ +
n
.
2. PT
1
< minT
2
, . . . , T
n
=
1
/(
1
+ +
n
).
Exercise 4.73. Consider a stochastic process X with a nite state space S dened as follows.
1. For every x, y S, x ,= y, let N
(x,y)
denote a Poisson process with parameter g(x, y). Assume that
these processes are independent.
2. if X
t
= x, then X jumps to y if
y = arg min y; N
(x, y)
s
, s t.
3. The rst jump after time t takes place at min
y
N
(x, y)
s
: s t.
Show that X is a Markov process whose generator G can be represented by a matrix with xy-entry g(x, y).
4.6.2 Construction in the Case of Bounded Jump Rates
Let : S [0,
max
] be measurable. Dene
Gf(x) = (x)
_
S
(f(y) f(x)) (x, dy).
Exercise 4.74. Show that G satises the maximum principle.
We can construct the Markov process on S with generator G as follows:
Let Y
k
; k 0 be a Markov chain on S with initial distribution and transition function . In addition,
independent of the chain Y , let
k
; k 0 be independent exponentially distributed random variables with
parameter 1. Set
0
= 0,
k
=
k1
j=0
j
(Y
j
)
and
X
t
= Y
k
,
k
t <
k+1
.
Note that we can allow (x) = 0. In this case, once X arrives to x it remains there. Thus, if the Markov
process arrives at the site x is remains there an exponential length of time, parameter (x) and then jumps
to a new site according to the transition function .
Let us consider the alternative representation for G. Dene
(x, B) =
_
1
(x)
max
_
x
(B) +
(x)
max
(x, B).
Exercise 4.75. If

Gf(x) =
max
_
S
(f(y) f(x)) (x, dy), then G =

G
Write
Tf(x) =
_
S
f(y) (x, dy)
for the transition operator for the Markov chain

Y . Then
G =
max
(T I).
The semigroup generated by G is
T(t) = exp(tG) = e
maxt
k=0
k
max
k!
T
k
.
Let
Y
k
; k 0 be a Markov chain on S with initial distribution and transition function and let N
be an independent Poisson process with parameter
max
, and dene
X
t
=

Y
Nt
.
Proposition 4.76. Let T
t
= (

X
s
, N
s
); 0 s t, then

X is an T
t
-Markov process.
Proof. We begin with a claim.
Claim. E[f(
Y
k+Nt
)[T
t
] = T
k
f(

X
t
).
Fix A T
N
t
and B T
Y
m
. Then
E[f(
Y
k+Nt
); A B N
t
= m] = E[f(
Y
k+m
); A B N
t
= m] = P(A N
t
= m)E[f(
Y
k+m
); B]
= P(A N
t
= m)E[T
k
f(
Y
m
); B] = E[T
k
f(

X
t
); A B N
t
= m]
However, sets of the form AB N
t
= m are closed under nite intersection, and the claim follows from
the Sierpinski class theorem.
Now, use the independence of the increments for a Poisson process, to obtain that
E[f(

X
t+s
)[T
t
] = E[f(
Y
Nt+s
)[T
t
] = E[f(
Y
Nt+sNt+Nt
)[T
t
]
=
k=0
e
maxs
(
max
s)
k
k!
E[f(
Y
k+Nt
)[T
t
] =
k=0
e
maxs
(
max
s)
k
k!
T
k
f(

X
t
) = T(s)f(

X
t
)
Because G =

G, X is a Markov process whose nite dimensional distributions agree with

X.
Thus, all of the examples of generators with bounded rates are generators for a semigroup T for a Markov
process on C
0
(S).
Exercise 4.77. For a two state Markov chain on S = 0, 1 with generator
A =
_

_
,
show that
P(t, 0, 0) =

+
+

+
e
(+)t
.
4.6.3 Birth and Death Process
Example 4.78. 1. For a compound Poisson process,
X
t
=
Nt
n=1
Y
n
,
(x) is some constant , the rate for the Poisson process. If is the distribution of the Y
n
, then
(x, B) = (B x).
2. A process is called a birth and death process if S = N,
g(x, y) =
_
_
_
x
y = x + 1
(
x
+
x
) y = x
x
y = x 1
3. A pure birth process has
x
= 0 for all x.
4. A pure death process has
x
= 0 for all x.
5. A simple birth and death process has
x
= and
x
= for all x.
6. A linear birth and death process has
x
= x and
x
= x for all x.
7. A linear birth and death process with immigration has
x
= x and
x
= x +a for all x.
8. M/M/1 queue S = 0, 1, , N,
x
= , x < N,
n
= 0,
x
= . So customers arrive at an
exponential rate, parameter until the queue reaches length N at which point, all arrivals are turned
away. When they reach the front of the queue, they are served immediately. The service time is
exponential, parameter .
9. M/M/ queue S = N
x
= ,
n
= 0,
x
= x. So customers arrive at an exponential rate,
parameter . All arrivals are served immediately. The service time is exponential, parameter .
10. A logistic process S = 0, 1, , N,
x
= rx(Nx),
x
= x. This can be used to model an epidemic.
The susceptibility is of the N x non-infected individuals is proportional to the number x of infected
individuals. An infected stays in this state for an exponential rate of time, parameter .
11. An epidemic model with immunity can be constructed by having S = 0, 1, . . . , N
2
. The state (x, y)
gives the number of infected and the number of immune. The generator
Gf(x, y) = rx(Nxy)(f(x+1, y) f(x, y)) +x(f(x1, y +1) f(x, y)) +y(f(x, y 1) f(x, y)).
12. For the Moran model with mutation S = 0, 1, , N, consider a genetic model with two alleles, A
1
and A
2
X
t
- number of alleles of type A
1
at time t. The process remains at a site for an exponential
length of time, parameter , an individual is chosen at random to be replaced. A second individual
is chosen at random to duplicate. A mutation may also occur at birth. The mutation A
1
A
2
has
probably
1
and the mutation A
2
A
1
has probably
2
The generator
Gf(x) = (1
x
N
)
_
x
N
(1
1
) + (1
x
N
)
2
_
(f(x + 1) f(x))
+
x
N
_
(1
x
N
)(1
2
) +
x
N

1
_
(f(x 1) f(x))
13. For a random walk on the lattice S = Z
d
, X
t
- position of the particle at time t. The particle at a
site x remains at x for an exponential length of time, parameter (x), and then moves according to the
kernel p(x, y).
Thus,
y
p(x, y) = 1
and the generator
Gf(x) = (x)
y
p(x, y)(f(y) f(x)).
Exercise 4.79. For the compound Poisson process, if Y
n
; n 1 is a Bernoulli sequence, the X is a
Poisson process with parameter p
To nd harmonic functions

h for birth and death processes, note that
x
(
h(x + 1)
h(x)) +
x
(
h(x 1)
h(x)) = 0.
(
h(x + 1)
h(x)) =

x
x
(
h(x)
h(x 1)),
x
> 0.
Summing this on x, we obtain
(
h(x + 1)
h(x)) =
x
x=1
x
(
h(1)
h(0)).
Assume that

h(0) = 0 and

h(1) = 1 and all of the
x
> 0. Then any other non-constant harmonic
function is a linear function of

h. Then
h(y) =
y1
x=0
(
h(x + 1)
h(x)) =
y1
x=1
x
x=1
x
.
Thus, for a simple birth and death process and for a linear birth and death process,
h(y) =
y1
x=0
_
_
x
=
1 (/)
y
1 (/)
.
Let D
0
= 0 and D
1
= N, then
h(x) =

h(x)
h(N)
is a harmonic function satisfying
h(0) = 0, h(N) = 1.
Thus, for x 0, . . . , N
h(x) = P
x
0
>
N
Exercise 4.80. For a birth and death process, let = min

0
,
N
, for x 0, . . . , N.
1. Find E
x
.
2. Find E
x
e
, > 0.
3. Consider what happens as N .
Exercise 4.81. 1. For a simple birth and death process, X, nd E
x
X
t
and Var
x
(X
t
).
2. For a linear birth and death process, X, nd E
x
X
t
and Var
x
(X
t
)
4.6.4 Examples of Interacting Particle Systems
An interacting particle aystems is a Markov process
t
; t 0 whose state space S is the congurations on
a regular lattice , i.e, the state is a mapping
: F
where F is a nite set. Examples of lattices are Z
d
, Z/MZ)
d
, hexagonal lattice, and regular trees. Examples
of F are
0,1 = vacant, occupied
-1,1= spin up, spin down
0,1, ,k = vacant, species 1, species 2, , species k
, 0, , N = vacant, 0 individuals having allele A, , N individual having allele A
Example 4.82 (interacting particle systems). 1. Exclusion Process (Spitzer, 1970). F = 0, 1, =
Z
d
.
If initially we have one particle at x
0
, i.e.,
0
(x) =
_
1, if x = x
0
,
0, if x ,= x
0
.
Set X
t
=
1
t
(1). Then X
t
; t 0 is the random walk on the lattice.
When there are many particles, then each executes an independent random walk on the lattice excluding
transitions that take a particle to an occupied site. The generator
(F() =
x,y
(, x)p(x, y)(x)(1 (y))(F(
xy
) F())
where
xy
(z) =
_
_
(x), if z = y,
(y), if z = x,
(z), otherwise.
Lets read the generator:
Particle at x
0
moves in time t with probability (, x)t +o(t)
Particle at x
0
move to y with probability p(x
0
, y).
If site y is occupied, nothing happens.
2. Coalescing Random Walk. F = 0, 1, = Z
d
.
Each particle executes an independent random walk on the lattice until the walk takes a particle to an
occupied site, then this particle disappears. The generator
(F() =
x,y
(, x)p(x, y)(x)(1 (y))(F(
xy
) F()) +(y)(F(
x
) F())
where
x
(z) =
_
1 (x), if z = x,
(z), otherwise.
3. Voter Model (Cliord and Sudbury, 1973, Holley and Liggett, 1975). F = 0, 1=no, yes, = Z
d
.
An individual changes opinion at a rate proportional to the number of neighbors that disagrees with the
individuals present opinion. The generator
(F() =
x,y
(, x)(F(
x
) F())
where
(, x) =

2d
y;]yx]=1]
I
(y),=(x)]
.
4. Contact process (Harris, 1974). F = 0, 1 =uninfected, infected, = Z
d
.
The generator
(F() =
x,y
(, x)(F(
x
) F())
where
(, x) =
_
y;]yx]=1]
(y), if (x) = 0,
1, if (x) = 1.
Thus, is the relative infection rate.
5. For the stochastic Ising model, F = 1, 1. = Z
d
.
(F() =
x,y
exp
_
_
y:]xy]=1
(x)(y)
_
_
(F(
x
) F())
where

x
(z) =
_
(x), if z = x,
(z), otherwise.
4.7 Sample Path Regularity
The goal of this section is to show that for S separable, every Feller semigroup on C
0
(S) corresponds to a
Markov process with sample paths in D
S
[0, ). In addition, we show that every generator A of a positive
strongly continuous contraction semigroup on C
0
(S) corresponds to a Markov process.
We have seen that a jump Markov process with bounded jump rates : S [0, ], corresponds to a
Feller process X on S. Moreover, this process has a D
S
[0, ) version.
If the rates are bounded on compact sets, then because S is locally compact, we can write, for S separable,
S =
_
n=1
K
n
for an increasing sequence of compact sets K
n
; n 1 and dene
n
= inft > 0; X
t
/ K
n
.
Then,
n
; n 1 is an increasing sequence of stopping times. Dene
= lim
n
n
.
If this limit is nite, then we say that an explosion occurs. In these circumstances, we have that the transition
function fails to satisfy P(t, x, S) = 1 for all t > 0, and the semigroup T is no longer conservative.
4.7.1 Compactifying the State Space
To include this situation in the discussion of Markov processes, the state space S has adjoined to it a special
state and we write S
= S and let
X
t
= for t .
If S is not compact, then S
is the one point compactication of S.

This suggest the following modication.
Theorem 4.83. Let S be locally compact and separable and let T be a strongly continuous positive contraction
semigroup on C
0
(S). For each t 0, dene the operator T
(t) on C(S
) by
T
(t)f = f() +T(t)(f f()).

Then T
is a Feller semigroup on C(S
).
Proof. The verication that T
is a strongly continuous conservative semigroup on C(S
) is straightforward.
Claim 1. T
is positive. In other words, given c R

+
and f C
0
(S), c+f 0 implies that c+T(t)f 0.
Because T is positive,
T(t)(f
+
) 0 and T(t)(f
) 0.
Hence,
T(t)f T(t)(f
) and so (T(t)f)
T(t)(f
).
Because T(t) is a contraction, we obtain
[[T(t)(f
)[[ [[f
[[ c.
Therefore,
T(t)(f
)(x) c and so c +T(t)f 0.

Claim 2. T
is a contraction.
Because T
is positive,
[T
(t)f(x)[ T
(t)[[f[[ = [[f[[
and [[T
(t)[[ 1.
With this modication, the condition of being a Feller semigroup states that the distribution of the
Markov process at time t depends continuously on the initial state. In other words, for f C(S
),
lim
xx0
T(t)f(x) = T(t)f(x
0
),
lim
xx0
E
x
f(X
t
) = E
x0
f(X
t
),
lim
xx0
_
S
f(y)P(t, x, dy) =
_
S
f(y)P(t, x
0
, dy).
Thus P(t, x, ) converges in distribution to P(t, x
0
, ).
To check for explosions, consider a pure birth process. For
x
= inft 0 : X
t
= x, we have, if X
0
= 0,
x
=
1
+ +
x
where
x
is the length of time between the arrival of X to x 1 and x.
Then, the random variables
x
; x 1 are independent exponential random variables with parameters
x1
; x 1. Thus, the Laplace transform
E[e
u x
] =

x1
u +
x1
,
and
E[e
ux
] =
x
x=1
x1
x1
+u
=
_
x
x=1
(1 +
u
x1
)
_
1
.
For u > 0, the innite product,
x=1
(1 +
u
x1
),
converges if and only if
x=1
1
x1
< .
Thus, if the sum is innite, then we have no explosion.
In particular, a linear birth process has no explosion. However, if, for example,
liminf
x
x
x
p
> 0
for some p > 1, then the birth process has an almost surely nite explosion time.
For a birth and death process, X, we consider the pure birth process,

X with the same birth rates as X.
Couple the two processes together in the sense that the exponential time for a jump forward is the same for
the rst time each process reaches any site x. In this way,
X
t
X
t
for all t 0. Thus, if X has no explosion, neither does

X.
We can use the previous section to show that the Yosida approximation generates a Markov process. Write
A
= (R() I) We know that R() is a positive linear operator on C

0
(S). By the Riesz representation
theorem, there exists a positive Borel measure
R()f(x) =
_
S
f(y)
(x, dy).
Thus, for f T(G)
f(x) =
_
S
(I G)f(y)
(x, dy).
Because [[R()[[ 1, we have that
(x, S) 1. However, if G is conservative, we can choose f(x) = 1/

to obtain
1 =
(x, S).
Note that if we have that G generates a Markov process, then
R()f(x) =
_
S
e
t
E
x
f(X
t
) dt = E
x
f(X
)
where
is an exponential random variable, parameter , and independent from X. Thus,
(x, ) is the
distribution of X
under P
x
.
Exercise 4.84. let D be a dense subset of C
0
(S). Then a function x D
S
[0, ) if and only if f(x)
D
R
[0, ) for every f D.
Theorem 4.85. Let X be a T
X
t+
-Feller process, then X has a D
S
[0, )-valued modication.
Proof. Let X have generator G and let f T(G), then the martingale
M
f
t
= f(X
t
)
_
t
0
Gf(X
s
) ds
is right continuous on a set
f
of probability 1. Thus
f(X
t
) = M
f
t

_
t
0
Gf(X
s
) ds
is right continuous on
f
. Now take a countably dense set D T(A) of f. Then
fD
f
X D
S
[0, ).
Theorem 4.86. Let S be locally compact and separable. Let T be a positive strongly continuous contraction
semigroup on C
0
(S) and dene T
as above. Let X be a Markov process corresponding to T
with sample
paths in D
S
[0, ) and let
= inft 0 : X
t
= or X
t
= .
Then on the set < ,
X
+s
= for all s 0.
Proof. By the Urysohn lemma, there exists f C(S
) such that f > 0 on S and f() = 0. Then the

resolvent R
1
f > 0 on S and R
1
f() = 0. Note that e
t
R
1
f(X
t
) is a supermartingale. Because a right
continuous nonnegative supermartingale is zero after its rst contact with 0, the theorem follows.
Proposition 4.87. If, in addition to the conditions above, if the extention the semigroup T to the bounded
measurable functions is conservative, then (S) = PX
0
S = 1 implies that P
X D
S
[0, ) = 1.
Proof. Let G
be the generator of T
, then G
I
S
= 0, thus
E
I
S
(X
t
) = E
I
S
(X
0
) +E
[
_
t
0
G
I
S
(X
s
) ds] = E
I
S
(X
0
) + 0.
or
P
> t = P
X
t
S = P
X
0
S = 1.
Theorem 4.88. Let S be locally compact and separable. Let T be a Feller semigroup on C
0
(S). Then for
each probability measure on S, there exists a Markov process X with initial distribution and sample paths
in D
S
[0, ). Moreover, X is strong Markov with respect to the ltration T
X
t+
; t 0.
Proof. Let G be the generator for T and let G
, > 0, be its Yosida approximation. Then T
is the
semigroup for a jump Markov process X
. Moreover, for every t 0 and f C

0
(S),
lim
(t)f = T(t)f.
Let T
and T
be the corresponding semigroups on C(S
). Then, for each f C(S
),
lim
E[f(X
t
)] = lim
E[T
(t)f(X
0
)]
= lim
_
T
(t)f(x) (dx) =
_
T
(t)f(x) (dx).
Now f
_
T
(t)f(x) (dx) is a positive linear operator that maps the constant function 1 to the value
1, and thus, by the Riesz representation theorem is integration against a probability measure
t
. This gives
the one dimensional distributions of X. The proof that the nite dimensional distributions converge follows
from a straightforward induction argument.
To check the strong Markov property, let be a discrete T
X
t+
-stopping time, then for each > 0, and
A T
X
, A = t T
t+
. Thus, for s > 0,
E[f(X
+s
); A = t] = E[f(X
t+s
); A = t]
= E[T(s )f(X
t+
); A = t]
= E[T(s )f(X
+
); A = t].
Now use the right continuity of X and the strong continuity of T to conclude that
E[f(X
+s
); A = t] = E[T(s)f(X
); A = t]
Summing over the range of we obtain
E[f(X
+s
)[T
X
+
] = T(s)f(X
).
We can obtain the result for an arbitrary stopping time by realizing it as the decreasing sequence of
discrete stopping times, use the identity above, the right continuity of X, and the strong continuity of T.
Exercise 4.89. Complete the proof above by supplying the induction argument to show that the nite di-
mensional distributions converge.
In summary, we have
Theorem 4.90. Let S be locally compact and separable. Let G be a linear operator on C
0
(S) satisfying 1-3
of the Hille-Yosida theorem and let T be a strongly continuous postive contraction semigroup generated by
G. Then for each x S, there exists a Markov process X corresponding to T with initial distribution
x
and
with sample paths in D
S
[0, ) if and only if G is conservative.
4.8 Transformations of Markov Processes
Denition 4.91. A continuous additive functional A of a Markov process X is adapted process satisfying
1. A
0
= 0.
2. A is continuous and nondecreasing.
3. For all s, t 0, A
s+t
= A
s
+A
t

s
.
4. A is constant on [, ), = inft > 0; X
t
= .
The example that will occupy our attention in this section is A
t
=
_
t
0
a(X
u
) du for some nonnegative
measureable function a. We shall use these additive functionals in two settings: random time changes, and
killing.
Example 4.92. If N is one of the queues given in the examples, then
_
t
0
N
s
ds
is the total amount of service time for the queue up to time t.
_
t
0
I
0]
(N
s
) ds
is the amount of time that the queue is empty.
4.8.1 Random Time Changes
Let Y be a process with sample paths in D
S
[0, ) and choose c be a nonnegative continuous function on S
so that c Y is bounded on bounded time intervals. Think of c as the rate that the clock for the process
moves. Thus, for example, if c(x) > 1, the process moves more quickly through the state x
The goal is to describe solutions to
X
t
= Y
R
t
0
c(Xs) ds
If X solves this equation, then set
t
=
_
t
0
c(X
s
) ds =
_
t
0
c(Y
s
) ds.
Set
0
= inft > 0 :
_
t
0
1/c(X
s
) ds = , then for t <
0
,
t =
_
t
0
c(X
s
)
c(X
s
)
ds =
_
t
0
1
c(Y
u
)
du
and therefore, satises the ordinary dierential equation
t
t
= c(Y
t
),
0
= 0.
Because c(Y ) is bounded on bounded time intervals, the solution to the ordinary dierential equation
has a unique solution. Now,
X
t
= Y
t
.
Theorem 4.93. Let Y be a D
S
[0, )-valued Markov process with generator G on C
0
(S). If a unique solution
to
X
t
= Y
R
t
0
c(Xs) ds
exists, then X is a Markov process with generator cG.
Proof. Verifying that X is a Markov process is left as an exercise.
Note that
s
t =
_
t
0
1/c(X
u
) du) s
0
t T
Y
t+
. Thus,
s
is an T
t+
-stopping time. For
f T(A), the optional sampling theorem guarantees that
f(Y
t
)
_
t
0
Gf(Y
s
) ds = f(X
t
)
_
t
0
c(X
u
)Gf(X
u
) du
is an T
Y
t
-martingale.
Thus, f in in the domain of the generator of the Markov process X and this generator is cA.
Exercise 4.94. Verify that Y in the theorem above is a Marlov process.
4.8.2 Killing
We can choose to have a process X move to the state . The rate that this happens can be spatially
dependent. Thus, we dene the hazard function
k(x) = lim
h0
1
h
P
x
X
h
= .
The analysis follows the previous analysis for the case k constant. Write the additive functional
K
t
=
_
t
0
k(X
s
) ds
and dene
Y =
_

0
e
Ks
f(X
s
) ds, R
k
f(x) = E
x
Y.
Then
Y =
_
t
0
e
Ks
f(X
s
) ds +e
Kt
Y
t
Again, we have Doobs martingale
Z
k,f
t
= E[Y [T
X
t
] =
_
t
0
e
Ks
f(X
s
) ds +e
Kt
E[Y
t
[T
X
t
]
=
_
t
0
e
Ks
f(X
s
) ds +e
Kt
R
k
f(X
t
)
Exercise 4.95. Show that for f T(G), R
k
Gf = R
k
kf f or
R
k
(k G)f = f.
Now, let f = (k G)g, then the Doobs martingale above becomes
C
k,g
t
= e
Kt
g(X
t
) +
_
t
0
e
Ks
(k G)g(X
s
) ds
and the analog to Dynkins formula becomes
g(x) = E
x
[e
K
g(X
)] +E
x
[
_

0
e
Ks
(k G)g(X
s
) ds.
Let g satisfy Gg = kg with g = h on D, then
g(x) = E
x
[e
K
D
h(X
D
)].
Now dene the stochastic process

X as follows:
Let be an exponential random variable, parameter 1, independent of the process X. Then
X
t
=
_
X
t
if K
t
< ,
if K
t
.
Then, for f C
0
(S),
E
x
f(

X
t
) = E
x
[f(X
t
)I
Kt<]
] = E
x
[f(X
t
)E
x
[I
Kt<]
[T
X
t
]] = E
x
[e
Kt
f(X
t
)].
Exercise 4.96. For f C
0
(S), dene
T(t)f(x) = E
x
[e
Kt
f(X
t
)].
Then T is a positive continuous constraction semigroup with generator G k. Thus,

X is Markov process
with state space S
and generator Gk.

Corollary 4.97. (Feynman-Kac formula) Let X be a Feller process on S and generator G. Then the
equation
u
t
= Gu ku, u(0, x) = f(x)
has solution
u(t, x) = E
x
[e
Kt
f(X
t
)].
4.8.3 Change of Measure
Proposition 4.98 (Bayes formula). On the probability space (, T, P), let L be a non-negative random
variable satisfying EL = 1. Dene the probability measure Q(L) = E[L; A] and write E
Q
[Y ] = E[Y L]. Then
for any integrable random variable Z, and any -algebra ( T,
E
Q
[Z[(] =
E[ZL[(]
E[L[(]
.
Proof. Clearly the right hand side is (-measurable. Choose A (, then
E
Q
_
E[ZL[(]
E[L[(]
; A
_
= E
_
E[ZL[(]
E[L[(]
L; A
_
= E
_
E
_
E[ZL[(]
E[L[(]
L
(
_
; A
_
= E
_
E[ZL[(]
E[L[(]
E [L[(] ; A
_
= E [E[ZL[(]; A]
= E[ZL; A] = E
Q
Z.
Lemma 4.99. Assume Q[
Jt
<< P[
Jt
, and assume that L
t
is the Radon-Nikodym derivative, then Z is a
Q-martingale if and only if ZL is a P-martingale.
Proof. Because L is an adapted process, Z is adapted if and only if ZL is adapted. To show that L is a
martingale, choose A T
t
, then Q(A) = E[L
t
; A]. However, A T
t+s
and therefore
E[L
t
; A] = Q(A) = E[L
t+s
; A].
By Bayes formula,
E
Q
[Z
t+s
[T
t
] =
E[Z
t+s
L
t+s
[T
t
]
E[L
t+s
[T
t
=
E[Z
t+s
L
t+s
[T
t
]
L
t
.
Thus,
E
Q
[Z
t+s
[T
t
] Z
t
=
E[Z
t+s
L
t+s
[T
t
] Z
t
L
t
L
t
.
and, consequently,
E
Q
[Z
t+s
[T
t
] = Z
t
if and only if E[Z
t+s
L
t+s
[T
t
] Z
t
L
t
.
Note that L is a positive mean one martingale. In addition, is L
1
is Q-integrable, the L
1
is a Q-
martingale and P[
Jt
<< Q[
Jt
For X, a time homogeneous Markov process with generator G, we write g = Gf for f T(G), with
inf
x
f(x) > 0, recall that
f(X
t
)
f(X
0
)
exp
_
_
g(X
v
)
f(X
v
)
dv
_
is a mean 1 local martingale.. Write f(x) = exp w(x) and (w = e
w
Ge
w
, then this local martingale becomes
L
t
= exp
_
w(X
t
) w(X
0
)
_
t
0
(w(X
v
) dv
_
.
For the case of standard Brownian motion B, take w(x) = x. We shall soon learn that G =
1
2
, and
L
t
= exp(B
t

1
2
2
t)
is indeed a martingale.
Under the change of measure induced by L
QB
t
A = E[I
A
(B
t
) exp(B
t

1
2
2
t)] =
1
2t
_
A
exp(x
1
2
2
t)e
x
2
/2t
dx
=
1
2t
_
A
exp
1
2t
(x
2
2xt
2
t
2
) dx =
1
2t
_
A
exp
1
2t
(x t)
2
dx
=
1
2t
_
At
e
x
2
/2t
dx = PB
t
+t A
Exercise 4.100. Let B, P and Q be dened as above. For times 0 t
1
< t
2
t
n
, and Borel sets A
1
, . . . , A
n
,
QB
t1
A
1
, . . . B
tn
A
n
= PB
t1
+t
1
A
1
, . . . B
tn
+t
n
A
n
.
Thus, by the Daniell-Kolmogorov extension theorem, under the measure Q, B is Brownian motion with drift
.
Example 4.101. Recall that for standard Brownian motion, and for a 0, the hitting time for level a,
a
= inft > 0; B
t
> a, has density
f
a
(s) =
a
2t
3
e
a
2
/2s
.
If we consider the process B
t
+t, > 0 under the measure Q, then this process is standard Brownian motion
under the measure P. Thus,
Q
a
t = E[I
at]
L
t
] = E[E[I
at]
L
t
[T
mina,t]
]] = E[I
at]
L
mina,t]
] = E[I
at]
L
]
= E[I
at]
exp(a
1
2
a
)] =
_
t
0
exp(a
1
2
2
s)
a
2s
3
e
a
2
/2s
ds
Thus,
f
a,
(s) =
a
2s
3
e
(at)
2
/2s
.
gives the density of
a
for Brownian motion with constant drift.
4.9 Stationary Distributions
Denition 4.102. Call a stochastic process stationary if, for every n 1, and 0 s
1
< < s
n
, the nite
dimensional distributions
PX
t+s1
B
1
, . . . , X
t+sn
B
n
, B
1
, . . . , B
n
B(S).
are independent of t 0.
Denition 4.103. A probabiity measure for a Markov process X is called stationary if X is a stationary
process under P
.
Lemma 4.104. is a stationary measure for a Markov process X if and only if
P
X
t
B = (B), B B(S).
for all t 0.
Proof. The necessity is immediate. To prove sucency, note that for each t 0,
t
X is a Markov process
with initial distribution and the same generator as X, thus has the same nite dimensional distributions
as X.
Theorem 4.105. Let G generate a strongly continuous contraction semigroup T on C
0
(S) corresponding to
a Markov process X. In addition, assume that D is a core for G. Then for a probability measure on S the
following are equivalent:
1. is a stationary distribution for X.
2.
_
T(t)f d =
_
f d for f C
0
(S) and t 0.
3.
_
Gf d = 0 for f D.
Proof. (1 2)
_
T(t)f d = E
[T(t)f(X
0
)] = E
[f(X
t
)] = E
[f(X
0
)] =
_
f d
(2 1) By the lemma above, it suces to show that the one dimensional distributions agree.
E
[f(X
t
)] = E
[T(t)f(X
0
)] =
_
T(t)f d =
_
f d = E
[f(X
0
)].
(2 3) follows from the denition of G.
(3 2) By the denition of core, 3 holds for all f T(G).
_
(T(t)f f) d =
_ _
t
0
GT(s)f dsd =
_
t
0
_
GT(s)f dds = 0.
Because T(G) is dense in C
0
(S), 2 follows.
Exercise 4.106. Let be a probability measure on S
1. If
_
f d = lim
t
_
T(t)f d exists for every f C
0
(S), then is a stationary measure.
2. If
_
f d = lim
n
1
tn
_
tn
0
_
T(t)f d exists for every f C
0
(S), and some increasing sequence
t
n
; n 1, lim
n
t
n
= then is a stationary measure.
Denition 4.107. A Markov process X is said to be ergodic if there exist a unique stationary measure
and
lim
t
E
f(X
t
) =
_
f d.
for every initial distribution and every f C
0
(S).
Note that if X is a stationary process, then we may extend it to be a process X
t
; t R. As with
Markov chains, we have:
Denition 4.108. A probability measure is said to be reversible for a Markov process X if
E
[f
1
(X
0
)f
2
(X
t
)] = E
[f
2
(X
0
)f
1
(X
t
)],
or in terms of the semigroup T,
_
f
1
T(t)f
2
d =
_
f
1
T(t)f
2
d.
In other words, T(t) is a self adjoint operator with respect to the measure .
Proposition 4.109. If T is conservative, then a reversible measure is invariant.
Proof. Take f
2
= 1 in the denition.
Theorem 4.110. Suppose that T is a Feller semigroup on C
0
(S) with generator G and let be a probability
measure on S. Then the following statements are equivalent:
1. is reversible.
2.
_
f
1
Gf
2
d =
_
f
1
Gf
2
d for all f
1
, f
2
T(G).
3.
_
f
1
Gf
2
d =
_
f
1
Gf
2
d for all f
1
, f
2
D, a core for G.
4. is stationary. Let X be the Markov process obtained using the as the distribution of X
0
and
extending the time set to R. Then X
t
; t R and X
t
; t R have the same nite dimensional
distributions.
Proof. (1 2) is straightforward and (2 3) is immediate. (3 1) is nearly identitical to the proof of a
similar statement for stationary measures.
(4 1)
E
[f
1
(X
0
)f
2
(X
t
)] = E
[f
2
(X
0
)f
1
(X
t
)] = E
[f
2
(X
0
)f
1
(X
t
)].
(1 4) Now suppose that is reversible. Then given
f
0
= 1, f
1
, . . . , f
n
, f
n+1
= 1 C
b
(S)
and t
0
< < t
n
, dene
g
(x) = E
x
[
1
k=1
f
k
(X
t
t
k
)], and h
(x) = E
x
[
n+1
k=+1
f
k
(X
t
k
t
)].
Set
I
=
_
g
(x)f
(x)h
(x) (dx).
Then
I
0
= E
[
n
k=1
f
k
(X
t
k
t0
)], and I
n+1
= E
[
n
k=1
f
k
(X
tn+1t
k
)].
Claim. I
0
= I
n+1
.
Let 1 n, Then
I
=
_
g
d =
_
(g
)T(t
+1
t
)(f
+1
h
+1
) d
=
_
T(t
+1
t
)(g
)(f
+1
h
+1
) d
=
_
g
+1
f
+1
h
+1
d = I
+1
because
T(t
+1
t
)(f
+1
(x)h
+1
(x)) = T(t
+1
t
)
_
f
+1
(x)E
x
[
n+1
k=+2
f
k
(X
t
k
t
+1
)]
_
= T(t
+1
t
)E
x
[
n+1
k=+1
f
k
(X
t
k
t
+1
)] = E
x
[
n+1
k=+1
f
k
(X
t
k
t
)] = h
(x),
and
T(t
+1
t
)(g
(x)f
(x)) = T(t
+1
t
)
_
E
x
[
1
k=1
f
k
(X
t
t
k
)]f
(x)
_
= T(t
+1
t
)E
x
[
k=1
f
k
(X
t
t
k
)] = g
+1
(x).
Interate this to obtain the claim.
However, by stationarity,
I
0
= E
[
n
k=1
f
k
(X
t
k
)], and I
n+1
= E
[
n
k=1
f
k
(X
t
k
)].
and 4 follows.
For a countable state space, the identity
_
f
1
Gf
2
d =
_
f
2
Gf
1
d becomes
y
f
1
(x)g
xy
(f
2
(y) f
2
(x))x =
y
f
2
(x)g
xy
(f
1
(y) f
1
(x))x.
Simplifying, this is equivalent to
y
f
1
(x)g
xy
f
2
(y)x =
y
f
2
(x)g
xy
f
1
(y)x
or by changing indices
y
f
1
(x)g
xy
f
2
(y)x =
y
f
2
(y)g
yx
f
1
(x)y.
By writing f
1
and f
2
as sums of indicator functions of sites in S, we have that this statement is equivalent
to
xg
xy
= yg
yx
.
This is the equation of detailed balance.
Exercise 4.111. Find a reversible measure for a birth and death process and give conditions so that this
measure can be normalized to be a probability measure. Simplify this expression for the examples of birth and
death processes in the examples above.
4.10 One Dimensional Diusions.
Let r
< r
+
+ and let I = [r
, r
+
] R, int(I) = (r
, r
+
), and

I = [r
, r
+
].
Denition 4.112. A diusion on I is a Feller process with paths in C
I
[0, ). A diusion is called regular
if, for every x int(I) and y I,
P
x
y
< > 0,
where
y
= inft > 0 : X
t
= y.
Example 4.113. Brownian motion is a one dimensional diusion on R. The semigroup is
T(t)f(x) =
1
2t
_
R
f(y) exp(
(x y)
2
2t
) dy =
1
2
_
R
f(x +y
t) exp(
y
2
2
) dy.
Then, for f C
2
(R) C
0
(R),
1
t
(T(t)f(x) f(x)) =
1
2
_
R
1
t
((f(x +y
t) f(x)) exp(
y
2
2
) dy
=
1
2
_
R
1
t
(y
tf
t
(x) +
1
2
y
2
tf
tt
(x +y
t) exp(
y
2
2
) dy
=
1
2
_
R
1
2
y
2
f
tt
(x +y
t) exp(
y
2
2
) dy
for some (0, 1). Thus, the generator
Gf(x) =
1
2
f
tt
(x).
Exercise 4.114. For > 0, solve g
tt
= 2g and use this to nd the Laplace transform for exit times.
Exercise 4.115. Let C
2
(R) be increasing and let B be standard Brownian motion. Find the generator
of X = B.
The major simplifying circumstance for one-dimensional diusions is the intemediate value theorem, i.e.,
if x
0
is between X
s
and X
t
, s < t, then X
c
= x
0
for some c (s, t). Regularity implies a much stronger
condition.
Theorem 4.116. Let J be a nite open interval,

J I and let
J
be the time of the rst exit from J, then
if X is a regular diusion on I
sup
xJ
E
x
J
< .
Proof. We begin with a claim.
Claim. Assume that p
J
= sup
xJ
P
x
J
> t < 1, then
E
x
J

t
1 p
J
.
Write p
n
= sup
xJ
P
x
J
> nt. Then
P
x
J
> (n + 1)t p
n
P
x
J
> (n + 1)t[
J
> nt p
n
p
J
.
by the strong Markov property. Thus, p
n
p
n
J
and
E
x
J
=
_

0
P
x
J
> s ds =
n=0
_
(n+1)t
nt
P
x
J
> s ds t
n=0
p
n
J
=
t
1 p
J
.
To complete the proof, set J = (a, b) and pick y J. Then, by the regularity of X, there exists a t > 0
and p < 1 so that
P
y
a
> t p, P
y
b
> t p.
Thus, for x [y, b),
P
x
J
> t P
x
b
> t P
y
b
> t.
For x (a, y],
P
x
J
> t P
x
a
> t P
y
a
> t.
To set notation, for any open interval J = (a, b) with P
x
J
< = 1 for all x J, write
s
J
(x) = P
x
b
<
a
.
Clearly, s
J
(a) = 1 and s
J
(b) = 0. If a < x < y < b, then by the strong Markov property,
s
J
(x) = P
x
y
<
a
s
J
(y).
Lemma 4.117. s
J
is continuous and strictly increasing.
Proof. By the statements above, the theorem is equivalent to showing that for a < x < y < b, P
x
y
<
a
is continuous, strictly increasing as a function of y and has limit 1 as x y. (This give right continuity.
Left continuity follows from a symmetric argument.)
To this end,
P
x
y
<
a
= P
x
sup
ta
X
t
y P
x
sup
ta
X
t
> x = 1
because
a
> 0 a.s. P
x
We now show that P
x
y
<
a
= 1 contradicts the regularity of X. Dene stopping times
0
= 0,
n+1
= inft >
n
: X
t
= x = (
n
, x),
n+1
= inft >
n+1
; X
t
a, y. By the strong Markov
property at
n
,
P
y
n
< , X
n
= a = E
y
[P
x
a
<
y
;
n
< ] = 0
and thus,
P
x
a
< = 0.
4.10.1 The Scale Function
Denition 4.118. Let X be a regular diusion on I. A scale function for X is a function s : I R such
that for all a < x < b, a, b I,
P
x
b
<
a
=
s(x) s(a)
s(b) s(a)
.
If s(x) = x is a scale function for X, then we say that X is in its natural scale. In particular, if
x = (a +b)/2, then P
x
b
<
a
= 1/2.
Exercise 4.119. 1. Standard Brownian motion is on natural scale.
2. s is uniquely determined up to an increasing ane transformation.
3. A process X is on natural scale if and only if P
x
b
<
a
= 1/2 for every a, b I and x = (a +b)/2.
Theorem 4.120. Every regular diusion on I has a scale function.
Proof. If I is compact, then s
I
is a scale function. To see this, take I = [a, b] and a < c < x < d < b. Then,
by the strong Markov property
P
x
a
<
b
= P
x
d
<
c
P
d
a
<
b
+P
x
c
<
d
P
c
a
<
b
s
I
(x) s
i
(a)
s
I
(b) s
I
(a)
= P
x
d
<
c
s
I
(d) s
i
(a)
s
I
(b) s
I
(a)
+ (1 P
x
c
<
d
)
s
I
(c) s
i
(a)
s
I
(b) s
I
(a)
.
Solving, we obtain,
P
x
d
<
c
=
s
I
(x) s
I
(c)
s
I
(d) s
(
c)
.
Now, we extend. Take J
n
; n 1 to be an increasing sequence of compact intervals with union I.
Moreover, if I contains an endpoint, then construct each J
n
= [a
n
, b
n
] to include this endpoint. Dene
s
n
: J
n
R inductively. s
1
= s
J1
, and, for n 1,
s
n+1
= s
n
(a
n
) + ( s
n
(b
n
) s
n
(a
n
))
s
Jn+1
(x) s
Jn+1
(a
n
)
s
Jn+1
(b
n
) s
Jn+1
(a
n
)
.
Clearly, s
n+1
(a
n
) = s
n
(a
n
), and s
n+1
(b
n
) = s
n
(b
n
). Repeat the argument above to show that s
n+1
is
a scale function on J
n
. Because s
n
is an ane transformation of s
n+1
and the two functions agree at two
points, they must be equal on J
n
.
Consequently, we can dene, for x J
n
, s = s
n
.
Theorem 4.121. If X is a regular diusion on I, then

X = s(X) is a regular diusion in natural scale on
s(I).
Proof. Write a = s(a),

b = s(b), and
a
= inft > 0 :

X
t
= a
P
x

a
<
b
= P
x
a
<
b
=
s(x) s(a)
s(b) s(a)
=
x a
b a
.
4.10.2 The Speed Measure
The structure threorem for jump Markov processes gives to each site x in a birth and death process, and
function (x), that determines the length of time at x and a probability p
x
= P
x
X
1
= x + 1, the bias in
direction.
For a one dimensional regular diusion, the bias in direction is articulated through the scale function s.
The length of time near x is described by E
x
J
. In this section we shall show how these mean values can be
represented as integration of a kernel against a measure. To explain the kernel we have the following:
Exercise 4.122. For a simple symmetric random walk Y on Z, and let = minn; Y
n
M
, M
+
,
M
< M
+
and let N
y
be the number of visits to y before . Then
E
x
= (x M
)(M
+
x),
and
E
x
N
y
=
_
(xM)(M+y)
M+M
, x y
(yM)(M+x)
M+M
, x y.
For a bounded open interval J = (a, b) dene, for x, y J
G
J
(x, y) =
_
(xa)(by)
ba
x y,
(ya)(bx)
ba
x y.
Theorem 4.123. Let X be a one dimensional diusion on I on natural scale. Then there is a unique
measure m dened on B(int(I)) such that
1. m is nite on bounded sets, B,

B int(I).
2. For a bounded open interval J,
m(x, J) = E
x
J
=
_
J
G
J
(x, y)m(dy).
Proof. Let a = x
0
< x
1
< < x
n
= b be a partition [a, b] into subintervals of length h = (b a)/n.
Thus, x
k
= a +hk Dene J
k
= (x
k1
, x
k+1
).
The process restricted to the exit times of the intervals J
k
is a simple symmetric random walk Y on
(b a) +hZ. Let
j
denote the time between steps before
j
and zero after. Then for a partition point x,
E
x
J
=
j=0
E
x
j
.
and
E
x
j
= E
x
[E
x
[
j
[Y
j
]] =
n1
k=1
P
x
Y
j
= x
k
m(x
k
, J
k
).
Let n(x
k0
, x
k
) denote the expected number of visits of the random walk to x
k
starting at x
k0
. Then, by the
exercise,
n(x
k0
, x
k
) =
_
k0(nk)
n
x
k0
x
k
k(nk0)
n
x
k
x
k0
=
1
h
G(x
k0
, x
k
).
Thus,
E
x
J
=
j=0
n1
k=1
P
x
Y
j
= x
k
m(x
k
, J
k
) =
n1
k=1
(
j=0
P
x
Y
j
= x
k
)m(x
k
, J
k
)
=
n1
k=1
n(x, x
k
)m(x
k
, J
k
) =
n1
k=1
1
h
G
J
(x, x
k
)m(x, J
k
)
Dene the measure m
n
that gives mass m(x
k
, J
k
)/h to the point x
k
. Then the formula above becomes,
m(x
j
, J) =
_
J
G
J
(x
j
, y) m
n
(dy).
Now consider succesive renements, (say n = 2
). Then the representation above holds whenever the

ratio (xa)/(ba) is a dyadic rational. This represenation above also implies that limsup
n
m
n
(J) < .
Now use a variant of the Prohorov theorem to show that m
2
: 1 has compact closure in the topology
determined by weak convergence. Let m be a limit point for the m
2
: 1, then for any nite open
interval J with dyadic rational endpoints,

J I,
m(x, J) =
_
J
G
J
(x, y) m(dy),
x a
b a
a dyadic rational.
By the bounded convergence theorem, this can be extended to all such open intervals J.
To extend this to arbitrary y, use the strong Markov property to obtain the identity for ( a,
b) =

J J,
m(x, J) = m(x,

J
n
) +P
x
a
<
b
m(
b, J) +P
x
a
>
b
m( a, J).
Now let a
n
and

b
n
be dyadic rationals a
n
< x <

b
n
, then
lim
n
m( a, J) = lim
n
m(
b, J) = m(x, J)
and by the representation for dyadic rationals,
lim
n
m(x,

J) = 0.
Exercise 4.124. The measure m is unique.
Denition 4.125. The measure m is called the speed measure for the process.
Exercise 4.126. 1. If X is not on its natural scale, then the speed measure m can be dened by
m(x, J) =
_
J
G
J
(x, y) m(dy)
with
G
J
(x, y) =
_
(s(x)s(a))(s(b)s(y))
s(b)s(a))
x y
(s(y)s(a))(s(b)s(x))
s(b)s(a))
x y
2. Let X be standard Brownian motion, then for J = (a, b),
E
x
J
= (x a)(x b).
Thus, the speed measure is Lebesgue measure.
3. Conversely, if the speed measure is Lebesgue measure, then for J = (a, b),
E
x
J
= (x a)(x b).
4. Use the standard machine to prove that for a bounded measurable function,
E
x
[
_

J
0
f(X
t
) dt] =
_
J
f(y)G
J
(x, y) m(dy)
Theorem 4.127. Let X be a regular diusion on natural scale with speed measure m and let

X be the unique
solution to
X
t
= X
R
t
0
c(

Xs) ds
.
Then, X is a Feller process on natural scale with speed measure
m(A) =
_
A
c(x)
1
m(dx).
Proof. Set
t
=
_
t
0
c(

X
s
) ds.
Then, as before
t =
_
t
0
c(

X
s
) ds.
Denote
a
= inft > 0 : X
t
= a
a
= inft > 0 :

X
t
= a.
Then
P
x

a
<
b
= P
x
a
<
b
.
Thus,

X is on natural scale.
Let
J
be the rst exit time of

X from J. Then
J
=

J
or
J
=
_

J
0
1
c(X
s
)
ds.
Taking expectations, we obtain
E
x

J
= E
x
[
_

J
0
c(X
s
)
1
ds] =
_
J
G
J
(x, y)c(y)
1
m(dy).
Thus,

X has the asserted speed measure.
4.10.3 The Characteristic Operator
Because we are working with exit times in the denition of the speed measure, it is convenient to continue
this idea to give an operator similar to the generator.
Denition 4.128. Let x int(I), then
lim
Jx]
(J) = L
means that for any system of open sets J
n
; n 1 decreasing to the point x, we have
lim
n
(J
n
) = L.
Denition 4.129. Let X be a regular diusion on I. The characteristic operator
Cf(x) = lim
Jx]
E
x
f(X
J
) f(x)
E
x
J
.
The domain of C is the subspace of all f C
0
(I) for which the limit above exists.
Theorem 4.130. Let X be a regular diusion on I with generator A. Let f T(A) T(C), then Af(x) =
Cf(x) for all x int(I).
Proof. Choose f T(A), and x int(I). Set g = Af. Then Dynkins formula states that
E
x
[f(X
J
)] = f(x) +E[
_

J
0
g(X
s
) ds].
for an interval J,

J int(I). Choose > 0 so that g(J) (g(x) , g(x) +). Then,
g(x) <
E
x
f(X
J
) f(x)
E
x
J
< g(x) +
and Cf(x) = Af(x).
Theorem 4.131. If X is a regular diusion on the line on natural scale and with speed measure equal to
Lebesgue measure, the X is standard Brownian motion.
Proof. Let A be the generator for X and let C be its characteristic operator. Then, because R is open,
T(A) T(C). Write an interval containing x as J(, ) = (x +, x (1 )), (0, 1). Then,
E
x
J(,)
= (1 )
2
.
For f C
2
(R),
Ef(X
J(,)
) = (1 )f(x +) +f(x (1 )).
and
Ef(X
J(,)
) f(x) = (1 )(f(x +) f(x)) +(f(x (1 )) f(x)).
Now, choose sequences
n
0 and
n
(0, 1). Then, by LHopitals rule
lim
n
E
x
f(X
J
(n,n)
) f(x)
E
x
J
(
n
,
n
)
= lim
n
(1
n
)(f(x +
n
n
) f(x)) +
n
(f(x (1
n
)
n
) f(x))
n
(1
n
)
2
n
= lim
n
f
t
(x +
n
n
) +f(x (1
n
)
n
)
2
n
=
1
2
f
tt
(x).
Set aside for the moment behavior in the boundary of I. Beginning with any scale s and any speed
measure having a density c, we can transform it to Brownian by a change of scale and a time change. Thus,
by inverting these transformations, we can transform a Brownian motion into a regular diusion having scale
and speed measure
_
A
c(x) dx. This shows, among this class of scales and speed measure, that the behavior
of a regular diusion is uniquely determined in the interior of I by these two properties of the diusion.
For diusions X in this class, the generator takes the form
A =
1
2
a(x)
d
2
dx
2
+b(x)
d
dx
, a(x) > 0.
Let x
0
I and dene
s(x) =
_
x
x0
exp
_
2
_
y
a
b(z)
a(z)
2
dz
_
dy
Then s is a harmonic function
For J = (a, b) int(I), we have by Dynkins formula,
s(x) = s(a)PX
J
= a +s(b)PX
J
= b.
Therefore, s is a scale function.
Now, use the chain rule to see that s(X) has generator
1
2
a(x)s
t
(x)
2
d
2
dx
2
.
Thus, the speed measure has density 1/(a(x)s
t
(x)
2
).
Example 4.132. 1. The radial part of an n-dimensional Brownian motion R is called a Bessel process.
In this case, I = [0, ) and
A =
1
2
d
2
dr
2
+
n 1
2r
d
dr
.
2. The Ornstein-Uhlenbeck process is obtained by subjecting particles undergoing physical Brownian mo-
tion to an elastic force. The generator is
A =

2
2
d
2
dx
2
x
d
dx
, I = R.
3. Financial models of stock prices use geometric Brownian motion. Here, I = (0, ) and the generator
is
A =

2
2
x
d
2
dx
2
+x
d
dx
.
4. A diusion approximation of a neutral 2 allele model with scaled mutation rates
0
and
1
is
A =
1
2
x(1 x)
d
2
dx
2
+ (
1
x
0
(1 x))
d
dx
, I = [0, 1].
Exercise 4.133. 1. Find a scale function and the speed measure for the four examples above.
2. Show that a normal distribution is stationary for the Ornstein-Uhlenbeck process.
3. Show that a beta distribution is stationary for the diusion in example 4.
4.10.4 Boundary Behavior
We now have a good understanding of the behavior of a regular diusion on int(I). We conclude by examining
its behavior at the boundary.
Denition 4.134. Let I = [r
, r
+
] R. The r
+
is called an accessible is there exists t > 0 and x int(I)
such that
inf
y(x,r+)]
P
x
y
t > 0.
If r
+
is inaccessible, call it natural if there exists t > 0 and y int(I) such that
inf
y(x,r+)]
P
x
y
t > 0.
Call it entrance if for all t > 0 and y int(I) such that
lim
xr+
P
x
y
t = 0.
An accessible boundary may or may not be regular. If it is not, then call the boundary point an exit.
Corresponding statements hold for r
In schematic form, we have

r
+
int(I) r
+
, int(I)
int(I) r
+
regular exit
int(I) , r
+
entrance natural
Theorem 4.135. Fix x
0
I and dene
u(x) =
_
x
x0
s(y) m(dy).
Then r
+
is accessible if and only if u(r
+
) <
Proof. Let J = (x
0
, r
+
). By the lemma, for y J
> E
y
J
=
_
r+
x0
G
J
(y, z) m(dz)
s(r
+
) s(y)
s(r
+
) s(x
0
)
_
r+
y
(s(z) s(x
0
)) m(dz)
s(r
+
) s(y)
s(r
+
) s(x
0
)
(u(r
+
) u(y) s(x
0
)m(y, r
+
)) .
Because m(x, y) < and u(y) < , we have that u(r
+
) < .
Conversely, assume that u(r
+
) < and choose x (x
)
, r
+
). Then,
E
x
x0
= lim
yr+
m(x, (x
0
, y)) =
_
y
x0
G
(x0,y)
(x, s) m(dz).
Note that,
E
x
x0
is nondecreasing as x incresing.
and that
_
y
x0
G
(x0,y)
(x, s) m(dz) =
2(s(r
+
) s(x)
2(s(r
+
) s(x
0
)
_
x
x0
(s(z)s(x
0
)) m(dz)+
2(s(c) s(x
0
)
2(s(r
+
) s(x
0
)
_
x
x0
(s(r
0
)s(z)) m(dz).
This tends to zero as x r
+
. Thus,
lim
xr+
E
x
x0
= 0
and r
+
is accessible.
5 STOCHASTIC INTEGRALS 84
5 Stochastic Integrals
In this section we develop the theory of stochastic integrals with the primary intent to apply them to the
study of a class of Markov processes having continuous sample paths.
The theory of Riemann-Steitjes integral applies readily to continuous functions having nite variation on
compact intervals. Thus, we can readily obtain a stochastic integral for
(X V )
t
() =
_
t
0
X
s
() dV
s
().
for
1. progressively measurable continuous V having nite variation on compact intervals, and
2. progressively measurable continuous X bounded on compact intervals.
To extend this to continuous martingales, we must take into account the following.
Exercise 5.1. Let M be a martingale with EM
2
t
< for all t > 0. Then, for s, t > 0,
Var(M
s+t
[T
t
) = E[M
2
s+t
[T
t
] M
2
t
= E[(M
s+t
M
t
)
2
[T
t
].
Thus
E[(M
s+t
M
t
)
2
] = E[M
2
s+t
M
2
t
].
Denition 5.2. A partition of [0, T] is a strictly increasing sequence of real numbers 0 = t
0
< t
1
< <
t
n
= T. A partition of [0, ) is a strictly increasing unbounded sequence of real numbers 0 = t
0
< t
1
<
< t
k
, lim
k
t
k
= . The mesh() = supt
k+1
t
k
; k 0
For any process X and partition = t
0
< t
1
< < t
k
of [0, ), dene the simple process
X
t
=
k=0
X
t
k
I
[t
k
,t
k+1
)
(t).
Theorem 5.3. A continuous martingale having nite variation on compact intervals is constant.
Proof. By considering the martingale

M
t
= M
t
M
0
, we can assume that M
0
= 0. In addition, let
c
be
the time that the variation of M reaches C. Then by considering the martingale M
c
, we can assume that
M has variation bounded above by c.
Now, let = 0 = t
0
< t
1
< < t
n1
< t
n
= t, be a partitions of [0, t], then
EM
2
t
=
n
j=1
E[M
2
tj+1
M
2
j
] =
n
j=1
E[(M
tj+1
M
tj
)
2
] nE[ max
1jk
[M
tj+1
M
tj
[].
The random variable max
1jk
[M
tj+1
M
tj
[ is bounded above by c and converges to 0 almost surely as the
mesh of the partition tends to 0. Thus,
EM
2
t
= 0 and M
t
= 0 a.s.
5.1 Quadratic Variation
Denition 5.4. For the partition , 0 = t
0
< t
1
< < t
k
, of [0, ), the quadratic variation process
of a stochastic process X along is
Q
t
(X) =
j=1
(M
mint,tj+1]
M
mint,tj]
)
2
.
We say that X has nite quadratic variation if there exists a process X, X) such that
Q
t
(X)
P
X, X)
t
as mesh() 0.
Theorem 5.5. A continuous bounded martingale M has nite quadratic variation M, M)
t
that is the unique
continuous increasing adapted process vanishing at zero such that
M
2
t
M, M)
t
is a martingale.
Proof. The uniquess of M, M) is easy, let A and B be two such processes. Then,
A
t
B
t
= (M
2
t
B
t
) (M
2
t
A
t
)
as the dierence of two martingales is itself a martingale. Because it has nite variation on compact intervals,
it is constant.
For existence, we will prove:
Theorem 5.6. If
n
; n 1 is a sequence of partitions of [0, ) with mesh(
n
) 0 then
lim
n
E[ sup
0tT
(Q
n
t
(M) M, M)
t
)] = 0.
Exercise 5.7. Let B denote standard one-dimensional Brownian motion. With the partitions
n
described
as above, show that Q
n
t
(B)
L
2
t as n .
For this, we will introduce a second sequence I
n
of process, which we shall later recognize as the approx-
imation to the stochastic integral
_
t
0
M
s
dM
s
. Write
n
= 0 = t
n
0
< t
n
1
< < t
n
k
, then
I
n
t
=
k=0
M
mint,t
n
k
]
(M
mint,t
n
k+1
]
M
mint,t
n
k
]
).
Exercise 5.8. 1.
M
2
t
M
2
0
= Q
n
t
(M) + 2I
n
t
. (5.1)
2. I
n
is a mean zero martingale. Consequently, M
2
t
Q
n
t
(M) is a mean-zero martingale.
Lemma 5.9. Choose c sup
t0
[M
t
[, then
E[(I
n
t
)
2
] c
4
.
Proof. The quadratic variation of I
n
associated with
n
Q
n
t
(I
n
) =
k=0
M
2
mint,t
n
k
]
(M
mint,t
n
k+1
]
M
mint,t
n
k
]
)
2
c
2
Q
n
t
(M).
Thus,
E[Q
n
t
(I
n
)] c
2
E[Q
n
t
(M)] = c
2
E[M
2
t
M
2
0
] c
4
.
Lemma 5.10. With c sup
t0
[M
t
[,
lim
m,n
E[ sup
0tT
(I
n
t
I
m
t
)
2
] = 0. (5.2)
Proof. Write
n

m
= u
0
< u
1
< < u
k
.
Then
I
n
t
I
m
t
=
k=1
(M
n
mint,u
k
]
M
m
mint,u
k
]
)(M
mint,u
k+1
]
M
mint,u
k
]
)
is a mean zero martingale, and therefore, (I
n
t
I
m
t
)
2
Q
mm
t
(I
n
I
m
) is, by the exercise, a mean
zero martingale.. Thus,
Q
mm
t
(I
n
I
m
) =
k=1
(M
n
mint,u
k
]
M
m
mint,u
k
]
)
2
(M
mint,u
k+1
]
M
mint,u
k
]
)
2
sup
0st
(M
n
s
M
m
s
)
2
Q
mm
t
(M).
By Doobs inequality, and then by Schwartzs inequality, we have,
E[ sup
0tT
(I
n
t
I
m
t
)
2
] 4E[(I
n
T
I
m
T
)
2
] = 4E[Q
nm
T
(I
n
I
m
)]
4E[ sup
0tT
(M
n
t
M
m
t
)
4
]
1/2
E[Q
mm
T
(M)
2
]
1/2
.
In addition,
E[Q
mm
T
(M)
2
] 2(E[(M
2
T
M
2
0
)
2
] + 4E[(I
mn
T
)
2
]) 12c
4
.
Finally, sup
0tT
(M
n
t
M
m
t
)
4
is bounded and converges to zero almost surely as m, n . Thus, by
the bounded converges theorem, its expectation converges to zero and (5.2) holds.
Returning to the proof of the theorem.
Proof. The continuous processes I
n
are Cauchy in the norm [[XY [[
2,T
= E[sup
0tT
(X
t
Y
t
)
2
]
1/2
, thus,
by (5.1). so is Q
n
(M). Because the space is complete and the convergences is uniform almost surely, the
limit Q(M) is continuous. Choose, if necessary, a subsequence that converges almost surely.
Write

n
=
n
k=1
n
. Then for any s, t

n
, s < t
Q
n
t
(M) Q
n
s
(M).
Consequently, Q(M) is nondecreasing on the dense set
k=1
n
. Because Q(M) is continuous, it is nonde-
creasing. Finally, that fact that M
2
t
Q
t
(M) is a martingale can be seen by taking limits on the expectations
E[M
2
t
Q
n
t
(M); A] = E[M
2
s
Q
n
s
(M); A], A T
s
, s < t.
To extend this theorem to the broader class of local martingales.
Proposition 5.11. Let M be a bounded continuous martingale and let be stopping time. Then
M
, M
)
t
= M, M)
t
.
Proof. By the optional sampling theorem,
(M
)
2
M, M)
.
is a martingale. Now use the uniqueness of the quadratic variations process.
Theorem 5.12. A continuous local martingale M has associated to it M, M)
t
the unique continuous in-
creasing adapted process vanishing at zero so that
M
2
t
M, M)
t
is a local martingale.
In addition, for any T > 0 and any sequence of partitions
n
; n 1 of [0, ) with mesh(
n
) 0 then
sup
0tT
[Q
n
t
(M) M, M)
t
[
P
0
as n .
Proof. Choose
m
; m 1 be a reducing sequence for M so that M
m
I
m>0]
is bounded. Let Q
m
(M)
denote the unique continuously increasing process, Q
m
0
(M) = 0 so that
(M
m
)
2
I
m>0]
Q
m
(M)
is a bounded martingale. By the uniqueness property and the proposition above,
(Q
m+1
(M))
m
= Q
m
(M)
on
m
> 0. This uniquely denes M, M) on [0,
m
].
For the second statement, let , > 0. Choose m
0
so that
P
m0
> T < .
Then, on the random interval [0,
m0
],
Q
n
t
(M
m
0
) = Q
n
t
(M) and M
m
0
, M
m
0
)
t
= M, M)
t
.
Consequently,
P sup
0tT
[Q
n
t
(M) M, M)
t
[ > +P sup
0tT
[Q
n
t
(M
m
0
) M
m
0
, M
m
0
)
t
[ > .
The last term tends to 0 as mesh(
n
) 0.
For any quadratic form, we have a polarization identity to create a bilinear form. That is the motivation
behind the next theorem.
Theorem 5.13. Let M and N be two continuous local martingales, then there is a unique continuous adapted
process M, N) of bounded variation vanishing at zero so that
M
t
N
t
M, N)
t
In addition, set, for the partition for 0 = t
0
< t
1
< < t
k
, lim
k
t
k
= .
C
t
(M, N) =
j=1
(M
mint,tj+1]
M
mint,tj]
)(N
mint,tj+1]
N
mint,tj]
).
Then, for any t > 0 and any sequence of partitions
n
; n 1 of [0, ) with mesh(
n
) 0 then
sup
0tT
(C
n
t
(M, N) M, N)
t
)
P
0
as n .
Proof. Set
M, N)
t
=
1
4
(M +N, M +N)
t
) M N, M N)
t
)
and use an appropriate reducing sequence. Uniqueness follows are before.
Denition 5.14. We will call M, N) the bracket of M and N and M, M) the increasing process associated
with M
Exercise 5.15. 1. For a stopping time , M
, N
) = M, N
) = M, N)
.
2. M, M) = 0 if and only if M
t
= M
0
a.s. for every t.
3. The intervals of constancy of M and M, M) are the some. (Hint: For constancy on [a, b], consider
the martingale
a
M
mint,ba]
.)
4. The brackets process is positive denite, symmetric and bilinear.
5. If M and N are independent, then M, N) is zero.
For any positive denite, symmetric and bilinear form, we have
Proposition 5.16 (Cauchy-Schwartz). Let M and N be continuous local martingales. Then
[M, N)
t
M, N)
s
[ (M, M)
t
M, M)
s
)
1/2
(N, N)
t
N, N)
s
)
1/2
.
Soon, we shall develop the stochastic integral. We will begin with a class of simple functions and extend
using a Hilbert space isometry. The Kunita-Watanabe inequalities are central to this extension.
Proposition 5.17. Let M and N be local martingales and let H and K be measurable processes, then
[
_
t
0
H
s
K
s
dM, N)
s
[
__
t
0
H
2
s
dM, M)
s
_
1/2
__
t
0
K
2
s
dN, N)
s
_
1/2
.
Proof. By the standard machine, it suces to consider H and K of the form
K = K
0
I
0]
+
n
j=1
K
j
I
(tj1,tj]
, H = H
0
I
0]
+
n
j=1
H
j
I
(tj1,tj]
with 0 = t
0
< < t
n1
< t
n
= t..
Use the Cauchy-Schwartz inequality for sums.
[
_
t
0
H
s
K
s
dM, N)
s
[
n
j=1
[H
j
K
j
[[M, N)
tj+1
M, N)
t
]
[
j=1
[H
j
K
j
[
_
M, M)
tj+1
M, M)
t
]
_
1/2
_
N, N)
tj+1
N, N)
t
]
_
1/2
_
_
n
j=1
H
2
j
(M, M)
tj+1
M, M)
s
)
_
_
1/2
_
_
n
j=1
K
2
j
(N, N)
tj+1
N, N)
t
]
)
_
_
1/2
=
__
t
0
H
2
s
dM, M)
s
_
1/2
__
t
0
K
2
s
dN, N)
s
_
1/2
.
Applying the Holder inequality yields the following.
Corollary 5.18 (Kunita-Watanabe inequality). Let p and q be conjugate exponents. Then
E
_
_
t
0
H
s
K
s
dM, N)
__
t
0
H
2
s
dM, M)
_
1/2
L
p
__
t
0
K
2
s
dN, N)
_
1/2
L
q
.
We now introduce the class of processes that will become the stochastic integraters.
Denition 5.19. An continuous T
t
-semimartingale is a continuous process which can be written
X = M +V (5.3)
where M is a continuous local martingale and V is a continuous adapted process having nite variation on
a sequence of stopping time
n
; n 1, lim
n
n
= bounded time intervals.
Exercise 5.20. 1. Show that V has zero quadratic variation.
2. Using the notation above show that M, V ) = lim
mesh(n)0
C
n
(M, V ) = 0.
3. If V
0
= 0, show that the representation in (5.3) is unique.
Thus, if X + V and Y + W are semimartingales using the representation above with M and N local
martingales and V and W processes having bounded variation on bounded intervals, then
X, Y ) = M, N).
5.2 Denition of the Stochastic Integral
Stochastic integration will begin proceed using the standard machine. The extension of the stochastic integral
in the limiting process will be based on a Hilbert space isometry. Here are the spaces of interest.
Denition 5.21. 1. Dene H
2
to be the space of L
2
-bounded martingales. These are martingales M,
such that
sup
t0
EM
2
t
< .
Use H
2
0
for those elements M H
2
with M
0
= 0.
2. For M H
2
dene L
2
(M) to be the space of progressively measurable processes K such that
[[K[[
2
M
= E[K
2
M, M)] = E[
_

0
K
2
s
dM, M)
s
] < .
If M H
2
, then M is a uniformly integrable martingale, the its limit exists and is in L
2
(P). Thus we
can place the following norm
[[M[[
2
1
2 = EM
2
.
Proposition 5.22. A continuous local martingale M is in H
2
0
if and only if EM, M)
< . In this case,

M
2
M, M) is a uniformly integrable martingale.
Proof. Let
n
; n 1 be a sequence that reduces M. Then,
E[M
2
mint,n]
I
n>0]
] E[M, M)
mint,n]
I
n>0]
] = E[M
2
0
I
n>0]
].
Because M
L
2
(P), we can use the dominated and monotone convergence theorem to conclude that
E[M
2
] E[M, M)
] = E[M
2
0
]
and EM, M)
< .
Conversely, by the same equality,
E[M
2
mint,n]
I
n>0]
] E[M, M)
] +E[M
2
0
] C.
By Fatous lemma,
EM
2
t
liminf
n
E[M
2
mint,n]
I
n>0]
] C
Thus, M
mint,n]
I
n>0]
; n 1, t 0 is uniformly integrable. Take s < t and A T
s
, then
E[M
t
; A] = lim
n
E[M
mint,n]
I
n>0]
; A] = lim
n
E[M
mins,n]
I
n>0]
; A] = E[M
s
; A]
and M is a martingale.
Finally, note that
sup
t0
[M
2
t
M, M)
t
[ (M
2
)
+M, M)
,
an integrable random variable and thus M
2
t
M, M)
t
; t 0 is uniformly integrable.
Note that [[M[[
1
2 = EM, M)
A simple process has the form,

K = K
0
I
0]
+
n
j=1
K
j
I
(tj1,tj]
.
To be progressive, we must have that K
j
is T
tj1
-measurable. The stochastic integral
(K M) =
_
K
s
dM
s
=
n
j=1
K
j
(M
tj1
M
tj
).
Note that
E(K M)
2
= E[
n
j=1
n
k=1
K
j
(M
tj
M
tj1
)K
k
(M
t
k
M
t
k1
)].
If j < k,
E[K
j
(M
tj
M
tj1
)K
k
(M
t
k
M
t
k1
)] = E[K
j
(M
tj
M
tj1
)K
k
E[M
t
k
M
t
k1
[T
t
k1
]] = 0.
Thus,
E(K M)
2
= E[
n
j=1
K
2
j
(M
tj
M
tj1
)
2
] = E[
n
j=1
K
2
j
E[(M
tj
M
tj1
)
2
[T
tj1
]]
= E[
n
j=1
K
2
j
E[(M
2
tj
M
2
tj1
[T
tj1
]] = E[
n
j=1
K
2
j
E[M, M)
tj
M, M)
tj1
[T
tj1
]]
= E[
n
j=1
K
2
j
(M, M)
tj
M, M)
tj1
)] = E[
_

0
K
2
s
dM, M)
s
] = [[K[[
2
M
Thus, the mapping
K K M
is a Hilbert space isometry from L
2
to L
2
(M).
Exercise 5.23. Let N and M be square integrable martingales and let H and K be simple adapted functions,
then
E[(K M)(H N)] = E[(KH M, N))].
Theorem 5.24. Let M H
2
0
, then for each K L
2
(M), there is a unique element K M H
2
0
, such that
K M, N) = K M, N).
Proof. To verify uniqueness, note that if M
1
and M
2
are two martingales in H
2
0
, such that
M
1
, N) = M
2
, N) for every N H
2
0
,
Then,
M
1
M
2
, M
1
M
2
) = 0.
and M
1
M
2
is constant, hence 0.
By the Kunita-Watanabe inequality,
[E[
_

0
K
s
dM, N)
s
][ [[N[[
1
2[[K[[
1
2
(M)
.
Thus the map
N E[K M, N)]
is a continuous linear functional on H
2
. Consequently, there exists an element K M H
2
so that
E[(K M)
] = E[(K M, N))
].
Claim. (K M)N K M, N) is a martingale.
Recalling that element in H
2
0
are uniformly integrable. Thus, for any stopping time , we have
E[(K M)
] = E[E[(K M)
[T
]N
] = E[(K M)
]
= E[(K M)
] = E[(K M, N
))
]
= E[(K M, N)
] = E[(K M, N))
]
By the uniqueness of the covariation process, we have that K M, N) = K M, N)
This also shows that
[[K M[[
2
1
2 = E[(K M)
2
] = E[(K
2
M, N))
] = [[K[[
1
2
(M)
and thus
K K M
is an isometry.
Denition 5.25. The martingale K M is called the Ito stochastic integral of K with respect to M. It is
often also denoted by
(K M)
t
=
_
t
0
K
s
dM
s
.
From out knowledge of Riemann-Stieltjes integrals, we have that
J
t
=
_
t
0
K
s
dM, N)
s
implies
_
t
0
H
s
dJ
s
=
_
t
0
H
s
K
s
dM, N)
s
.
We would like a similar identity for stochastic integrals, namely
Y
t
=
_
t
0
K
s
dM
s
implies
_
t
0
H
s
dY
s
=
_
t
0
H
s
K
s
dM
s
.
This is the content of the next theorem.
Proposition 5.26. Let K L
2
(M) and H L
2
(K M), then
(HK) M = H (K M).
Proof. Apply the theorem above twice to see that K M, K M) = K
2
M, M). Thus, we have H K
L
2
(K M). For N H
2
0
, we also have
(HK) M, N) = (HK) M, N) = H (K M, N)) = H K M, N) = H (K M), N).
Now use the uniqueness of in the denition of the stochastic integral.
Proposition 5.27. Let be a stoppping time, then
K M
= KI
[0,]
M = (K M)
.
Proof. Claim. M
= I
[0,]
M.
Choose N H
2
0
,
M
, N) = M, N)
= I
[0,]
M, N) = I
[0,]
M, N).
Now use the preceding proposition,
K M
= K (I
[0,]
M) = KI
[0,]
M = I
[0,]
K M = I
[0,]
(K M) = (K M)
,
which completes the proof.
We now we shall use localization to extend the denition of the stochastic integral.
Denition 5.28. Let M be a continuous local martingale, then L
2
loc
(M) is the space of progressively mea-
surable functions K for which there exists an increasing sequence of stopping times
n
; n 1 increasing
to innity so that
E[
_
n
0
K
2
s
dM, M)
s
] < .
Exercise 5.29. For any K L
2
loc
(M), there exists a continuous local martingale K M such that for any
continuous local martingale N,
K M, N) = K M, N).
Denition 5.30. 1. Call a progressively measurable process K locally bounded if there exists a sequence
of stopping times
n
; n 1 increasing to innity and constants C
n
such that K
n
C
n
.
2. Let K be locally bounded and let X = M+V be a continuous semimartingale, X
0
= 0, the Ito stochastic
integral of K with respect to X is the continuous semimartingle
K X = K M +K V
where K M is the stochastic integral dened above and K V is the usual Stieltjes integral.
Exercise 5.31. The map K K X has the following properties:
1. H (M X) = (HM) X.
2. (K M)
= (KI
[0,]
) M) = K M
.
3. If X is a local martingale or a process of nite variation, so is K X.
4. If K is an progressively measurable simple function,
K = K
0
I
0]
+
j=1
K
j
I
(tj1,tj]
, 0 = t
0
< t
1
< , lim
j
t
j
= ,
then
(K X)
t
=
j=1
K
j
(X
mint,tj]
X
mint,tj1]
.
Theorem 5.32 (bounded convergence theorem). Let X be a continuous semimartingale and let K
n
; n 1
be locally bounded processes converging to zero pointwise with [K
n
[ K for some locally bounded process K,
then for any T > 0 and > 0,
lim
n
P sup
0tT
[(K
n
X)
t
[ > = 0.
Proof. If X is a process of nite variation, then we can apply the bounded convergence theorem for each
.
If X is a local martingale and it reduces X, then by the bounded convergence theorem, (K
n
)
converges
to zero in L
2
(X
) and thus (K
n
X)
converges to zero in H
2
. Now repeat the argument on the convergence
of the quadratic variation over a partition for a local martingale.
Exercise 5.33. Let K be left-continuous and let
n
; n 1 be a sequence of partitions of [0, ), with
n
= 0 = t
n
0
< t
n
0
< , mesh(
n
) 0, then
lim
n
P sup
0tT
[(K X)
t

j=1
K
t
n
j1
(X
mint,t
n
j
]
X
mint,t
n
j1
]
[ > = 0.
Exercise 5.34. 1. Let X be a continuous semimartingale and let the stochastic integrals H X and H X
exists, then for a, b R
_
t
0
(aH
s
+bK
s
) dX
s
= a
_
t
0
H
s
dX
s
+b
_
t
0
K
s
dX
s
.
2. Let X and

X be two continuous semimartingales and let K and

K be two predictable locally bounded
processes. Prove that if the nite dimensional distributions of (K, X) and (

K,

X) agree, then so do the
nite dimensional distributions of (K, X, K X) and (

K,

X,

K

X).
5.3 The Ito Formula
The Ito formula is the change of variable formula for stochastic integrals. The formula gives explicitily the
decomposition of functions of a semimartingale as a local martingale and a process of bounded variation.
Proposition 5.35. Let X be a continuous semimartingale, then
X
2
t
= X
2
0
+ 2
_
t
0
X
s
dX
s
+X, X)
t
.
Proof. Let
n
; n 1 be a sequence of partitions of [0, ),
n
= 0 = t
n
0
< t
n
0
< , mesh(
n
) 0.
Expand to obtain,
j=1
(X
mint,tj]
X
mint,tj1]
)
2
= X
2
t
X
2
0
2
j=1
X
mint,tj1]
(X
mint,tj]
X
mint,tj1]
).
Then,
j=1
(X
mint,tj]
X
mint,tj1]
)
2
X, X)
t
.
and
j=1
X
mint,tj1]
(X
mint,tj]
X
mint,tj1]
)
_
t
0
X
s
dX
s
uniformly in probability on compact intervals.
Corollary 5.36. (Integration by parts formula) Let X and Y be continuous semimartingales, then
X
t
Y
t
= X
0
Y
0
+
_
t
0
X
s
dY
s
+
_
t
0
Y
s
dX
s
+X, Y )
t
.
Proof. The corollary follows from polarization of the identity in the previous proposition.
Example 5.37. Let B be standard Brownian motion, then
B
2
t
t =
_
t
0
B
s
dB
s
.
Denition 5.38. A d-dimensional vector (continuous) local martingale is an R
d
-valued stochastic process
X = (X
1
, . . . , X
d
) such that each component is a (continuous) local martingale. A complex (continuous)
local martingale is one in which both the real and imaginary parts are (continuous) local martingales.
Theorem 5.39 (Ito formula). Let f C
2
(R
d
, R) and X = (X
1
, . . . , X
d
) be a vector continuous semimartin-
gale, then f X is a continuous semimartingale and
f(X
t
) = f(X
0
) +
d
i=1
_
t
0
f
x
i
(X
s
) dX
i
s
+
1
2
d
i=1
d
j=1
_
t
0
2
f
x
i
x
j
(X
s
) dX
i
, X
j
)
s
.
Thus, the class of semimartingales is closed under composition with C
2
-functions.
Proof. If the formula holds for f, then, by the integration by parts formula, the formula holds for g(x) =
x
k
f(x). Thus, the Ito formula holds for polynomials. Now, for any compact set K, stop the process at
K
.
By the Stone-Weierstrass theorem, any f C
2
(K, R) can be uniformly approximated by polynomials. Thus
the formula holds up to time
K
by the stochastic dominated convergence theorem. Now, extend the result
to all of R
d
.
From a vector semimartingale X we often write the stochastic integral
Y
t
= Y
0
+
d
i=1
_
t
0
H
i
s
dX
i
s
in dierential form
dY
t
=
d
i=1
H
i
s
dX
i
s
.
Using this notation, the Ito formula becomes
df(X
t
) +
d
i=1
f
x
i
(X
s
) dX
i
s
+
1
2
d
i=1
d
j=1
2
f
x
i
x
j
(X
s
) dX
i
, X
j
)
s
.
Example 5.40. 1. let f : R R
+
C satisfy
f
t
+
1
2
2
f
x
2
= 0
Then, if M is a local martingale, then so is the process f(M
t
, M, M))
t
; t 0. In particular,
c
t
(M) = exp(M
t

2
2
M, M)
t
), C
This local martingale satises the stochastic dierential equation
dY
t
= Y
t
dM
t
.
2. Note that for standard Brownian motion,
exp(B
t

2
2
t)
is a martingale. Use the standard machine to show that
c
f
t
(B) = exp
__
t
0
f(t) dB
s

1
2
_
t
0
f(t)
2
ds
_
is a martingale.
3. If B is a d-dimensional Brownian motion and if f C
2
(R
+
R
d
), then
f(t, B
t
)
_
t
0
(
1
2
f +
f
t
)(s, B
s
) ds
is a martingale. Thus, if f is harmonic in R
d
, then f B is a local martingale. In addition,
1
2
is the
restriction of the generator of B to C
2
(R
d
).
Exercise 5.41. If B = (B
1
, . . . , B
d
) is d-dimensional Brownian motion then B
i
, B
j
) =
ij
t.
Example 5.42. If f : R
d
R is a function of the radius, then the Laplacian can be replaced by its radial
component. Writing R
t
= [B
t
[, we have
f(t, R
t
)
_
t
0
(Gf +
f
t
)(s, R
s
) ds
is a martingale if G is the generator for the Bessel process in d dimensions.
Check that s(r) = ln r is a scale function for the Bessel process in 2 dimensions and that s(r) = r
2d
is a scale function for dimension d 3. So if r
i
< r < r
o
, then
P
r
ro
>
ri
=
s(r
o
) s(r)
s(r
o
) s(r
i
)
.
Now let r
o
, then for d = 2,
P
r
ri
< = 1.
Thus, in 2 dimensions, Brownian motion must return innitely often to any open set containing the origin.
For d 3
P
r
ri
< =
_
r
i
r
_
d2
and Brownian has a positive probability of never returning to a given neighborhood of the origin. In short,
Brownian motion is recurrent in dimension 2 and transient in dimensions higher than 2. This conforms with
similar results for random walks.
Theorem 5.43 (Levys characterization theorem). Assume that X is a R
d
valued stochastic process satis-
fying X
0
= 0. Then the following are equivalent.
1. X is d-dimensional standard Brownian motion.
2. X is a continuous local martingale and X
i
, X
j
) =
ij
t.
Proof. 1 2 is the exercise above.
(2 1) Take = i and M = X in the exponential martingale, then
M, M)
t
= [[
2
t
and we have the local martingale
exp(i X
t
+
1
2
[[
2
t).
Because it is bounded, it is a martingale. Thus, for s < t
E[exp(i X
t
+
1
2
[[
2
t)[T
s
] = exp(i X
s
+
1
2
[[
2
s).
or
E[exp(i (X
t
X
s
)[T
s
] = exp(
1
2
[[
2
(t s)).
Thus, X has stationary and independent Gaussian increments and X
1
has a normal distribution with the
identity matrix as its covariance matrix. Thus, it is standard Brownian motion.
5.4 Stochastic Dierential Equations
Denition 5.44. Let f and g be adapted functions taking values in R
dr
and R
d
respectively and let B be
a standard r-dimensional Brownian motion. Consider the stochastic dierential equation
X
i
t
= X
i
0
+
r
j=1
(
_
t
0
f
ij
(s, X) dB
j
s
+
_
t
0
g
i
(s, X) ds)
or, in vector dierential form,
dX
t
= f(s, X) dB
s
+g
i
(s, X) ds.
A solution is a pair (X, B) of T
t
-adapted processes that satisfy this equation.
The variety of notions of equivalence for stochastic processes leads to diering denitions of uniqueness
of uniqueness for solutions to stochastic dierential equations.
Denition 5.45. 1. Let (X, B) and (

X, B) be two solutions on the same probability space with the same
ltration and the same initial conditions. Then the stochastic dierential equation above is said to
satisfy pathwise uniqueness if X and

X are indistinguishable.
2. Let (X, B) and (

X,

B) be two solutions so that the distributions of X
0
and

X
0
are equal. Then the
stochastic dierential equation above is said to satisfy uniqueness in law if X and

X are two versions
of the same process.
We will be focused on pathwise uniquesness. We begin with the basic estimate used to study the
dependence of a solution on initial condiations.
Theorem 5.46. (Gronwalls inequality) Let f, g : [0, T] R be continuous and nonnegative. Suppose
f(t) A+
_
t
0
f(s)g(s) ds, A 0.
Then, for t [0, T],
f(t) Aexp(
_
t
0
g(s) ds).
Proof. First suppose A > 0 and set
h(t) = A+
_
t
0
f(s)g(s) ds.
Thus h(t) > 0. Then,
h
t
(t) = f(s)g(s) h(t)g(t) or
h
t
(t)
h(t)
g(t).
Integration gives
f(t) h(t) Aexp(
_
t
0
g(s) ds).
For A = 0, take limits to see that f is the zero function.
The basic hypothesis for existence and uniqueness that we will use is an assumption of the following
Lipschitz condition.
Let x, x C
R
r [0, ). There exists a constant K such that for every t > 0,
[f(t, x) f(t, x)[ +[g(t, x) g(t, x)[ K sup
st
[x
s
x
s
[.
Theorem 5.47. Let T
t
be a right-continuous complete ltration on (, TP) and let Z be a continuous
r-dimensional semimartingale. If the progressive process f(t, X) satises the the Lipschitz condition above
and if for every t > 0,
E[ sup
0st
__
s
0
f(r, x) dr
_
2
] < ,
where x is a constant function for some x
0
R
r
, then there is a pathwise unique process X such that
X
t
= x +
_
t
0
f(s, X) dZ
s
.
Proof. Note that the Lipschitz condition guarantees that if the condition above holds for one choice x
0
then
it holds for all x R
r
.
Write the semimartingale Z = M +A in its canonical decomposition. Let [A[
t
denote the variation of A
up to time t
Case 1. The measures associated to M, M) and [A[ are absolutely continuous with respect to Lebesgue
measure and for all i and j.
dM
i
, M
j
)
t
dt
1,
d[A[
t
dt
1.
Fix x R
r
and dene the mapping
(SU)
t
= x +
_
t
0
f(s, U) dZ
s
.
and the norm
[[U V [[
2
2,t
= E[ sup
0st
[U
s
V
s
[
2
].
Then, by the Doob and Cauchy-Schwartz inequality.
[[SU SV [[
2
2,t
2E[ sup
0st
[
_
s
0
(f(r, U) f(r, V )) dM
s
[
2
] + 2E[ sup
0st
[
_
s
0
(f(r, U) f(r, V )) d[A[
s
[
2
]
8E[
_
t
0
(f(r, U) f(r, V )) dM
s
[
2
+ 2E[
r
j=1
[A
i
[
t
_
t
0
(f
j
(r, U) f
j
(r, V ))
2
d[A
j
[
s
[]
8E[
r
i=1
r
j=1
_
t
0
(f
i
(r, U) f
i
(r, V ))(f
j
(r, U) f
j
(r, V )) dM
i
, M
j
)
s
]
+2tE[
r
j=1
_
t
0
(f
j
(r, U) f
j
(r, V ))
2
d[A
j
[
s
[]
2K
2
(4r
2
+rt)
_
t
0
[[U V [[
2
2,r
dr
This sets us up for the Picard-Lindelof iteration scheme. Set
X
0
= x, and X
n
= SX
n1
and write
C = 2K
2
(4r
2
+rT).
Now, check, by induction, that
[[X
n+1
X
n
[[
2
2,t

(Ct)
n
n!
[[X
1
X
0
[[
2
2,T
.
Thus, X
n
; n 1 is a Cauchy sequence in [[ [[
2,T
. Call the limit X. Then X is a continuous process
and satises X = XS. In other words, X is a solution.
To show that X is unique, let

X be a second solution. Dene
k
= inft > 0 : [X
t
[ k or [

X
l
[ k. Use
Gronwalls inequality, with f(t) = [[X
k
[[
2,t
and g(t) = C to see that

X
0
= x, implies

X
k
= X
k
.
Now, let T and k go to innity.
Case 2. The general case.
Dene the strictly increasing process
A
t
= t +
r
i=1
r
j=1
M
i
, M
j
)
t
+
r
i=1
[A
i
[
t
Consider the time change C
t
=

A
1
t
and write
M
t
= M
Ct
,

A
t
= A
Ct
, and

Z
t
=

M
t
+

A
t
.
Then

Z satises the circumstances in case 1 with the ltration

T
t
= T
Ct
. Thus,
X
t
= x +
_
t
0
f(s,

X) d
Z
s
has a unique solution. Now, X
t
=

X
At
is the unique solution to the equation in the statement of the
theorem.
5.5 Ito Diusions
Denition 5.48. A continuous stochastic process X is called a time homogeneous Ito diusion is there
exists measurable mappings
1. : R
d
R
dr
, (the diusion matrix), and
2. b : R
d
R
d
, (the drift)
and an r-dimensional Brownian motion B so that
X
i
t
= X
i
0
+
r
j=1
ij
(X
s
) dB
j
s
+
_
t
0
b
i
(X
s
) ds
has a solution that is unique in law.
If and b satisfy an appropriate Lipschitz hypothesis, then we have pathwise uniqueness. To show that
this suces, we have the following.
Theorem 5.49. Let and b be locally bounded and Borel measurable, and let be a probability measure on
R
d
. Then pathwise uniqueness to the Ito stochastic dierential equation above implies distribution uniqueness.
We begin with a lemma.
Lemma 5.50. The following condition is sucient for uniqueness in law:
Let x R
d
. Whenever (X, B) and (

X,

B) are two solutions satisfying
X
0
= x,

X
0
= x, a.s.
Then the distribution of X and

X are equal.
Proof. Let P be the distribution of (X, B) on the space C
R
r [0, ) and write the pair (, ) for the coordinate
functions for the rst d and last r coordinates respectively. Because C
R
d+r [0, ) is a Polish space, there is
a regular conditional probability
P(B[B
0
)(, ) = P(B, (, )).
In particular, has the distribution of r-dimensional Brownian motion. The triple (X, B, (X) B) and
(, , () ) have the same distribution and thus (, ) solves the stochastic dierential equation.
Repeat this letting

P be the distribution of (

X,

B). Then, the regular conditional probabilities agree
P(B, (, )) = P(B, (, ))
and so if X
0
and

X
0
have the same distribution, then P =

P.
Theorem 5.51. If pathwise uniqueness holds for an Ito type stochastic dierential equation, then uniqueness
in law holds. Moreover, there exists a measureable map
F : C
R
d[0, ) C
R
r [0, )
such that F() is adapted to the natural ltration B
r
t
=
s
; s t and for every solution (X, B), we have
that
X
t
() = F(B())
t
a.s.
Proof. Let (X, B) be a solution on (, P) and let (

X,

B) be a solution on (
,

P). With, for some x R
d
,
PX
0
= x =

P

X
0
= x = 1.
For the solution (X, B) let Q be the image of P under the map
(X(), B()).
For the solution (

X,

B) let

Q be the image of

P under the map
(

X( ),

B( )).
In addition, let W be Wiener measure, that is the law of r-dimensional Brownian motion, Dene a probability
measure on C
R
d[0, ) C
R
d[0, ) C
R
d[0, ) by
(d, d
, d) = Q(, d)

Q(, d
)W(d)
and consider the ltration
T
t
= (
s
,

s
,
s
); 0 s t.
Claim 1. is an r-dimensional T
t
Brownian motion.
Let A
s
; 0 s t,

A
s
; 0 s t, and B
s
; 0 s t and let s, t > 0, then
E
x
[expi,
t+s

t
); A

A B] =
_
B
expi,
t+s

t
)Q(, A)

Q(,

A)W(d)
= exp(
1
2
[[
2
t)
_
B
Q(, A)

Q(,

A)W(d)
= exp(
1
2
[[
2
t)(A

AB)
Thus has Gaussian independent increments with identity covariance matrix.
Claim 2. (, ) and (
, ) are two solutions to the stochastic dierential equation on the space

(C
R
d[0, ) C
R
d[0, ) C
R
r [0, ), )
with ltration T
t
; t 0.
The joint law of (X, B, (X) B) under P is the same as the joint law of (, , () ) under . Similarly,
the joint law of (

X,

B, (

X)

B) under

P is the same as the joint law of (
, , (
) ) under . Because
0
=

0
= x a.s. , the property of pathwise uniqueness implies that and

are indistinguishable. Thus,
uniqueness in law holds.
Solution is Markov.
Exercise 5.52. If X is an Ito diusion and if f C
2
(R
d
, R), then
f(X
t
) = f(X
0
) +
d
i=1
_
t
0
f
x
i
(X
s
) dX
i
s
+
1
2
d
i=1
d
j=1
_
t
0
2
f
x
i
x
j
(X
s
) dX
i
, X
j
)
s
= f(X
0
) +
d
i=1
_
t
0
(b
i
(X
s
)
f
x
i
(X
s
) +
1
2
(
T
)
ij
(X
s
)) ds +
d
i=1
r
k=1
_
t
0
ik
(X
s
)
f
x
i
(X
s
) dB
k
s
.
In particular, we have the martingales
M
f
t
= f(X
0
) +
_
t
0
Gf(X
s
) ds.
with
Gf(x) =
d
i=1
b
i
(x)
f
x
i
(x) +
1
2
d
i=1
d
j=1
a
ij
(x)

2
f
x
i
x
j
(x)
and a =
T
.
square roots
Denition 5.53. A stochastic process X is called a solution to the martingale problem for the collection /
if
f(X
t
)
_
t
0
g(X
s
) ds
is a martingale for every (f, g) /.
For a probability distribution , we say that X is a solution to the martingale problem for (/, ) if X is
a solution to the martingale problem and = PX
1
0
.
Uniqueness holds for the martingale problem (/, ) if any two solutions have the same nite dimensional
distributions. In this case, we say that the martingale problem is well posed. If uniqueness holds for any
initial distribution , then we say that the martingale problem / is well posed.
The relationship to Ito diusions is as follows.
Theorem 5.54. Let
: R
d
R
dd
, and b : R
d
R
d
and let be a probability measure on R
d
. Dene A as above and / = (f, Af) : f C
c
(R
d
). Then there
exists a solution to the Ito stochastic dierential equation corresponding to and b with initial distribution
if and only if there exists a solution to the martingale problem (/, ).
Moreover, uniqueness in law to the stochastic dierential equation corresponding to and b with initial
distribution if and only if the martingale problem (/, ) is well posed.
This will follow from a sequence of lemmas:
Lemma 5.55. Assume that and b are locally bounded and measurable. In addition,
1
exists and is
locally bounded. If X is a solution to the C
R
d[0, )-martingale problem for (/, ) with respect to a complete
ltration T
t
; t 0, then there exists a d-dimensional Brownian motion B such that (X, B) is a solution
to the stochastic dierential equation.
Proof. For every f C
c
(R
d
)
M
f
t
= f(X
t
) f(X
0
)
_
t
0
Af(X
s
) ds
is a continuous martingale, we have that this expression gives a continuous local martingale for every f
C
(R
d
). Choose f(x) = x
i
to obtain the martingale
M
i
t
= X
i
t
X
0

_
t
0
b
i
(X
s
) ds
is a local martingale. Choose f(x) = x
i
x
j
,
M
i
, M
j
)
t
= X
i
, X
j
)
t
=
_
t
0
a
ij
(X
s
) ds.
Claim.
B
t
=
_
t
0
1
(X
s
) dM
s
is a d-dimensional T
t
-adapted Brownian motion.
B is a vector valued continuous local martingale and B
0
= 0. Thus, by Levys characterization, it suces
to show that B
i
, B
j
)
t
=
ij
t.
B
i
, B
j
)
t
=
d
k=1
d
=1
1
(X)
ik
M
k
,
1
(X)
j
M
)
t
=
d
k=1
d
=1
_
t
0
(
1
ik
a
k
(
T
)
1
j
)(X
s
) ds =
ij
t.
Thus,
_
t
0
(X
s
) dB
s
=
_
t
0
dM
s
= X
t
X
0
_
t
0
b(X
s
) ds
and the stochastic dierential equation has a solution.
To consider the case of singular we have the following,
Lemma 5.56. There exists Borel measurable functions
, : R
d
R
dd
such that
a
T
+
T
= I
d
, = 0, (I
d
)a(I
d
)
T
= 0.
Lemma 5.57. Let B
t
be a d-dimensional T
t
t
-Brownian motion on (
t
, P
t
). Dene
=
t
,

P = P P
t
, and

X
t
(,
t
) = X
t
().
Then there exists a d-dimensional T
t
T
t
t
-Brownian motion

B such that (

X,

B) is a solution to the stochastic
dierential equation.
Proof. With M as above, dene
M
t
(,
t
) = M
t
() and

B
t
t
(,
t
) = B
t
t
(
t
).
Because

M and

B are independent, M
k
,

B
)
t
= 0 for k, = 1, . . . d.
Claim.
B
t
=
_
t
0
(X
s
) d

M
s
+
_
t
0
(X
s
) d

B
t
s
denes a d-dimensional Brownian motion.
As before, we have that

B is a vector valued continuous local martingale.
B
i
,

B
j
)
t
=
d
k=1
d
=1
(X)
ik

M
k
, (X)
j

M
)
t
+
d
k=1
d
=1
(X)
ik

B
tk
, (X)
j

B
t
)
t
=
d
k=1
d
=1
_
t
0
(
ik
a
k
j
)(X
s
) ds +
d
k=1
d
=1
_
t
0
(
ik
jl
)(X
s
) ds
=
_
t
0
(a
T
+
T
)
ij
(X
s
) ds =
ij
t
To show that this Brownian can be used to solve the stochastic dierential equation, note that,
_
t
0
(

X
s
) d

B
s
=
_
t
0
()(

X
s
) d

M
s
+
_
t
0
()(

X
s
) d

B
s
=
_
t
0
()(

X
s
) d

M
s
=

M
t

_
t
0
(I
d
)(

X
s
) d

M
s
=

M
t
=

X
t

X
0

_
t
0
b(X
s
) ds
because
k=1
(I
d
)(

X
s
)
ik

M
k
,
d
=1
(I
d
)
ik
(

X
s
)

M
k
)
t
=
d
k=1
d
=1
_
t
0
(I
d
)
ik
a
k
(I
d
)
i
(

X
s
) ds = 0
and the lemma follows.

Notesc

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Notesc

Uploaded by

Copyright:

Available Formats

Stochastic Processes in Continuous Time

metric is not separable. Hint: Find an uncountable collection of

are dened for continuous time processes

. Consequently, for each t 0, we must show

= x; d(x, A) < . Thus

have the same distribution.

is concentrated on the set 0, . Set p

() of real-valued random variables on and dene the metric

to show the existence of left limits.

exists almost surely with E[X

is a Levy process with respect to the ltration T

, > 1/2, then B

x whenever < . Consequently,

n+1 is a normal random variable with mean 0 and variance

be an exponential random variable with parameter , then as ,

is a bounded linear operator on L and T

for all , > 0

and its semigroup T

. Then by the lemma

(t)f converges to to T(t)f as .

satisfy the eigenvalue problem Gf

is an exponential random variable with parameter

Exercise 4.80. For a birth and death process, let = min

is the one point compactication of S.

(t)f = f() +T(t)(f f()).

is a Feller semigroup on C(S

is a strongly continuous conservative semigroup on C(S

is positive. In other words, given c R

)(x) c and so c +T(t)f 0.

= (R() I) We know that R() is a positive linear operator on C

(x, S) 1. However, if G is conservative, we can choose f(x) = 1/

is an exponential random variable, parameter , and independent from X. Thus,

as above. Let X be a Markov process corresponding to T

) such that f > 0 on S and f() = 0. Then the

, > 0, be its Yosida approximation. Then T

. Moreover, for every t 0 and f C

be the corresponding semigroups on C(S

). Then, for each f C(S

and generator Gk.

). Then the representation above holds whenever the

In schematic form, we have

< . In this case,

A simple process has the form,

, ) are two solutions to the stochastic dierential equation on the space

You might also like