Reversible Markov chains

with nonnegative spectrum

Roberto Imbuzeiro Oliveira (IMPA)
Results/proofs obtained with Yuval Peres
Outline (as of right now)
Lecture 1
Basics, examples, extremal chains.

Lecture 2 (today)
Hitting times, concentration.

Lecture 3
Meetings and intersections, improved bounds on graphs.
Recall the definition
A MC with transition matrix on a finite set with n elements is a
random process

such that for all times

Only dependence on the past comes from previous state.

Probability of a path
If distribution of (row vector),

and
.

A stationary distribution satisfies .

Fundamental theorem of Markov chains
(Theorem)

If P is irreducible and aperiodic, there exists a unique stationary

distribution and:

Chain forgets initial state.

Asymptotic distribution is
Reversibility
If is irreducible, aperiodic, and reversible with respect to

• is the unique stationary distribution;

• self-adjoint for inner product
;
• spec ()= {
• (right) eigenvectors are orthonormal:
Our objects of study

Definition ( chain)
A chain is a Markov chain that is reversible with respect to
(assumed fully supported), has spectral gap and has nonnegative
eigenvalues.

chain is irreducible and aperiodic.

irreducible +for all for some
Extremal chains
Definition (extremal chain)
matrix with for all
The extremal chain is given by the transition matrix:

Intuition
At each step, flip a coin with probability of heads.
If you do get heads, jump according to . If you get tails, stay put.
Extremal chains are extremal
Lemma
Given write if for all
Then for any chain and any
.
Hitting times
Hitting times

Time 0

Set 𝐴
Hitting times

Time 1

Set 𝐴
Hitting times

Time 2

Set 𝐴
Hitting times

Set 𝐴 Time 3
Hitting times

Set 𝐴 Time 4
Hitting times

Set 𝐴

Time 5
Hitting times

Time 6

Set 𝐴
𝜏 𝐴 =6
Hitting times
Definition (hitting time)

The hitting time of is defined as:

1. What chain has the largest hitting time for a set A?

2. What can we say about the distribution of the hitting time?
Hitting times
(Theorem 1) For any chain, we have that:

and the upper bound is achieved by the extremal chain.

Geometric approximation of hitting times
(Theorem 1) For any
(Aldous/Brown’92 *) chain, we have that:
For any chain, with: ,
and the upper bound is achieved by the extremal chain.

(*) A/B prove this in continuous time with complicated proof.

All bounds are exact for extremal chain.
Interpretation
if for all

Hitting times upper bounded by geometric random variables

At least when initial distribution is

If the set is “hard to reach”, hitting time

“Hard to reach” means
= exp. hitting time = relaxation time
An example of a hard-to-reach set
Biased random walk on

1/3 1/6

(...)

A
1/2
Notation

Space of vectors with coordinates indexed by also the space of

functions

The above space with inner product

We always assume is a probability measure and

Matrices with entries indexed by .

Understanding hitting times
Write down the relevant event
equivalent form
sum over paths
probs. of paths
Now use indicators
Recall if and if

)
Understanding hitting times
Introduce a new matrix
Define a diagonal matrix with

Consequence for our proof

)
=
Understanding the matrix

Properties of
Reversible, nonnegative entries (easy to check).
Since we obtain:
.
Proof that
Largest eigenvalue for such matrices
Variational formula
, achieved by .
Limit formula
For any strictly positive
Largest eigenvalue = operator norm
= :=sup { : =1}
Submultiplicativity
Consequences

Corollaries for
Operator norm = largest eigenvalue
Corresponding eigenvector is nonnegative
Nonnegative spectrum, .
Variational formula monotone in matrix order:
Hitting times
(Theorem 1) For any chain, we have that:

and the upper bound is achieved by the extremal chain.

Proof sketch for Theorem 1
For any chain , and any diagonal with nonnegative entries,

Recall Note that

where
Geometric approximation of hitting times
(Theorem 1) For any
(Aldous/Brown’92 *) chain, we have that:
For any chain, with: ,
and the upper bound is achieved by the extremal chain.

(*) A/B prove this in continuous time with complicated proof.

All bounds are exact for extremal chain.
Proof sketch for Aldous/Brown
with nonnegative entries:

where normalized eigenvector for .

Proof sketch for Aldous/Brown
Get an upper bound on by contraction:

which gives and

.
Simple bound for diagonal matrices
Proposition I
For any chain , and any diagonal with nonnegative entries,

Uses the fact that the extremal chain is extremal

+
For nonnegative self adjoint matrices,
Concentration and large
deviations
The problem
Given with

and a chain, find good bounds for:

(.

Concentration: inexact, nonasymptotic bounds.

Large deviations: exact limits
One example

How likely is it
that RW spends
an atypical amount
of time in A?

Set 𝐴
Functions
(
The Benrnstein trick
Markov’s inequality
For a nonnegative random variable,

Moment generating functions

The trick
I(
.
The Bernstein trick
Moment generating function

The trick
I(.

Is this a good bound?

Yes, esp. with optimal . See Gä rtner-Ellis Theorem.
The case of equal functions
(Theorem 3) For any chain and any fixed

where is the unique solution to the equation

Upper bound becomes “=“ when for extremal chain.

Varying functions: concentration bound
(Theorem 4) For any chain and any
Varying functions: Bernstein concentration
(Theorem 5) For any and any -chain,

(.

where and

Can also reobtain previous bounds by several authors,

including Gilman, Lézaud, Leó n & Perron, Rao & Regev.
Previous bounds only sharp for taking 2 values.
Proof elements for equal functions
First step – matrix form
, where is diagonal and
Proof sketch
Proof elements for equal functions
Second step – compare to extremal chain
is reversible and has nonnegative entries

Limit formula for

=,
where follows the extremal chain.
Recursive structure of extremal chain
• D and , independently;
• Set for
• Draw fresh and , independently;
• Set for
• Draw fresh and , independently;
• Set for

.
Recursive structure of extremal chain
; want

Condition on first geometric

=
future indep. from
stationarity
Recursive structure of extremal chain

Unconditional recursion
Proof elements for equal functions
Complex analysis and the radius of convergence:

For any power series ,

where
Proof elements for equal functions
Generating function of MGF of extremal chain:

If we write

then
Proof elements for equal functions
Recursions and identification of convergence radius:
Define another analytic function:

and use the recursion of the extremal chain to obtain: