Professional Documents
Culture Documents
Lectures On Spectral Graph Theory: Fan R. K. Chung
Lectures On Spectral Graph Theory: Fan R. K. Chung
Fan R. K. Chung
Author address:
1.1. Introduction 1
2.1. History 23
5.1. Quasi-randomness 73
Bibliography 91
CHAPTER 1
1.1. Introduction
Spectral graph theory has a long history. In the early days, matrix theory
and linear algebra were used to analyze adjacency matrices of graphs. Algebraic
methods are especially eective in treating graphs which are regular and symmetric.
Sometimes, certain eigenvalues have been referred to as the algebraic connectivity
of a graph [126]. There is a large literature on algebraic aspects of spectral graph
theory, well documented in several surveys and books, such as Biggs [25], Cvetkovic,
Doob and Sachs [90, 91], and Seidel [224].
In the past ten years, many developments in spectral graph theory have often
had a geometric avor. For example, the explicit constructions of expander graphs,
due to Lubotzky-Phillips-Sarnak [193] and Margulis [195], are based on eigenvalues
and isoperimetric properties of graphs. The discrete analogue of the Cheeger in-
equality has been heavily utilized in the study of random walks and rapidly mixing
Markov chains [224]. New spectral techniques have emerged and they are powerful
and well-suited for dealing with general graphs. In a way, spectral graph theory
has entered a new era.
The study of graph eigenvalues realizes increasingly rich connections with many
other areas of mathematics. A particularly important development is the interac-
tion between spectral graph theory and dierential geometry. There is an interest-
ing analogy between spectral Riemannian geometry and spectral graph theory. The
concepts and methods of spectral geometry bring useful tools and crucial insights
to the study of graph eigenvalues, which in turn lead to new directions and results
in spectral geometry. Algebraic spectral methods are also very useful, especially
for extremal examples and constructions. In this book, we take a broad approach
with emphasis on the geometric aspects of graph eigenvalues, while including the
algebraic aspects as well. The reader is not required to have special background in
geometry, since this book is almost entirely graph-theoretic.
1
2 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH
From the start, spectral graph theory has had applications to chemistry [27].
Eigenvalues were associated with the stability of molecules. Also, graph spectra
arise naturally in various problems of theoretical physics and quantum mechanics,
for example, in minimizing energies of Hamiltonian systems. The recent progress
on expander graphs and eigenvalues was initiated by problems in communication
networks. The development of rapidly mixing Markov chains has intertwined with
advances in randomized approximation algorithms. Applications of graph eigen-
values occur in numerous areas and in dierent guises. However, the underlying
mathematics of spectral graph theory through all its connections to the pure and
applied, the continuous and discrete, can be viewed as a single unied subject. It
is this aspect that we intend to cover in this book.
Before we start to dene eigenvalues, some explanations are in order. The eigen-
values we consider throughout this book are not exactly the same as those in Biggs
[25] or Cvetkovic, Doob and Sachs [90]. Basically, the eigenvalues are dened here
in a general and normalized form. Although this might look a little complicated
at rst, our eigenvalues relate well to other graph invariants for general graphs in
a way that other denitions (such as the eigenvalues of adjacency matrices) often
fail to do. The advantages of this denition are perhaps due to the fact that it is
consistent with the eigenvalues in spectral geometry and in stochastic processes.
Many results which were only known for regular graphs can be generalized to all
graphs. Consequently, this provides a coherent treatment for a general graph. For
denitions and standard graph-theoretic terminology, the reader is referred to [31].
In a graph G, let dv denote the degree of the vertex v. We rst dene the
Laplacian for graphs without loops and multiple edges (the general weighted case
with loops will be treated in Section 1.4). To begin, we consider the matrix L,
dened as follows:
dv if u = v,
L(u, v) = 1 if u and v are adjacent,
0 otherwise.
Let T denote the diagonal matrix with the (v, v)-th entry having value dv . The
Laplacian of G is dened to be the matrix
1 if u = v and dv = 0,
1
L(u, v) = if u and v are adjacent,
du dv
0 otherwise.
We can write
L = T 1/2 LT 1/2
For readers who are familiar with terminology in homology theory, we remark
that S can be viewed as a boundary operator mapping 1-chains dened on
edges (denoted by C1 ) of a graph to 0-chains dened on vertices (denoted by
C0 ). Then, S is the corresponding coboundary operator and we have
S
C1 C
0
S
Since L is symmetric, its eigenvalues are all real and non-negative. We can
use the variational characterizations of those eigenvalues in terms of the Rayleigh
quotient of L (see, e.g. [162]). Let g denote an arbitrary function which assigns to
each vertex v of G a real value g(v). We can view g as a column vector. Then
g, Lg
g, T 1/2 LT 1/2 g
=
g, g
g, g
f, Lf
=
T 1/2 f, T 1/2 f
(f (u) f (v))2
uv
(1.1) =
f (v)2 dv
v
4 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH
where g = T 1/2 f and denotes the sum over all unordered pairs {u, v} for which
uv
u and v are adjacent. Here
f, g = f (x)g(x) denotes the standard inner product
x
in Rn . The sum (f (u) f (v))2 is sometimes called the Dirichlet sum of G and
uv
the ratio on the left-hand side of (1.1) is often called
the Rayleigh quotient. (We
note that we can also use the inner product
f, g = f (x)g(x) for complex-valued
functions.)
From equation (1.1), we see that all eigenvalues are non-negative. In fact, we
can easily deduce from equation (1.1) that 0 is an eigenvalue of L. We denote the
eigenvalues of L by 0 = 0 1 n1 . The set of the i s is usually called
the spectrum of L (or the spectrum of the associated graph G.) Let 1 denote the
constant function which assumes the value 1 on each vertex. Then T 1/2 1 is an
eigenfunction of L with eigenvalue 0. Furthermore,
(f (u) f (v))2
uv
(1.2) G = 1 = inf .
f T 1 f (v)2 dv
v
|f |2
M = inf M ,
2
|f |
M
f = 0.
M
We remark that the corresponding measure here for each edge is 1 although in the
general case for weighted graphs the measure for an edge is associated with the edge
weight (see Section 1.4.) The measure for each vertex is the degree of the vertex.
A more general notion of vertex weights will be considered in Section 2.5.
1.2. THE LAPLACIAN AND EIGENVALUES 5
N
By substituting for f and using the fact that N (ai a)2 = (ai aj )2
i=1 i<j
N
for a = ai /N , we have the following expression (which generalizes the one in
i=1
[126]):
(f (u) f (v))2
(1.5) 1 = vol G inf uv ,
f (f (u) f (v))2 du dv
u,v
where denotes the sum over all unordered pairs of vertices u, v in G. We can
u,v
characterize the other eigenvalues of L in terms of the Rayleigh quotient. The
largest eigenvalue satises:
(f (u) f (v))2
(1.6) n1 = sup uv .
f f 2 (v)dv
v
For a general k, we have
(f (u) f (v))2
uv
(1.7) k = inf sup
f gPk1 (f (v) g(v))2 dv
v
(f (u) f (v))2
uv
(1.8) = inf
f T Pk1 f (v)2 dv
v
6 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH
The dierent formulations for eigenvalues given above are useful in dierent
settings and they will be used in later chapters. Here are some examples of special
graphs and their eigenvalues.
Example 1.1. For the complete graph Kn on n vertices, the eigenvalues are
0 and n/(n 1) (with multiplicity n 1).
Example 1.2. For the complete bipartite graph Km,n on m + n vertices, the
eigenvalues are 0, 1 (with multiplicity m + n 2), and 2.
Example 1.3. For the star Sn on n vertices, the eigenvalues are 0, 1 (with
multiplicity n 2), and 2.
Example 1.4. For the path Pn on n vertices, the eigenvalues are 1 cos n1
k
for k = 0, 1, , n 1.
Example 1.5. For the cycle Cn on n vertices, the eigenvalues are 1 cos 2k
n
for k = 0, , n 1.
n 2k
Example n
1.6. For the n-cube Qn on 2 vertices, the eigenvalues are n (with
multiplicity k ) for k = 0, , n.
Roughly speaking, half of the main problems of spectral theory lie in deriving
bounds on the distributions of eigenvalues. The other half concern the impact and
consequences of the eigenvalue bounds as well as their applications. In this section,
we start with a few basic facts about eigenvalues. Some simple upper bounds and
lower bounds are stated. For example, we will see that the eigenvalues of any graph
lie between 0 and 2. The problem of narrowing the range of the eigenvalues for
special classes of graphs oers an open-ended challenge. Numerous questions can
be asked either in terms of other graph invariants or under further assumptions
imposed on the graphs. Some of these will be discussed in subsequent chapters.
Lemma 1.7. For a graph G on n vertices, we have
(i):
i n
i
with equality holding if and only if G has no isolated vertices.
(ii): For n 2,
n
1
n1
with equality holding if and only if G is the complete graph on n vertices.
Also, for a graph G without isolated vertices, we have
n
n1 .
n1
1.3. BASIC FACTS ABOUT THE SPECTRUM OF A GRAPH 7
Equality holds for i = n 1 when f (x) = f (y) for every edge {x, y} in G.
Therefore, since f = 0, G has a bipartite connected component. On the other hand,
if G has a connected component which is bipartite, we can choose the function f
so as to make n1 = 2.
(i): G is bipartite.
(ii): G has i + 1 connected components and nj = 2 for 1 j i.
(iii): For each i , the value 2 i is also an eigenvalue of G.
8 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH
1
1
D vol G
1
(f (x) f (y)) = G f (x).
dx y
yx
1.3. BASIC FACTS ABOUT THE SPECTRUM OF A GRAPH 9
We have
(f (x) f (y))2
x,yV
xy
f2 (x)dx
xV
2%(f (x0 ) f (y)) 2%(f (y) f (y ))
(f (x) f (y))2 +
y dx0 y
vol G dx0
x,yV y
xy yx0 y=x0
yy
=
2%
f 2 (x)dx + 2%f (x0 ) f (y)dy
vol G dx0
xV y=x0
2
+O(% )
2% (f (x0 ) f (y)) 2% (f (x0 ) f (y))
y
yx0
y
yx0
(f (x) f (y))2 + +
dx0 vol G dx0
x,yV
xy
= 2%f (x0 )dx0
f 2 (x)dx + 2%f (x0 ) +
vol G dx0
xV
2
+O(% )
since f (x)dx = 0, and (f (y) f (y )) = 0. The denition in (1.2)
xV y y
implies that
(f (x) f (y))2 (f (x) f (y))2
x,yV x,yV
xy xy
f2 (x)dx f 2 (x)dx
xV xV
One can also prove the statement in Lemma 1.10 by recalling that f = T 1/2 g,
where Lg = G g. Then
T 1 Lf = T 1 (T 1/2 LT 1/2 )(T 1/2 g) = T 1/2 G g = G f,
and examining the entries gives the desired result.
With a little linear algebra, we can improve the bounds on eigenvalues in terms
of the degrees of the vertices.
There are relatively easy ways to improve the upper bound for 1 . From the
characterization in the preceding section, we can choose any function f : V (G) R,
and its Rayleigh quotient will serve as an upper bound for 1 . Here we describe an
upper bound for 1 (see [204]).
Lemma 1.14. Let G be a graph with diameter D 4, and let k denote the
maximum degree of G. Then
k1 2 2
1 1 2 1 + .
k D D
We note that Lemma 1.14 gives a proof (see [5]) that for any xed k and for
any innite family of regular graphs with degree k,
k1
lim sup 1 1 2 .
k
This bound is the best possible since it is sharp for the Ramanujan graphs (which
discussed in Chapter ??). We note that the cleaner version of 1
will be
1 2 k 1/k is not true for certain graphs (e.g., 4-cycles or complete bipar-
tite graphs). This example also illustrates that the assumption in Lemma 1.14
concerning D 4 is essential.
Before dening weighted graphs, we will say a few words about two dierent
approaches for giving denitions. We could have started from the very beginning
with weighted graphs, from which simple graphs arise as a special case in which
the weights are 0 or 1. However, the unique characteristics and special strength of
12 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH
graph theory is its ability to deal with the {0, 1}-problems arising in many natural
situations. The clean formulation of a simple graph has conceptual advantages.
Furthermore, as we shall see, all denitions and subsequent theorems for simple
graphs can usually be easily carried out for weighted graphs. A weighted undirected
graph G (possibly with loops) has associated with it a weight function w : V V
R satisfying
w(u, v) = w(v, u)
and
w(u, v) 0.
We note that if {u, v} E(G) , then w(u, v) = 0. Unweighted graphs are just the
special case where all the weights are 0 or 1.
vol G = dv .
v
Let T denote the diagonal matrix with the (v, v)-th entry having value dv . The
Laplacian of G is dened to be
L = T 1/2 LT 1/2 .
We can still use the same characterizations for the eigenvalues of the generalized
versions of L. For example,
g, Lg
(1.13) G := 1 = inf
gT 1/2 1
g, g
f (x)Lf (x)
xV
= inf
! f
f (x)dx =0
f 2 (x)dx
xV
(f (x) f (y))2 w(x, y)
xy
= inf .
! f
f (x)dx =0
f 2 (x)dx
xV
G H
The proof follows from the fact that an eigenfunction which achieves H for H
can be lifted to a function dened on V (G) such that all vertices in G that contract
to the same vertex in H share the same value.
since the ratio is maximized when w(xi , xi+1 ) = k(k 1)i1 = w(yi , yi+1 ). This
completes the proof of the lemma.
It is easy to see that necessary conditions for the ergodicity of P are (i) irre-
ducibility, i.e., for any u, v V , there exists some s such that P s (u, v) > 0 (ii)
aperiodicity, i.e., g.c.d. {s : P s (u, v) > 0} = 1. As it turns out, these are also
sucient conditions. A major problem of interest is to determine the number of
steps s required for P s to be close to its stationary distribution, given an arbitrary
initial distribution.
(ii) implies n1 < 2. As we will see later in (1.15), together (i) and (ii) deduce
ergodicity.
We remind the reader that an unweighted graph has w(u, v) equal to either 0
or 1. The usual random walk on an unweighted graph has transition probability
1/dv of moving from a vertex v to any one of its neighbors. The transition matrix
P then satises
1/du if u and v are adjacent,
P (u, v) =
0 otherwise.
In other words,
1
f P (v) = f (u)
u du
uv
where
1 if 1 1 n1 1
=
2 n1 otherwise.
So, after s 1/ log(maxx dx /% miny dy ) steps, the L2 distance between f P s
and its stationary distribution is at most %.
Although occurs in the above upper bound for the distance between the
stationary distribution and the s-step distribution, in fact, only 1 is crucial in the
following sense. Note that is either 1 or 2 n1 . Suppose the latter holds,
i.e., n1 1 1 1 . We can consider a modied random walk, called the lazy
walk, on the graph G formed by adding a loop of weight dv to each vertex v. The
new graph has Laplacian eigenvalues k = k /2 1, which follows from equation
(1.13). Therefore,
1 1
1 n1 0,
and the convergence bound in L2 distance in (1.14) for the modied random walk
becomes
maxx dx
2/1 log( ).
% miny dy
In general, suppose a weighted graph with edge weights w(u, v) has eigenvalues
i with n1 1 1 1 . We can then modify the weights by choosing, for some
constant c,
w(v, v) + cdv if u = v
(1.15) w (u, v) =
w(u, v) otherwise.
The resulting weighted graph has eigenvalues
k 2k
k = =
1+c n1 + k
where
1 + n1 1
c= 1 .
2 2
Then we have
n1 1
1 1 = n1 1 = .
n1 + 1
Since c 1/2 and we have k 2k /(2 + k ) 2k /3 for k 1. In particular we
set
21 2
= 1 = 1 .
n1 + 1 3
Therefore the modied random walk corresponding to the weight function w has
an improved bound for the convergence rate in L2 distance:
1 maxx dx
log .
% miny dy
1.5. EIGENVALUES AND RANDOM WALKS 17
y T 1/2 = i i .
i
where i s denote the eigenfunction of the Laplacian L of the weighted graph asso-
ciated with the random walk. In particular,
dx
0 = ,
vol G
1
0 = .
vol G
1 vol G
t log % minx dx ,
1
vol G vol G
(t) et exp2t1 /(2+1 ) .
minx dx minx dx
(t) ec
if
1 vol G
t log
minx dx
The total variation distance is bounded above by the relative pointwise distance,
since
T V (s) = max max | (P s (y, x) (x)) |
AV (G) yV (G)
xA
volA volG
2
max (x)(s)
AV (G)
xA
volA volG
2
1
(s).
2
Therefore, any convergence bound using relative pointwise distance implies the
same convergence bound using total variation distance. There is yet another notion
of distance, sometimes called -squared distance, denoted by (s) and dened by:
1/2
(P (y, x) (x))
s 2
(s) = max
yV (G) (x)
xV (G)
max | P s (y, x) (x) |
yV (G)
xV (G)
= 2T V (s),
using the Cauchy-Schwarz inequality. (s) is also dominated by the relative point-
wise distance (which we will mainly use in this book).
1/2
(P s (x, y) (y))2
(s) = max
xV (G) (y)
yV (G)
1
max ( ((s))2 (y)) 2
xV (G)
yV (G)
(s).
We note that
(P s (x, y) (y))2
(s)2 (x)
x y
(y)
= x T 1/2 (P s I0 )T 1 (P s I0 )T 1/2 x
x
= x ((I L)2s I0 )x ,
x
where I0 denotes the projection onto the eigenfunction 0 , i denotes the i-th
orthonormal eigenfunction of L and x denotes the characteristic function of x.
Since
x = i (x)i ,
i
20 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH
we have
(1.16) (s)2 x ((I L)2s I0 )x
x
= ( i (x)i )((I L)2s I0 )( i (x)i )
x i i
= 2i (x)(1 i ) 2s
x i=0
= 2i (x)(1 i )2s
i=0 x
= (1 i )2s .
i=0
The above theorem is often derived from the Plancherel formula. Here we have
employed a direct proof. We remark that for some graphs which are not vertex-
transitive, a somewhat weaker version of (1.17) can still be used with additional
work (see [81] and the remarks in Section 4.6). Here we will use Theorem 1.18 to
consider random walks on an n-cube.
Example 1.19. For the n-cube Qn , our (lazy) random walk (as dened in
(1.15)) converges to the uniform distribution under the total variation distance, as
estimated as follows:
From Example (1.6), the eigenvalues of the Qn are 2k/n of
multiplicity nk for k = 0, , n. The adjusted eigenvalues for the weighted graph
corresponding to the lazy walk are k = 2k /(n1 + 1 ) = k n/(n + 1). By using
Theorem 1.18 (also see [104]), we have
n
1 1 n 2k 2s 1/2
T V (s) (s) ( (1 ) )
2 2 k n+1
k=1
1 n
4ks
( ek log n n+1 )1/2
2
k=1
c
e
if s 14 n log n + cn.
We can also compute the rate of convergence of the lazy walk under the rela-
tive pointwise distance. Suppose we denote vertices of Qn by subsets of an n-set
1.5. EIGENVALUES AND RANDOM WALKS 21
This implies
n
n 2k s
(s) = (1 )
k n+1
k=1
n
2ks
ek log n n+1
k=1
c
e
if
n log n
s + cn.
2
So, the rate of convergence under relative pointwise distance is about twice
that under the total variation distance for Qn .
In general, T V (s), (s) and (s) can be quite dierent [81]. Nevertheless, a
convergence lower bound for any of these notions of distance (and the L2 -norm) is
1 . This we will leave as an exercise. We remark that Aldous [4] has shown that
if T V (s) %, then P s (y, x) c (x) for all vertices x, where c depends only on
%.
Notes
For an induced subgraph of a graph, we can dene the Laplacian with boundary
conditions. We will leave the denitions for eigenvalues with Neumann boundary
conditions and Dirichlet boundary conditions for Chapter ??.
22 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH
The Laplacian for a directed graph is also very interesting. The Laplacian for
a hypergraph has very rich structures. However, in this book we mainly focus on
the Laplacian of a graph since the theory on these generalizations and extensions
is still being developed.
vol G
In some cases, the factor log min x dx
in the upper bound for (t) can be further
reduced. Recently, P. Diaconis and L. Salo-Coste [100] introduced a discrete ver-
sion of the logarithmic Sobolev inequalities which can reduce this factor further for
certain graphs (for (t)). In Chapter 12, we will discuss some advanced techniques
for further bounding the convergence rate under the relative pointwise distance.