MARKOV CHAINS AND RANDOM WALKS. > Definition 6.8: An ergodic state is one that is aperiodic and non-null persistent. > Definition 6.9: An ergodic Markov chain is one in which all states are ergodic. The following basic theorem on Markov chains may be found in most texts ‘on stochastic processes. Theorem 6.2 (Fundamental Theorem of Markov Chains): Any irreducible, finite, and aperiodic Markov chain has the following properties. 1, All states are ergodic. 2. There is a unique stationary distribution m such that, for 1 0. 3. For Sin, fu =1 and hy = 1/n 4. Let N(i,t) be the number of times the Markov chain visits state i in t steps. Then, tim N&O ioe 6.3. Random Walks on Graphs Let G = (V,E) be a connected, non-bipartite, undirected graph where |V| = n and |E| = m, It induces a Markov chain Mg as follows: the states of the Mg are the vertices of G, and for any two vertices u, v € V, P, -{% if (uv) EE “(0 otherwise, where d(w) is the degree of vertex w. Because G is connected, Mg is irreducible. For a connected, undirected graph G, the periodicity of the states in Mg is the greatest common divisor (gcd) of the length of all closed walks in G, where a closed walk is any walk that starts and ends at the same vertex. As G is undirected, there are closed walks of length 2 that traverse the same edge twice in succession. Further, since G is non-bipartite it has odd cycles that give closed walks of odd length. It follows that the ged of the closed walks is 1, and hence ‘Mg is aperiodic. Noting that G is finite, Theorem 6.2 now implies that Mg has unique stationary distribution x. Lemma 6.3: For all v € V, x, = d(v)/2m. PRoor: Let [xP], denote the component corresponding to vertex v in the probability vector xP. Then, Pl = Smo 13263 RANDOM WALKS ON GRAPHS = ao Sie 2m” du) =x (coyee 2m = “Om a As a direct consequence of Theorem 6.2 and Lemma 6.3, we obtain the following lemma. Lemma 6.4: For all v € V, Iyy = 1/m = 2m/d(v). > Definition 6.10: The hitting time hy (sometimes called the mean first passage time) is the expected number of steps in a random walk that starts at u and ends upon first reaching o. > Definition 6.11: We define Cy», the commute time between u and v, to be Cy = Iw + hou = Cow This is the expected time for a random walk starting at u to return to w after at least one visit to v. > Definition 6.12: Let C,(G) denote the expected length of a walk that starts at u and ends upon visiting every vertex in G at least once. The cover time of G, denoted C(G), is defined by C(G) = maxyC,(G). > Example 6.1: A graph that tells us a great deal about the behavior of random walks is the n-vertex lollipop graph Ly (Figure 6.1). This graph consists of a clique on n/2 vertices, and a path on the remaining vertices. There is a vertex w in the clique to which the path is attached; let » denote the other end of the path. Figure 6.1: The lollipop graph La. By elementary probability (or using methods for studying random walks that we will encounter shortly), it turns out that in Ly, hue is O(r?), whereas hey is (rn). Thus, in general, hy # hw, and the asymptotic difference (as in this case) can be as much as a factor of n Another misconception that Ly dispels is that “adding more edges should help reduce the cover time C(G).” This is false, because L, has cover time O(n); on 133MARKOV CHAINS AND RANDOM WALKS. the other hand, it can be built by adding edges to a chain on n vertices, which can be shown to have cover time @(n?). In turn, the complete graph K, can be built by adding edges to Lp, and the cover time of K, is @(nlogn). Thus the cover time of a graph is not monotone in the number of edges. ‘The following lemma establishes an important property of the commute time across an edge and will prove useful in Section 6.5 below. Lemma 65: For any edge (u,b) € E, hay + hoy < 2m. PRoor: The proof considers a new Markov chain defined on the edges of G. The current state is defined to be the pair composed of the edge most recently traversed in the random walk, together with the direction of this traversal; equivalently, replacing each undirected edge by two oppositely directed edges, the directed edges form the state space. There are 2m states in this new Markov chain. The transition matrix Q for this Markov chain has non-zero entry Qusriem = Pow = 1/d(v), corresponding to an edge (v,w). This matrix is doubly stochastic, meaning that not only do the rows sum to one (as in every Markov chain), but the columns sum to one as well. To see this, fix a (directed) edge (v,w) and observe that the column sum corresponding to this state is given by LC Menem = YO Quoneem xeV, yer Ge) were) = DPw vero) = awxt de) aL Noting the result in Problem 6.6, it follows that the uniform distribution on the edges is stationary for this Markov chain, so the stationary probability of each directed edge is 1/2m. By part (3) of Theorem 6.2, we can conclude that the expected time between successive traversals of the directed edge (v,u) is 2m. Consider now hy + hou, and interpret this as the expected time for a walk starting from vertex u to visit vertex and return to u. Conditioned on the event that the initial entry into u was via the directed edge (v,u), we conclude that the expected time to go from there to v and then to w along (v,u) is 2m. ‘The memorylessness property of a Markov chain now allows us to remove the conditioning: since the sequence of transitions from u onward is independent of the fact that we arrived at u along (p,u) at the start of the commute, the expected time back to u is at most 2m. o ‘We emphasize that the result in Lemma 6.5 is valid only for vertices u and v that are connected by an edge in G 13464 ELECTRICAL NETWORKS 6.4. Electrical Networks Many random variables associated with the simple random walk on an undi- rected graph are studied conveniently using the tools and the language of electrical network theory. Our focus here will be on characterizing hy and Cy in terms of properties of the graph G. We begin with a review of some basics of resistive electrical networks. A resistive electrical network is an undirected graph; each edge has associated with it a positive real branch resistance. The flow of current in such networks is governed by two rules: Kirchhoff’s Law and Ohm's Law. Kirchhoff’s Law stipulates that the sum of the currents entering a node in the network equals the sum of the currents leaving it. Ohm’s Law states that the voltage across a resistance equals the product of the resistance and the current through it. 6 = Figure 6.2: A resistive electrical network. Each rectangle signifies a branch resistance. Consider the simple example in Figure 62. If a current of one ampere were injected into node b and removed from node ¢ in this network, a simple calculation using Kirchhoff's Law and Ohm’s Law yields the following: half an ampere of current flows along the branch bc, and the other half ampere through branch ba and onto ac. The voltage difference between c and b is one volt, while the voltage difference between c and a (and between a and b) is half a volt. One final notion we need is that of the effective resistance between two nodes in a network. The effective resistance between two nodes u and v is the voltage difference between u and v when one ampere is injected into u and removed from v; equivalently, one ampere could be injected into v and removed from u. The effective resistance between u and v is always at most the branch resistance between u and v and can be much less, as we shall see. This distinction between branch and the effective resistances is important. In the example in Figure 6.2, for instance, the effective resistance between b and c is 1, whereas the branch resistance is 2. Given an undirected graph G, let V(G) be the electrical network defined as follows: it has a node for each vertex in V; for every edge in E, it has a one ohm resistance between the corresponding nodes in \V(G). For two vertices u,v € V, Ra denotes the effective resistance between the corresponding nodes in N/(G). The following theorem establishes a close relation between commute times for the simple random walk on G and effective resistances in the electrical network Ni). Theorem 6.6: For any two vertices u and v in G, the commute time Cy = 2mRuy- 135MARKOV CHAINS AND RANDOM WALKS proor: For a vertex x in G, let T(x) denote the set of vertices in V that are adjacent to x, and let d(x) denote its degree |I(x)|. Let .» denote the voltage at u in N(G) with respect to o, if d(x) amperes of current are injected into each node x € V, and 2m amperes are removed from v. We will first prove that for allue V, haw = dw (61) Using Kirchhoff’s Law and Ohm’s Law, we obtain that for all u € V \ {v}), Au) = SO (bee — bw. (62) “rw By the definition of expectation, for all w € V \ {v}), 1 Iw = YO ay (+ hw). (63) Jui) Equations (6.2) and (6.3) are both linear systems with unique solutions; further- more, they are identical if we identify $4. in (6.2) with hy in (6.3). This proves (6.1). To complete the proof of the theorem, we note that hy, is the voltage dex at v in M(G) measured with respect to u, when currents are injected into all nodes and removed from u. Changing signs, 4.. is now the voltage at u relative to v when current is injected at u, and removed from all other nodes. Since resistive networks are linear, we can determine Cj, by super-posing (taking care with the sign!) the networks on which gy and $., are measured. Currents at all nodes except u and v cancel, resulting in Cw» being the voltage between u and when Sy 1, it proceeds at the next step to i—1 or to / +1 with equal probability. Show that the expected number of steps that elapse before the walk first reaches n is (n—1)?. Exercise 6.5: Prove Theorem 6.1. Why does the bound of O(n?) steps hold only for finding some satisfying assignment, rather than the specified assignment A? What happens if each clause has 3 literals rather than 27 The effective resistance between two nodes u and v is at most the length of the shortest path between them in G. This observation yields an alternative proof of Lemma 6.5. The length of the shortest path between any two vertices of G is at most the diameter of G. We thus have the following corollary, which by Example 6.1 is asymptotically tight. 13665 COVER TIMES Corollary 6.7: In any n-vertex graph, and for all vertices u and v, Cu Cow = (oner Since the vertices vj, v4; are adjacent for all j, we have by Lemma 6.5 that Cayo S 2m, Since there are n — 1 edges in T, C,,(G) < 2m(n — 1). But this upper bound holds no matter which vertex of G we designate to be the starting point yp in the traversal; therefore C(G) < 2m(n — 1) a Note that Theorem 6.8 gives (asymptotically) the right answer for the lollipop graph: C(L,) is ©(n°). On the other hand, it gives the same O(n) upper bound for the complete graph K,, whereas we have already seen (Exercise 6.1) that C(K,) is @(nlogn). Theorem 6.8 can be slack for some graphs: in the proof, we measure the time for the vertices of G be visited in one specific order. In fact, we can often refine the upper bound on cover time as follows. Let R(G) = maxucey Rw; We call R the resistance of G. The resistance of a graph characterizes its cover time fairly tightly: Theorem 6.9: mR(G) < C(G) < 2e?mR(G)Inn +n. PRooF: The proof of the lower bound follows from the fact that there exist vertices u,v such that R(G) = Rw and max(hy, hy.) 2 Cw/2; the bound then follows from Theorem 6.6. 137MARKOV CHAINS AND RANDOM WALKS. For the upper bound, we will show that the probability that all the vertices are not visited within 2e’mR(G)Inn steps is at most 1/n?; this, together with Corollary 6.7 will yield the result. Divide the random walk of length 2e'mR(G)Inn into Inn epochs each of length 2e*mR(G). For any vertex v, the hitting time hy is at most 2mR(G), regardless of the vertex u at which an epoch starts. By the Markov inequality, the probability that v is not visited during any single epoch is at most 1/e’. Thus, the probability that v is not visited during any of the Inn epochs is at most 1/n’. Summing this probability over the n choices of the vertex v, the probability that any vertex is not visited within 2e'mR(G) Inn steps is at most 1/n?, When this happens (there is a vertex that has not been visited within 2e?mR(G)Inn steps), we continue the walk until all vertices are visited, and n> steps suffice for this (by Corollary 6.7). Thus the expected total time is at most 2e?mR(G)Inn-+ (1/n?)n? = 2e*mR(G) Inn +n. a The bounds in Theorem 6.9 cannot in general be improved; the upper bound is tight (within constant factors) for the complete graph (Problem 6.10 below) and the lower bound is tight for the chain on n vertices. ‘There are also graphs for which neither bound of Theorem 6,9 is tight. Note that Theorem 69 gives an estimate for the cover time that is tight to within a factor of logn. This is because effective resistances in a graph (and therefore the resistance of the graph, R(G)) can be computed efficiently using matrix inversions. Note also that neither Theorem 6.8 nor 6.9 is universally superior; we have already seen that for the complete graph K,, Theorem 6.8 gives a loose upper bound. For the lollipop graph L,, Theorem 6.9 gives an upper bound of O(n? log n), which is an overestimate by a factor of log n. Often, we are interested not so much in determining the cover time of a single graph, as in bounding the cover times of a family of graphs. A simple fact that is of great use in bounding effective resistances in electrical networks is following Rayleigh's Short-cut Principle: Effective resistance is never raised by lowering the resistance on an edge (eg, by “shorting” two nodes together), and is never lowered by raising the resistance on an edge (eg, by “cutting” it). Similarly, resistance is never lowered by “cutting” a node, leaving each incident edge attached to only one of the two resulting halves of the node. A second useful fact about effective resistances in an electrical network is that they obey the triangle inequality. As one very simple application of these facts, observe that in a graph with minimum degree d, R > 1/d: short all vertices except the one of minimum degree. Another simple application is the following Jemma. 13866 GRAPH CONNECTIVITY Lemma 6.10: Suppose that g contains p edge-disjoint paths of length at most ¢ from s to t. Then, Ra <¢/p. 6.6. Graph Connectivity We are now ready for our first algorithmic application of random walks. Two vertices in an undirected graph G are said to be connected if there exists a path between them. A connected component of G is a (maximal) subset of vertices in which every pair of vertices is connected. 6.6.1. Undirected Graphs ‘The undirected s-t connectivity (USTCON) problem is the following: given an undirected graph G and two vertices s and t in G, decide whether s and ¢ are in the same connected component. The USTCON problem is important in the study of space-bounded complexity classes and is a natural abstraction of a number of graph search problems. It is easy to see that a standard graph search algorithm such as depth-first search solves the problem in O(m) steps. In doing so, the algorithm keeps track of all the vertices of G that the search has visited and, therefore, uses workspace at least linear inn A probabilistic log-space Turing machine for a language L is a probabilistic Turing machine using space O(log n) on instances of size m, and running in time polynomial in n. We say that a language (equivalently, a decision problem) A is in RLP if there exists a probabilistic log-space Turing machine M such that on any input x, 21/2 xed 0 x¢A. Here space O(logn) refers to the workspace of the Turing machine; the input is given on a read-only tape, and the only storage available to it with write-access is a log-space tape. Pr[M accepts x] { (64) Theorem 6.11: USTCON € RLP. ProoF: The log-space probabilistic Turing machine simulates a (simple) random walk of length 2n? through the input graph, starting from s. If it encounters the vertex t in the course of this walk, it outputs YES; otherwise it outputs No. Clearly the machine will never output yes on an instance of USTCON in which sand 1 are not in the same connected component. What is the probability that it outputs No when it should have said ves? By Theorem 6.6, hy < n°. By the Markov inequality, if t is in the same component of G as s, the probability that it is not visited in a random walk of 2n? steps starting from s is at most 1/2. The Turing machine uses its workspace 139