Introduction to dynamical large deviations of Markov processes∗

Hugo Touchette
National Institute for Theoretical Physics (NITheP), Stellenbosch 7600, South Africa and
Institute of Theoretical Physics, Department of Physics,
arXiv:1705.06492v2 [cond-mat.stat-mech] 25 May 2017

University of Stellenbosch, Stellenbosch 7600, South Africa
(Dated: May 26, 2017)
These notes give a summary of techniques used in large deviation theory to study the
fluctuations of time-additive quantities, called dynamical observables, defined in the context
of Langevin-type equations, which model equilibrium and nonequilibrium processes driven
by external forces and noise sources. These fluctuations are described by large deviation
functions, obtained by solving a dominant eigenvalue problem similar to the problem of
finding the ground state energy of quantum systems. This analogy is used to explain the
differences that exist between the fluctuations of equilibrium and nonequilibrium processes.
An example involving the Ornstein-Uhlenbeck process is worked out in detail to illustrate
these methods. Exercises, at the end of the notes, also complement the theory.

I. INTRODUCTION

My aim in these notes is to give an introduction to the large deviation techniques used to
calculate the probability distribution of physical quantities, called dynamical observables, which
are time-integrated functionals of Markov processes. These observables have come to play in the
last years an important role in the context of nonequilibrium systems, and the recent discovery of
fluctuation symmetries (also called fluctuation relations) generally satisfied by these systems. They
also appear naturally when defining energy-like quantities, such as work and heat, in the context
of noisy systems modelled by Markov processes, a field of research commonly referred to now as
stochastic thermodynamics (see Seifert’s contribution to this volume).
The results covered are known in large deviation theory, but are not widespread, certainly not
in statistical physics, and cannot be found all put together in a single reference. The invitation to
lecture at the FPSP School (to which I participated as a student in 2005) offers a good opportunity
to summarize them by appealing to what physics students know best – quantum mechanics.
The link with quantum mechanics is a natural one because all large deviation calculations
related to time-additive functionals of Markov processes reduce, when one does not consider the
low-noise limit, to an eigenvalue calculation, which is very close in spirit to finding the energy levels
of a quantum system and, particularly, its ground state energy. For this reason, it is not surprising
to see many techniques of quantum mechanics (such as the Bethe ansatz and the density matrix
renormalization group) being applied to study the fluctuations of nonequilibrium processes. In a
way, if you know quantum mechanics, you also know Markov processes.
Compared to quantum systems, however, there is a fundamental difference that arises when
dealing with Markov processes, namely, that the operator or matrix that we must consider to
study their steady state and fluctuations is, in general, not hermitian. This raises many technical
but important questions, which are precisely the questions that are hard to find in the literature,
such as:
• What can we say in general about the spectrum of non-hermitian operators?
• Under which conditions is that spectrum real?
• Which function space must we use to solve the eigenvalue problem?

Lecture notes for the 2017 Summer School on Fundamental Problems in Statistical Physics XIV, 16-29 July 2017,
Bruneck (Brunico), Italy.

Typeset by REVTEX

as always. for clarity. For the purpose of these notes. that there is or will be a general theory of these systems (not at least in the way many physicists picture it). by having F (Xt . we consider here m = n. but it is also possible to have Wt ∈ Rm with m > n or m < n. Stochastic differential equations We consider throughout these notes Markov processes defined by the following stochastic differential equation (SDE): dXt = F (Xt )dt + σdWt . but we do not consider this possibility here. We call this function the force or the drift of the system. such a pedagogical goal cannot be achieved in 20-odd pages of notes (a book is in progress). With this. For Wt ∈ Rn . in fact. II. 2 • What are the boundary conditions for the eigenfunctions? • What is their normalization? • What distinguishes equilibrium from nonequilibrium processes in terms of fluctuations. and I do not believe. The SDE above governing the evolution of Xt (as an infinitesimal difference equation) is called in mathematics an Itô SDE. but at the same time so much more interesting. (2) . so I decided to focus here on Langevin-type equations modelling noisy systems driven by external forces and baths. one should keep in mind that all the results discussed can be applied more generally (sometimes with minor modifications) to a large class of Markov processes. dy- namical or otherwise? The fact that we have to deal with non-hermitian operators is the main reason why studying nonequilibrium processes is more challenging than studying equilibrium processes. t). whose increments dWt are Gaussian-distributed with zero mean and variance dt. Obviously. (1) where • Xt is a vector in Rn representing the state of the system at the time t. We are far now from completely understanding nonequilibrium systems. we can consider Xt to be defined equivalently by a noisy ordinary differential equation (ODE) having the more common form Ẋt = F (Xt ) + σξt . sacrificing mathematical rigor. it is an n × n matrix. But I do believe that having a good mathematical grounding in the subject – at least as good as the one we expect in quantum mechanics or any modern physics subject – is necessary for making progress. Some references and exercises on those are given in the text. so that Wt ∈ Rn . It can explicitly depends on time. MARKOV PROCESSES A. In many applications. Note that we do not use bold symbols to represent vectors. For simplicity. assumed here not to depend on Xt to simplify the discussion. including Markov chains and Markov jump processes. Wt is taken to have as many components as Xt . for example. • Wt is a vector of independent Brownian or Wiener motions. • σ is the noise matrix. • F : Rn → Rn is a vector field that drives the deterministic evolution of Xt when there is no noise (σ = 0). That matrix has dimensions n × m to match the dimensions of Xt and Wt .

To emphasize that this partial differential equation (PDE) is linear. starting from an initial distribution p(x. (3) where i and j denote specific components of ξ(t) and h·i denotes the expectation or mean (with respect to whatever random variable or process found in the brackets). hξti ξtj′ i = δij δ(t − t′ ). as long as one is aware that the rules for calculating derivatives and integrals of stochastic processes differ slightly from those of normal calculus. called the covariance matrix. t) + ∇ · Jt (x) = 0. as is well known. Some important differences between stochastic and normal calculus will be pointed out along the notes. (4) 2 which involves the symmetric matrix D = σσ T . For background references on SDEs and stochastic calculus. I assume in these notes that the reader knows that Brownian motion is nowhere differentiable. t). t). we can express it as ∂t p(x. For the general SDE shown in (1). When the process Xt is ergodic. which explains why it is preferable to express a diffusion process in the Itô rather than in the noisy ODE form. t). I also assume some familiarity with stochastic calculus. (8) This form is useful for interpreting the nature of the stationary density ps (x) satisfying L† p s = 0 (9) or equivalently ∇ · Js = 0. 0) that represents the distribution with which we sample the initial state X0 . t) = −∇ · (F (x)p(x. The Fokker-Planck equation can also be rewritten. t)) + ∇ · D∇p(x. 3 where ξt is an n-dimensional Gaussian white noise corresponding formally to the time-derivative of Wt and defined by the properties hξt i = 0. B. t) = L† p(x. that distribution is known to be given by the Fokker-Planck equation. (7) 2 called the Fokker-Planck current. The analogy with the Schrödinger equation of quantum mechanics should be obvious. t) − ∇p(x. t) → ps (x) from any initial density as t → ∞. . see the Further reading section. (5) where 1 L† = −∇ · F + ∇ · D∇ (6) 2 is a linear differential operator called the Fokker-Planck generator. p(x. t). as ∂t p(x. State distribution and generator The first natural problem to consider when studying a Markov process is to determine its probability distribution P (Xt = x) = p(x. as a conservative equation involving the (vector) probability current D Jt (x) = F (x)p(x. though this is not essential. 1 ∂t p(x. This shows that ps (x) is the eigenfunction of L† with eigenvalue 0. where Js is the stationary current associated ps .

not the position. as opposed to the Schrödinger picture which describes the evolution of probabilities. heat baths and noise sources in general. C. (16) where c is a normalization constant (see Exercise 3). Indeed. and is equal here to 1 L = F · ∇ + ∇ · D∇. acts on the function f rather than the density p. II F. • Gradient SDEs: dXt = −∇U (Xt )dt + σdWt (15) with σ proportional to the identity matrix. • Overdamped Langevin equation: dqt = Γ−1 (−∇V + φt )dt + p 2Γ−1 /β dWt . Examples There are many standard SDEs used in physics to model noisy systems driven by forces. it can be proved using the natural inner product defined by the expectation (see Appendix A) that ∂t hf (Xt )i = h(Lf )(Xt )i. (12) 2 This result is much less known in physics than the Fokker-Planck equation. see the reading list in Sec. . that is. σ = ε11. t)f (x)dx. it is useful to note that the Fokker-Planck equation also determines the evolution of expectations of Xt having the general form Z hf (Xt )i = p(x. (11) where L is the adjoint of L† (see Appendix A). V (q) is a potential. to the Heisenberg picture of quantum mechanics which describes the evolution of observables. φt is an external force. 4 For the next sections. The stationary density of this SDE is the Gibbs distribution 2 ps (x) = c e−2U (x)/ε . pt the momentum. The generator will become useful for treating large deviations. • Kramers or underdamped Langevin equation: pt dqt = dt m  pt  p dpt = −∇V (qt ) + φt − Γ dt + 2Γβ dWt . (14) This is an SDE for the position obtained by taking the overdamped (m → 0) limit of Kramers equation. (13) m where qt is the position. and β is the inverse temperature of the thermal noise. This operator. external reservoirs. The noise acts as a force in Newton’s equation and so only affects the momentum. It corresponds. in a way. which is simply called the generator of Xt . The following is a representative list – for more examples. as is explicit from the notation above. (10) where f is any smooth function of the process. though it is as important.

Comparison with quantum mechanics It is useful at this point to reflect on the structure of Markov diffusions. equivalently. the central object in both theories is the generator of that evolution. . E. D. then the process is nonequilibrium. is that a process is an equilibrium process if the probability of any given trajectory is the same as the probability of that trajectory reversed in time. Both are linear theories of evolution for a “vector” corresponding to distribution p(x. especially the linear form of the Fokker-Planck equation and its generator. which is the Fokker-Planck operator L† for Markov processes and the Hamiltonian H for quantum systems. t) for quantum systems. it has a stationary distribution and its initial condition X0 is drawn according to that distribution. Comparison of equilibrium and nonequilibrium (ergodic) Markov processes. that is to say. assuming that the process Xt is stationary. If that is not the case. From this table. As a result. This is obviously a gradient SDE with quadratic potential U (x) = γx2 /2 having a Gibbs stationary distribution. all of these notions are related to the eigenvalues of the generator L. essentially. For Markov processes. We summarize these connections in Table I. so we only summarize them. Moreover. detailed balance. as announced in the introduction. which is itself related (in most cases) to having a vanishing probability current in the Fokker-Planck equation (see Exercise 2). t) for Markov processes and the wavefunction ψ(x. by comparing it to what we know about the evolution of quantum systems – see Table II. • Ornstein-Uhlenbeck process: dXt = −γXt dt + σdWt (18) with Xt ∈ R and Wt ∈ R. • Linear diffusions: dXt = −M Xt dt + σdWt . It would take too much space to fully explain these notions. Equilibrium versus nonequilibrium processes The distinction between equilibrium and nonequilibrium system in the context of stochastic processes is based on the notion of time reversibility or. (17) where M is an n × n matrix assumed to be positive definite (positive eigenvalues) in order for Xt to have a stationary density (see Exercise 4). The idea. 5 Equilibrium Nonequilibrium Reversible Non-reversible Js (x) = 0 Js (x) 6= 0 although ∇ · Js = 0 Spectrum of L real Spectrum of L complex TABLE I. it can be proved that this definition of equilibrium in terms of “forward” and “backward” trajectories is equivalent to the notion of detailed balance. you should see that the theory of Markov processes has much in common with quantum mechanics.

a property deeply related to whether we are dealing with an equilibrium or nonequilibrium process. ψi = hψ|ψi Duality hp. t)|2 Evolution Fokker-Planck Schrödinger Generator L H (hamiltonian) † Propagator U (t) = eL t U (t) = e−iHt/~ Inner product hp. In the last 20 years or so. it should be compulsory reading for any serious students in statistical physics. that “classical” Markov processes are more general than quantum systems. unlike the hamiltonian. Comparison between Markov processes and quantum mechanics. • Theory of Markov processes focusing more on Markov chains and jump processes (also called continuous-time Markov chains): A very good reference. f i hψ. (19) 0 where the circle ◦ indicates that the integral is to be calculated using the midpoint Riemann integral rule. the generator of Markov processes is not always self-adjoint. in general. This means. with eigenvectors or eigenfunctions (see Appendix B). . since they contain as a special case self-adjoint generators! F. • Fokker-Planck equation: [7]. Further reading • SDEs and Langevin equations: [1].1 This random variable depends obviously not only 1 The work is a scalar quantity. T ]. It also means. • Stochastic calculus: [1] and [4] for the theory. Though a maths textbook. also called the Stratonovich convention. An example is the mechanical work done by the force F on Xt over a time interval [0. For a leaner textbook. its average. in addition to its variance and covariance. f i hψ. DYNAMICAL OBSERVABLES The study of Markov processes in physics has focused a lot in the past on the statistical proper- ties of the state Xt – its distribution. 6 Markov Quantum State Xt |ψ(t)i or ψ(x. so the product F (Xt ) ◦ dXt is also a scalar product. Af i = hA† p. for its scope. see [2] and especially [3]. For more technical yet readable presentations. The main difference between the two sides is that. Aψi = hAψ. • Stochastic processes with applications in physics: [8]. see [2]. t) |ψ(x. is [6]. clarity and number of exercises. as calculated by Z T WT = F (Xt ) ◦ dXt . researchers have also become interested in the statistics of time-integrated quantities involving Xt . t) Distribution p(x. which can be related to diffusion and transport coefficients [9]. in practice. III. ψi = hψ|A|ψi Self-adjoint? Not necessarily Always TABLE II. and [5] for simulations. that we have to be careful when diagonalizing matrices or operators and dealing. in a funny way.

to make AT intensive in time and. but only if the integral is interpreted with the Stratonovich convention. • Entropy production: Z T ΣT = 2 (D −1 F (Xt )) ◦ dXt . corresponding to the left-point Riemann rule. It is more aptly called the empirical current because it represents a flow at each point x that converges in probability to the stationary Fokker-Planck current Js (x) as T → ∞ (see Exercise 8). T → ∞. more physically. for different (random) trajectories. The Stratonovich rule is also important for this convergence. (21) T 0 This random function represents the fraction of time in [0. the work is the change of potential. a dynamical observable. related to the non- reversibility (or nonequilibrium nature) of Xt and. For this reason. This choice of convention is actually not important – we could use the Itô convention instead with a slight modification of the results that will come next. and ◦ denotes as before the Stratonovich (scalar) product. • Empirical current: T 1 Z JT (x) = δ(Xt − x) ◦ dXt . we say that ρT converges in probability to ps as T → ∞. . following the examples of the empirical density and empirical current. (23) 0 This is an important quantity in the theory of nonequilibrium systems. as is clear from its definition. (24) T 0 T 0 where f (scalar) and g (vector) are two arbitrary functions that depend on the system and phys- ical quantity considered. WT is often called an additive functional of the process or. that AT converges in probability to a constant (its mean) in the long-time limit. From these examples. it converges to the stationary density ps for almost all trajectories. What is more important is the factor 1/T which is there for two reasons: first. This explains why the work WT must also be defined in the Stratonovich convention: for a potential force. For an ergodic process. second. We list next other physical examples: • Potential energy: The change in time of potential energy can be written as Z T ∆UT = U (XT ) − U (X0 ) = ∇U (Xt ) ◦ dXt . to guarantee. T ] that the process Xt “takes” the value x. but on the whole trajectory of this process between t = 0 and t = T . also to the work WT [10]. Thus. (20) 0 This holds whether the SDE of Xt is gradient or not. Mathematically. there would be additional terms in the integral. it is natural to consider a general class of dynamical observables having the form 1 T 1 T Z Z AT = f (Xt )dt + g(Xt ) ◦ dXt . In the Itô convention. if we view dXt formally as Ẋt dt. one typically gets different (random) work values. • Empirical distribution: T 1 Z ρT (x) = δ(Xt − x) dt. 7 on the state Xt at time t. (22) T 0 This random field represents intuitively the local mean “velocity” of Xt at x.

in its simpli- fied version. The meaning of this approximation. there is usually only one point a∗ where I(a) = 0. for additive functionals of Markov processes. In general. The theorem says. is that the dominant contribution of P (AT = a) is a decaying exponential. In particular. it is not a parabola. In general. it must concentrate on a∗ by conservation of probability. (26) T →∞ T known as the scaled cumulant generating function (SCGF). however. LARGE DEVIATIONS A. This is not an easy task to carry in many applications. for example. Spectral problem The Gärtner-Ellis Theorem is based on the following function: 1 λ(k) = lim lnheT kAT i. which is called the large deviation principle (LDP). • For Markov processes. except for values a such that I(a) = 0. B. 8 IV. • The zero of I(a) is a reflection of the Law of Large Numbers: it gives the most probable or typical value of AT in the long-time limit and coincides with the mean of AT . to calculating a dominant eigenvalue of a linear operator. In an experiment where the work WT per unit time would be measured. one would see that most trajectory are “pulled” by doing work close to its mean value hWT i. • The rate function I(a) characterize the rare fluctuations of AT around this typical value. These properties will become clear once we start calculating rate functions for specific processes and observables. We describe next the most common way of obtaining rate functions using a result of large deviation theory known as the Gärtner-Ellis Theorem. so P (AT = a) decays exponentially fast with T . and so that any corrections to that contribution is sub-exponential in T . but it is definitively simpler than calculating the exact distribution of AT . it is known that is has the following general asymptotic form: P (AT = a) ≈ e−T I(a) (25) in the limit of large observation time T → ∞. In many cases. This is the most important point about large deviations – the fact that it characterizes fluctuations beyond Gaussian fluctuations and so beyond the Central Limit Theorem. so if P (AT = a) decays where I(a) > 0. • I(a) ≥ 0 for all a. it is very difficult to obtain that distribution exactly. that if λ(k) exists for k ∈ R and is differentiable in k. which essentially boils down. so fluctuations of dynamical observables are in general not Gaussian. Large deviation principle Our goal now is to study the probability distribution P (AT = a) of a given dynamical observable AT and Markov process Xt . The exponent or rate function I(a) is therefore the essential information that we need to find in order to characterize the fluctuations of AT . Only rarely would we see trajectories that require more or less work than this average. then .

compared to quantum mechanics. and then study the asymptotic evolution of that PDE to realize that it is dominated by the dominated eigenvalue of Lk . (29) 2 where f and g are the functions entering in AT (see Appendix C) and Exercise 9). which turns out to be Z rk (x) lk (x) dx = 1 (33) . 2) Its rate function I(a) is the Legendre-Fenchel transform of λ(k): I(a) = max{ka − λ(k)}. (27) k∈R The question now is. by requiring overall the following boundary condition: |x|→∞ rk (x)lk (x) −→ 0. so its distribution has the scaling form (25). To be more precise. (30) where rk (x) is the “right” eigenfunction associated with the dominant eigenvalue λ(k). (31) where lk (x) is the corresponding “left” eigenfunction. namely. except that Lk is not in general a hermitian operator. In fact. For the SDE (1) and the observable AT defined in (24). This duality also determine the correct normalization to use for the eigenfunctions. 9 1) AT satisfies a large deviation principle. λ(k) = ζmax (Lk ). These steps are presented in Appendix C and lead to the main result of these notes. how do we calculate λ(k)? For this. guaranteeing that λ(k) is real. known as the Feynman-Kac formula. We must also solve in parallel the dual eigenvalue problem L†k lk (x) = λ(k)lk (x). Therefore. called the tilted generator. we see that. which involves some linear operator Lk . (28) where ζmax (Lk ) denotes the dominant eigenvalue of Lk . we must now not only consider the eigenvalue problem Lk rk (x) = λ(k)rk (x). contrary to quantum mechanics. and applies essentially whenever the spectrum Lk is gapped. The resulting dominant eigenvalue problem is similar to a quantum eigenvalue problem. This is an important precision (see Exercise 17). in short. This makes the calculation of λ(k) more complicated. to cast the evolution of the generating function of AT in a linear PDE similar to the Fokker-Planck equation. we cannot just solve the direct eigenvalue problem for Lk by requiring that rk (x) alone decays to 0 at infinity – we must consider the direct and dual eigenvalue problems at the same time with (32) to find the correct dominant eigenvalue. is that the duality between Lk and L†k is equivalent to performing integration by parts (Appendix A) and rk (x)lk (x) is the boundary term in that integration that must vanish at infinity. This result is a PDE generalization of the Perron-Frobenius Theorem about positive matrices. we can use another result of probability theory. Note that Lk=0 = L and so λ(0) = ζmax (L) = 0. λ(k) is equal to ζmax (Lk ) whenever λ(k) exists as a SCGF. the tilted generator is explicitly given by 1 Lk = F · (∇ + kg) + (∇ + kg) · D(∇ + kg) + kf. (32) The reason for this.

the two are related together (see Exercise 13) by p ψk (x) = ps (x)rk (x). (39) where ψk is the eigenfunction of Hk associated with λ(k). The eigenvalue λ(k) is. this is indeed possible and the unitary transformation or symmetrization that we must use is given by 1 √ Hk = ps−1/2 Lk p1/2 s = √ L k ps . . Our eigenvalue problem thus reduces to Hk ψk = λ(k)ψk . Note that this is an operator transformation: when applied to a function φ. In general. when dealing with gradient SDEs and observables such that g = 0. to have a real spectrum even though it is not hermitian. 10 (see Appendix B). up to a minus sign.2 This happens. not a necessary condition. of course. For a gradient SDE defined in (15) and an observable AT such that f 6= 0 but g = 0. Hk acts as follows: Hk φ = ps−1/2 (Lk p1/2 s φ). C. and |∇U (x)|2 ∆U (x) Vk (x) = 2 − − kf (x) (38) 2ε 2 is an effective quantum-like potential. (36) 1/2 −1/2 so that Lk is applied to the product ps φ. are different from the eigenfunctions rk of Lk . we also impose Z lk (x) dx = 1. the same for Lk and Hk since these two operators are unitarily related. For the case of gradient SDEs mentioned. (35) ps where ps (x) is the Gibbs stationary distribution of the SDE shown in Eq. in many cases. Symmetrization The tilted generator Lk is known. The sign difference means that what we find as the dominant (largest) eigenvalue of Hk corresponds to the ground state (smallest) energy of −Hk . In this case. with ε2 = ~2 /m and potential Vk . so both normalization conditions reduce to ps (x)dx = 1 for k = 0. it is not difficult to see by direct calculation (see Exercise 12) that Hk has the form ε2 Hk = ∆ − Vk . For convenience. for example. (16) (see also Exercise 3). however. We recognize in this equation the time-independent Schrödinger equation. The resulting function is then multiplied by ps . (37) 2 where ∆ = ∇2 is the Laplacian. (34) Note R that lk=0 = ps and rk=0 = 1 (see Exercise 10). The eigenfunctions ψk of Hk . ε is the noise amplitude of the SDE. (40) 2 Being hermitian is only a sufficient condition for having a real spectrum. it should be possible to transform Lk to a hermitian operator Hk in an unitary way (so as to preserve the spectrum) and then work only with Hk using techniques from quantum mechanics to find λ(k).

This symmetrization method greatly simplifies. Finally. then Lk can be symmetrized. the calculation of large deviation functions. A natural question is. Moreover. it can be verified that p ψk (x) = lk (x)/ ps (x). it also implies that we can now focus on only one eigenvalue problem with natural (quantum) boundary conditions imposed only on ψk . then Lk cannot be symmetrized because L itself cannot be symmetrized. To obtain the rate function. Spectral problem for equilibrium and nonequilibrium processes. if Xt is reversible but g is not gradient. to which class of Markov processes can it be applied to besides gradient SDEs? Table III gives some answers. (43) In a more practical way. if we consider an equilibrium process and an “equilibrium-type” observable characterized by g = 0 or g gradient. The basic idea is that. k ∈ R. (42) which is the normal quantum boundary condition (modulo a complex conjugate). If Xt is a nonequilibrium process. Our goal is to find the rate function I(a) characterizing the fluctuations of the following observable: 1 T Z AT = Xt dt. while the nor- malization condition in (33) reduces to Z ψ(x)2 dx = 1. (45) dx 2 dx2 . Example: Ornstein-Uhlenbeck process We close these notes by applying the material of the previous sections to a specific example involving the Ornstein-Uhlenbeck process defined in (18). and which can be related physically to the mechanical work performed by a laser tweezer on a Brownian particle immersed in water [11]. obviously. (41) This is interesting because it implies that the boundary condition (32) that we have for the full eigenvalue problem now reduces to |x|→∞ ψ(x)2 −→ 0. (44) T 0 which represents mathematically the area under the paths of the process. noting in this case that f (x) = x and g = 0: d σ 2 d2 Lk = L + kf = −γ + + kx. 11 Xt L symmetrizable? g Lk symmetrizable? Reversible Yes 0 Yes Gradient g = ∇ϕ Yes Non-gradient No Non-reversible No Any No TABLE III. D. then Lk generally cannot be symmetrized because the fluctuations of the observable that we consider are essentially created in a nonequilibrium way (see Exercise 21). we first write down the tilted generator associated with this process and observable.

but since Xt is gradient (any SDE on R is gradient) and g = 0. note that rk=0 (x) = 1 while lk=0 (x) = ps (x). (52) πε2 4ε2 This clearly shows that rk (x) or lk (x) do not decay to zero as x → ±∞ – it is their product again that does so. For the quantum oscillator. Finally. (48) 2γ 2 This exists and is differentiable for all k ∈ R. it is instructive to compute them. The exercises found at the end of these notes extend this simple calculation to other equilibrium- type observables. it is well known that the eigenfunctions are given in terms of Hermite polynomials. so we can the apply the Legendre-Fenchel transform (27). a result consistent with the fact that linear integrals of Gaussian processes are Gaussian. which in this case reduces to a simple Legendre transform (why?). For more information about nonequilibrium large deviations. is therefore found to be ε2 k2 λ(k) = . it can be symmetrized with its (Gaussian) Gibbs distribution r γ −γx2 /ε2 2 ps (x) = e ∝ e−2U (x)/ε (46) πε2 to ε2 d2 γ 2 x2 γ Hk = − + + kx. Although we do not need the eigenvectors rk (x) and lk (x) in the calculation of the SCGF and rate function. we find here 2 !  γ 1/4 γ x − ε2 k/γ 2 ψk (x) = exp − . and applying the normalization conditions (33) and (34). 12 This operator is not hermitian. (47) 2 dx2 2ε2 2 We recognize the hamiltonian of the 1D quantum harmonic oscillator if we multiply by −1 and shift the space to x → x + ε2 k/γ 2 . (49) 2ε2 This shows that the fluctuations of AT are Gaussian around the typical value AT = 0. in agreement with (32). . (50) πε2 2ε2 Using the transformations (40) and (41). to finally obtain γ 2 a2 I(a) = . The SCGF. and the current research done on this topic. see the pointer references next. before slowly moving into the realm of nonequilibrium processes. Considering only the ground state. we then find kx 3ε2 k2   rk (x) = exp − (51) γ 4γ 3 and 2 ! γ 2x − ε2 k/γ 2 r γ lk (x) = exp − . as pointed out before (see also Exercise 10). corresponding to minus the known ground state of the quantum oscillator (see any textbook on quantum mechanics).

(A4) which leads to (∇)† = −∇ for the gradient and (∆)† = ∆ for the Laplacian. 22]. • Fluctuations of interacting particle systems modelling particle and energy transport: [17]. we say that L† is the adjoint of L with respect to the Lebesgue measure. t)f (x) dx (A1) of a function of Xt defines the following natural scalar or inner product in the theory of Markov processes: Z hp.) Applying an operator either on f or on p leads us to define the notion of dual or adjoint operator: if L acts on f . • Low-noise large deviations: [12]. 13]. the duality between L and L† simply corresponds to performing integration by parts: Z Z Z hp. [18]. For differential operators. (See Table II for a comparison of this inner product and the one used in quantum mechanics. we must then have (4).5 of large deviations): [23]. Appendix A: Dual spaces for Markov processes The expectation Z hf (Xt )i = p(x. 13 E. 20]. the Lebesgue measure. • Large deviation simulations: [24]. Further reading • Theory of large deviations and applications in statistical physics: [12. and [19]. (A2) which connects the space of normalized probability densities and the space of functions of Xt . . • Recent theory of fluctuation processes explaining. (A3) Because the integral defining the inner product is performed with dx. The Fokker-Planck equation also derives from this duality by noting from (11) that ∂t hf (Xt )i = hpt . then its adjoint L† acts on p according to hp. if we choose p and f such that the boundary term (usually at infinity) vanishes. • Fluctuations of empirical density and current (level 2. • Fluctuation relations and symmetries: [10. (A5) Since this holds for any test functions. ∇f i = p∇f dx = pdf = pf |boundary − f dp. Lf i = hL† pt . f i. • Entropy production: [10]. • Stochastic thermodynamics: [16]. f i = p(x)f (x) dx. 15]. how large deviations are created in time: [21. also called test functions or observables. Lf i = hL† p. f i. with modified SDEs. • Quantum approach to large deviations: [14.

(C1) where U (t) = e tL† is an operator acting on the initial density. The only difference is that. the dual eigenvalue problem L† u(x) = βu(x) (B2) has a spectrum that is the complex conjugate of the spectrum of L. 0). To be precise. As a result. If L is not self-adjoint. and their corresponding eigenfunctions v(x). then the dual eigenvalue problem (B2) is equivalent to u† L = β † u† = λu† . if L is a matrix. If L is self-adjoint. non-symmetric). (B4) i a property known in quantum mechanics as the completeness relation. t) as p(x.4 of [7]. let us consider a linear differential operator L and consider the eigenvalue problem Lv(x) = λv(x). then u = v and we recover the usual complete basis of quantum mechanics. for its application to our problem of finding large deviation functions. As in quantum mechanics. see the next appendix on the Feynman-Kac formula.3 This operator is well known in quantum mechanics and leads to what we call a semi-group structure 3 This applies to homogeneous SDEs with time-independent drift. being non-hermitian (essentially. we see that the set of eigenfunctions u and v form a complete basis onto which any function can be decomposed. In this sense. 14 Appendix B: Non-hermitian operators Non-hermitian or non-self-adjoint operators (we do not make a difference between the two here) can be diagonalized in the same way as hermitian operators in quantum mechanics – just think about symmetric versus non-symmetric matrices. Moreover. Moreover. (B5) where u† is now seen as a row vector multiplying L. we have to distinguish between “left” and “right” eigenfunctions (or eigenvectors) to build their spectral decomposition. it is common to call u the left eigenvector of L and v the right eigenvector of L. 5. but will in general have a completely different set of eigenfunctions u(x). eigenfunctions u or v associated with distinct eigenvalues are orthog- onal. it can be shown that X δ(x − x′ ) = u∗i (x)vi (x′ ). t) = U (t)p(x. λ = β ∗ . Appendix C: Feynman-Kac formula The linear structure of the Fokker-Planck equation (5) means that we can write down the time-dependent density p(x. (B1) which can be solved for a set of eigenvalues λ. We do not put indices to these objects as is commonly done – their numbering or labelling is implicit. called the spectrum of L. They are also orthogonal to each other so that Z hui . . For a time-dependent drift. see Sec. that is. For an application of this decomposition for solving the Fokker-Planck equation. vj i = u∗i (x)vj (x) dx = δij (B3) by properly normalizing them. the generator is formally the time-ordered exponential of the time-dependent generator. called the propagator.

(C5) where p(ξ) is the probability density of the increment added to X0 = x. From there. t + dt) = e p(ξ) G(x + ξ. The probability density of Xt or the wavefunction of a quantum system are not the only objects whose evolution forms a semi-group. t) = eζi t rk (x). is what is called the Feynman-Kac formula. t) dξ. t) = hetkAt ix = (etLk 1)(x). so that (i) X G(x. some of which based on analogies with quantum mechanics [15]. The expectation hf (Xt )i also does. 0) = he0 ix = 1. Therefore. X0 = x with probability 1. Specifically. (C4) x The initial condition for the expectation in this formula does not match the start of the integral. we can use the results of Appendix B about the spectral decomposition of non-hermitian operators to expand the initial unit function in the eigenbasis of Lk . coupled with the initial condition G(x. that is. (C8) i . this semi-group property is the Markov property and L† is simply the generator of that semi-group. t) (or the wavefunction) as a result of the fact that U (t) = U (s)U (s′ ) for t = s + s′ . which enables us to write G(x. provided that the function c is smooth enough and is such that the expectation exists. with the completeness relation (B4) and the normalization (34) adopted. its generator being L. This yields. (C3) where Lc = L + c. amazingly. Here. 15 for the evolution of p(x. t) = e 0 c(Xs )ds (C2) x also has. corresponding to c(x) = kf (x) so that Lc = Lk . To connect this result to our goal of calculating large deviations. so we should propagate X0 to Xdt with the SDE (1) to obtain Z f (x)dt G(x. he showed that ∂t G(x. t + dt) to write D R t+dt E G(x. Mark Kac (pronounced khats) showed that the general exponential functional of a Markov process Xt defined as D Rt E G(x. (C7) i (i) where rk denote an eigenfunction of Lk (not necessarily all real). since its evolution is linear. t). For stochastic processes. at least for g = 0. There are many derivations of that formula. and taking the expectation with respect to ξ. the notation h·ix means that Xt is started at x. we then finally arrive at the correct PDE. this generating function satisfies the Feynman-Kac equation. t + dt) = ef (x)dt e dt f (Xs )ds . The simplest I could find is very similar to what we call in physics the Kramers- Moyal expansion (see [9]) and proceeds by considering G(x. we only need to notice that the generating function heT kAT i entering in the definition of the SCGF λ(k) in Eq. a semi-group structure. This linear PDE. 0) = 1 to yield some non-trivial function of x at time t. (C6) where the propagator etLk acts on the initial condition G(x. t) = Lc G(x. L being the generator of Xt . X (i) 1= rk (x). (26) is a particular case of Kac’s functional. t) to second order around x. In the 1940s. By Taylor-expanding G(x + ξ.

[10] The conservation of probability in quantum mechanics is expressed by the requirement that H is hermitian. Prove this result. we arrive at the result mentioned in Sec. Calculate its associated stationary Fokker-Planck current. 50 = research problem. [20] The form of the tilted generator Lk is derived from the Feynman-Kac equation in Appendix C for the case g = 0. 20 = medium. 7. 40 = term project. What is the equation satisfied by C? What is the stationary current Js (x)? When is this density Gibbsian? 5. with attractor at x = 0. 10 = simple. (29). . Then use this result to show that Hk . [15] For gradient SDEs. 9. 30 = moderately hard. [15] Show that the expectation of the empirical current JT (x) defined in (22) is the stationary Fokker- Planck current. the eigenvalue with largest real part will dominate the sum (if there is a gap) and. EXERCISES All exercises are rated. Feel free to contact me for any questions or comments. (C9) 2π 2 where C is a symmetric. 1. positive matrix. Do we still have that stationary distribution when σ is not proportional to the identity matrix? Calculate the associated stationary current. Lf ips = −kσ∇f k2ps . [15] Show that the stationary density of the linear SDE (17). in the absence of external forces (φt = 0). thus. 3. IV B that the SCGF is the largest eigenvalue of Lk . f ips = hf. [10] Show that lk=0 (x) = ps (x) and rk=0 (x) = 1 for all x. (C11) where k · kp is the norm weighted by p. is hermitian. hLf. Does it vanish? Discuss the consequence of this result in view of the fact that equilibrium systems are supposed to have vanishing currents. [20] Prove that the generator L of gradient SDEs is self-adjoint with respect to the following inner product Z hf. The same result holds for g 6= 0. [15] Show that the stationary density of Kramers equation (13). 10. f ips is a so-called Dirichlet form: hLf. according to Knuth’s logarithmic rating system whereby 00 = immediate. [20] Show that the expectation of the empirical density ρT (x) defined in (21) is the stationary distribution ps (x) when Xt is ergodic. as defined by the symmetrization (35). 11. What do you conclude for the spectrum of L? 6. Explain why this must also be the most probable value of ρT in the long-time limit. is a Gaussian distribution of the form r   det C 1 ps (x) = exp − x · Cx . 16 where ζi denotes an eigenvalue of Lk . is the Gibbs distribution (16) with potential V . [15] Show for gradient SDEs that Lps = ps L† . [10] Derive for gradient SDEs the expression of the quantum generator Hk shown in (37). together with the corresponding potential Vk (x) shown in (38). [Hint: What is the distribution of ρT ? Does it satisfy an LDP?] 8. In the long-time limit. Adapt the proof for the case g 6= 0 leading to the full expression of Lk shown in Eq. (C10) where ps is the Gibbs stationary distribution. 12. since there is also a Feynman-Kac formula for this case. one in fact that was never considered by Kac (see Exercise 9). rather subjectivity. [20] Prove that the stationary density of the gradient SDE (15) is the Gibbs distribution (16). 4. What is the corresponding property of L for Markov processes? 2. gips = f (x)g(x) ps (x) dx.

IV D for T 1 Z VT = Xt2 dt. What happens when γ → 0? Sources: [26. [40] Repeat the previous exercise by replacing Xt2 by Xtk . (C14) T 0 where χS (x) is the indicator function equal to 1 if x ∈ S and 0 otherwise. [25] Combine the calculation of Sec. [30] Calculate for the Ornstein-Uhlenbeck process the rate function I(r) associated with T 1 Z rT = χ[−1. 20. first. [25] Repeat the exercise about the occupation time for χ[0. ii) X0 = a and XT ∈ R. 1].] 18. [10] Derive the relations (40) and (41) between ψk . This observable represents the fraction of time Xt spends in the interval [−1. and lk using the relation (35) between Hk and Lk . (C12) T 0 which represents the empirical variance of Xt . but by integrating XT over R. 17 13.1] (Xt ) dt. [20] Study the large deviations of the entropy production ΣT . k > 2. [20] Consider the SDE dθt = γdt + σdWt . so it is a random variable taking values in [0. 15. IV D with the previous exercise to find the rate function of T X02 X2 1 Z QT = − T + Xt dt (C13) T T T 0 for the Ornstein-Uhlenbeck process. You should see in each case a very different effect of the “boundary terms” X0 and XT . (23). 1]. 27]. assume that X0 is distributed according to ps (x). of the Ornstein- Uhlenbeck process. Obtain the rate function for the three cases considered before: i) X0 = a. rk (x) and lk (x) (correctly normalized). that X0 and XT have fixed values. Finally. (C16) T 0 What are the natural boundary conditions for rk (x) and lk (x) in this case? Can Lk be symmetrized when γ = 0? Source: [28] and references therein. θt ∈ [0. 19. iii) X0 ∼ ps (x) and XT ∈ R. [40] Show that the occupation fraction rT defined in the previous exercise can be obtained by integrating the empirical distribution ρT (x) defined in (21) over x ∈ [−1.. What is the related quantum problem? Find λ(k). X0 = a and XT = b. 1]. although ζmax (Lk ) is defined for all k ∈ R. You should see that the region where λ(k) < ∞ is different in each case. [20] Repeat the calculations of Sec. Source: [15]. This observable can be related to the heat exchanged by a Brownian particle with its environment when manipulated by laser tweezers [25]. rk . 16. Is this a gradient (equilibrium) system? What is the stationary Fokker-Planck current Js (x)? Obtain the rate function I(j) characterizing the fluctuations of the total empirical current T 1 Z Z JT = dθt = JT (x)dx. 14. . [Hint: You will need the generating function of the χ2 distribution. (C15) representing the motion of a Brownian particle on the unit circle with a drive or torque γ. 2π). XT = b. 17. e. Can you use this result to derive the rate function I(r) from the known rate function(al) I(ρ) of ρT (x)? Source: [26]. as defined in Eq. Then repeat the calculation for X0 = a.∞) (x) so as to study the fraction of time Xt stays positive. 21. Assume.g.

[40] Rewrite all the large deviations of these notes for Markov chains. I also thank Marco Baiesi. Louw for comments on these notes. 1999). 23. [9] L. (Springer. Is this SDE gradient? Why is it called “transversal”? What do you obtain if you try to symmetrize it? Source: [29]. Chemistry and the Natural Sciences. “An algorithmic introduction to numerical simulation of stochastic differential equations. Øksendal. that is. [25] Calculate the rate function of the entropy production ΣT for the 2D linear transversal SDE defined by ! ! −1 −1 x F (x. With this. in the mapping to the quantum hamiltonian. Cambridge. New York. Probability and Random Processes (Oxford University Press. Oxford. [8] C. [2] Z. the noise parameter is essentially ~. Zastawniak. Reichl. Berlin. you must specify the stochastic convention used for interpreting the product σ(x)dWt . (C18) where c is some constant. V of [30] and Sec. Gardiner. [50] Do the eigenvectors rk and lk have any probabilistic or physical interpretation? Sources: [21. A. 2nd ed. Springer Series in Synergetics. A Modern Course in Statistical Physics. [5] D. Risken. Vol. ACKNOWLEDGMENTS I thank Jan C. Pavliotis. Derive from this a symmetry satisfied by the SCGF and rate function. 3rd ed. (Wiley. E. 2016). Source: [22]. 25. [4] B. [30] Derive the expression of the tilted generator Lk for SDEs in which the noise is multiplicative. [25] Show that the tilted generator associated with the entropy production ΣT satisfies the symmetry L†k = L−k−c . and Thomas Speck for the invitation to the 2017 School on Fundamental Problems in Statistical Physics. [25] Another approach for deriving the SCGF is to discretize in time an SDE to obtain a Markov chain for which AT is then a sum. Stochastic Differential Equations (Springer. Brzeźniak and T. 4th ed. 2014). comment on the following: The low-noise limit of SDEs is equivalent to the semi-classical limit of quantum mechanics. In this case. Use this approach to confirm that the SCGF λ(k) is given by the dominant eigenvalue of Lk . Stochastic Processes for Physicists: Understanding Noisy Systems (Cambridge University Press. 2001). 27. 1985). My trip there is funded by the National Research Foundation of South Africa (Grant no. Source: Chap. 26. 18 22. New York. Then do the same for Markov jump processes or continuous-time Markov chains. 2000). 28. Source: [10]. W. Jacobs. 24. [3] G. New York. [6] G. Springer Undergraduate Mathematics Series (Springer. [25] We have seen that. 22]. J. Higham. [7] H. Basic Stochastic Processes. [1] K. Alberto Rosso. 2010). y) = (C17) 1 −1 y and σ = ε11. 13 (Springer. Handbook of Stochastic Methods for Physics.” SIAM Review 43. Grimmett and D. . 1996). The Fokker-Planck Equation: Methods of Solution and Applications . Such a symmetry is known as a fluctuation relation. 7 of my course on large deviations. 525–546 (2001). 96199). New York. Stochastic Processes and Applications (Springer. in which the noise matrix σ depends on x. Source: [12]. Stirzaker..

Touchette. Phys. P07020 (2007). Gabrielli. 89. Stat. Touchette. Chetrite and H. Derrida. Stat.” Phys. 046102 (2003).” Ann. Angeletti and H.” Phys. Lett. “Brownian functionals in physics and computer science. A. van Zon and E.” J. Majumdar. Tsobgni Nyawo and H. “A minimal model of dynamical phase transition. Phys. Landim. “Nonequilibrium Markov processes conditioned on large deviations. [24] H. “A basic introduction to large deviations: Theory. Seifert. Stat. [18] B. “Nonequilibrium microcanonical and canonical ensembles and their equivalence. Mech.” in Nonequilibrium Statistical Physics of Small Systems: Fluctuation Relations and Beyond .” Phys. 056121 (2004). E 67.” Phys. E 65. G.” in Modern Computational Science 11: Lecture Notes from the 3rd International Oldenburg Summer School . E 94.” Rep. Poincaré A 16. 2076–2092 (2005). [30] J. Vol. New York.07707. “Fluctuations and correlations in nonequilibrium systems. Noh. Touchette. [28] P. Jarzynski (Wiley- VCH. “Stochastic thermodynamics. 95. “Large-deviation functions for nonlinear functionals of a Gaussian stationary Markov process. Stat.” Physica A 418. 335–360. “The exclusion process: A paradigm for non-equilibrium behaviour. Just.” J. and C. D. P01013 (2014). Rev. Touchette. Barato and R. [14] S.” J. Bray. Sci. K. P07014 (2007). Tsobgni Nyawo and H. Klages. applications. Rev. 2013) pp. edited by R. Weinheim. 2007. [27] P. “A Gallavotti-Cohen-type symmetry in the large deviation functional for stochastic dynamics. Rep. [19] L.” Phys. Stat. “Non-equilibrium steady states: Fluctuations and large deviations of the density and of the current. “Stationary and transient work-fluctuation theorems for a dragged Brownian particle. N. J. G. 333–365 (1999). Rev. Cohen. [29] J. Harris and H. 126001 (2012). Prog. Hartmann (BIS-Verlag der Carl von Ossietzky Universität Oldenburg. Bucklew. van Zon and E. [13] R. “The large deviation approach to statistical mechanics. D. J. [21] R. Lett. G. [26] F. Schütz. edited by R. Mech. [25] R. simulations. Cohen. [22] R. arxiv:1611. D. “Stochastic interacting particle systems out of equilibrium. 19 [10] J. Spohn. L. Chetrite and H.5 large deviations and fluctuation relations. Lebowitz and H. “Extended heat-fluctuation theorems for a system with deterministic and stochastic forces. Majumdar and A. .” J. 023303 (2016). Reviews of Nonlinear Dynamics and Complexity. 2007. 50009 (2016). “Diffusions conditioned on occupation measures.” Europhys. Touchette. P07023 (2007). 2005–2057 (2015). Inst. Math. 1–69 (2009). 75. 53. 116.” J. 1154–1172 (2015). Simulation and Estimation. E 69. Harris and G. Touchette. and C. Rev. Chetrite. 2007. [17] K. Stat. Mech. Rev. [23] A. 6. De Sole. Touchette.” J. Large Deviation Techniques in Decision. Touchette. “Fluctuation theorems for stochastic dynamics.” Phys. Leidl and A. “Large deviation approach to nonequilibrium systems. [15] S. Mallick. J. “Large deviations of the current for driven periodic diffusions. D. 032101 (2016). 111. [12] H. Phys. Phys. “A formal view on 2. 160. A. [16] U. N. Wiley Series in Probability and Mathematical Statistics (Wiley Interscience. Mech. 478. 2011). fluctuation theorems and molecular machines. Jona-Lasinio. 1990). 2014. 120601 (2013). [20] R.” Curr.” J. 17–48 (2015). W. Bertini. [11] R. 051112 (2002). M.