Proc. Nat. Acad. Sci. USA Vol. 71, No. 7, pp.

2618-2622, July 1974

Entropy, Dynamics, and Molecular Chaos

(symmetry breaking and XC-theorem/McKean-Kac models).

F. HENIN* AND I. PRIGOGINE*t * Facult6 des Sciences, Universit6 Libre de Bruxelles, Belgium; and t Center for Statistical Mechanics and Thermodynamics,
The University of Texas at Austin, Texas 78712

Contributed by 1. Prigogine, April 16, 1974

With the help of simple probabilistic ABSTRACT models of Kac and McKean, we discuss the meaning of the generalized expression for entropy that was recently introduced by our group and compare it with Boltzmann's expression. We emphasize the fact that Boltzmann's formulation in terms of the single particle distribution function, fi, requires very restricted assumptions about the preparation of the system (chaos) and the nature of the collision mechanism (Markov processes). Our generalized 3C-theorem, however, refers to the complete system; in general, it does not lead to an 5C-theorem for the single particle distribution function, f.1" It is valid whatever the preparation of the system. In McKean's model, situations exist where it gives the correct behavior while the Boltzmann's expression for entropy becomes meaningless. In addition, in Kac's model, we show that correlations reach equilibrium more rapidly than fi and that there is an asymptotic regime where both formulations give the same result.
1. Introduction

(see ref. 3):

liM
N--+ co

f2(NV)(VIV2,t)

N--+ co

liM fi(N)(vlt) } { UM fi(N)(v2,t)

N--* co

[11.3]

At the very core of the second law of thermodynamics we find the distinction between reversible and irreversible processes. In irreversible processes such as heat conduction, the time reversal invariance t -- - t is broken. This distinction leads to the introduction of entropy. Well-known physicists such as Ostwald, Duhem, Mach, and others considered the second law the indication that mechanics had to be given up on the atomic level and be replaced by a more phenomenological "energetics" that would incorporate, from the start, the second law. On the contrary, Maxwell and Boltzmann firmly believed in the "mechanical interpretation" of the second law. (For the historical development, see ref. 1.) However, it would be more appropriate to speak about the "statistical interpretation," as Boltzmann had to introduce in his derivation of the second law probabilistic elements quite foreign to dynamics (2). The status of the second law has been the subject of constant controversy in the last century. We would thus like to make first some brief historical remarkst. From his kinetic equation for the velocity distributionf(v,t):

bf/bt

fdnfdva(QJv, - vl)lv' - vl(f'fi' - ffi)

[1.1]

However, there are examples (such as in the velocity inversion experiment, associated with the Loschmidt paradox) where Eq. 1.3 does not apply (4). Then, for macroscopic times Boltzmann's entropy (Eq. 1.2) decreases! It is very hard to accept such deviations from the second law, which would upset the very distinction between reversible and irreversible processes (2, 4). The validity of the second law would then depend critically on the preparation of the system. A somewhat similar conclusion was reached by Kac (3) in his interesting work on master equations and their relation to the Boltzmann entropy. As he has shown, chaos propagates in time. Therefore, he argues "that the nonlinear character of Eq. 1.1 is due solely to the extremely special assumption which the initial distribution has to satisfy." But how can we then base the derivation of a very general law such as the second principle on such a special equation? Moreover, it has never been possible to extend Boltzmann's argument to wider classes of systems. A quite different point of view thus has to be adopted. We have always considered that only a thorough theoretical investigation of N-body systems can lead to the elucidation of the microscopic model of entropy (5) . The surprising result is that the second law becomes a theorem in dynamics of systems satisfying well-defined dynamic conditions. From a certain point of view the "energetists" were right, as we have to deal with an extended form of dynamics, but of course the "mechanicists" were right, too, as this does presuppose the laws of mechanics for simple systems such as atoms and molecules. Let us summarize some aspects of our results that are relevant for the discussion of entropy. The starting point is the von Neumann-Liouville equation [1.4] aplat = -iLp for the density matrix (quantum case) or the distribution
We shall not discuss here proposals to "explain" irreversibility through various forms of coarse graining, introduction of information concepts, or appeal to cosmological time arrows. Such assumptions do not lead to thermodynamic irreversibility as expressed, e.g., by the semigroup properties of the Fourier equation; also, it then becomes hard to understand why irreversibility appears in some dynamic systems and not in others.

Boltzmann obtained his microscopic model for entropy [1.2] SB = -kfdvf(v,t) ln f(v,t) Equation 1.1 is based on the "molecular chaos assumption"

t These remarks will be developed more completely in a forthcoming paper.

2618

Proc. Nat. Acad. Sci. USA 71

(1974)

Entropy, Dynamics, and Molecular Chaos

2619

function (classical case). This equation is "Lt" invariant (it does not change when L -- - L and t -- - t). For a classical system, L inversion can be realized through inversion of the velocities of all particles. Boltzmann's equation (1.1) is not Lt invariant. The origin of this symmetry breaking is due to the explicit consideration of causality corresponding to initial value problems. In brief, for systems for which the operator L has a continuous spectrum, causality requires analytical continuation of the resolvent (L - z)'- associated with Eq. 1.4 (for more details see refs. 4 and 6). This then leads to the appearance of even terms in L in the evolution equations. These even terms break the Lt symmetry. The causality requirement can also be introduced into Eq. 1.4 with the help of a specific class of nonunitary (called starunitary) transformations A:

differences between Eq. 1.10 and the Gibbs expression for entropy SG = -k Tr p In p. First, the Gibbs entropy is an invariant. We have to work in the noncanonical "physical" representation (Eq. 1.5) to display explicitly, the ihcrease of entropy. This is why Eq. 1.9 does not contradict the fact that SG is constant. Secondj the functional dependence on p is different. SG (with p replaced by p9) is a valid form for entropy when the process described through Eq. 1.5 is markoffian. This, requires, however, more conditions than the existence of the "friction" (Eq. 1.7). In simple cases such as dilute gases these conditions are indeed satisfied for the evolution of the diagonal elements of the density matrix (4). We may then take for the entropy (using a notation appropriate for classical

systems)
[1.11] SB = -kf(dv)Npo({v},t) ln po({V},t) The factor '/2 in Eq. 1.1O has been chosen in such a way that, in the markoffian case, Eqs. 1.10 and 1.11 become identical in some neighborhood of the microcanonical equilibrium. However, the markoffian form (Eq. 1.11) is very restrictive. This is why it was not possible to extend Boltzmann's original arguments to a wider class of systems. Moreover, if nondiagonal elements of the density matrix are retained (or space dependent correlations in classical systems), the markoffian form fails. An example has been given elsewhere (ref. 4, appendix IV).

and [1.5] apl/at = -ikpp with 0 = A-1 LA = OM + P() and p(0)(L) = - (0) (-L) [1.61 ,(()= 46) (-L), The important point is that, if the even operator OM) exists, it is negative; [1.71 -iTrf1qs(')f ( 0
= A-1p

9P

Such systems will be called digsipative systems. Once Eq. 1.7 is satisfied, one can construct the Liapounoff function P 0 [1.8] Q = Trpv~p' such that

Q/bt

-2iTrpPt,()pP

% 0

[1.9]

It is precisely through the damping mechanism expressed in Eq. 1.7 that the system will reach equilibrium. It is important to notice that this damping mechanism becomes explicit only after the transformation to a nonunitary representation has been made. Microcanonical equilibrium corresponds to the minimum of Q. Therefore, there must be a close connection between Eq. 1.4 and entropy. Additivity of entropy for independent systems leads to

Seq - (k/2) ln [Tr pvtpv/(Tr pptpV),.] [1.10] with Sq = -k log (Tr pPtpP)6, = k log e, 8 being the number of accessible states. We shall come back to the choice of the factor 1/2. Let us make a few general remarks: (a) As mentioned before, the second law becomes a theorem in dynamics; inequality (Eq. 1.7) expresses a dynamic property. The symmetry breaking comes not from the preparation of the system as argued by Boltzmann (see especially ref. 7) but directly through the supplementary conditions imposed by the initial value problem. (b) The entropy is expressed in terms of the full N particle density matrix. If we expect the second law to be valid whatever the initial state of the system, entropy obviously has to depend in general on all dynamic variables of the problem. It is only in special cases, such as near-equilibrium conditions, that entropy may be expressed in terms of reduced distribution functions as in Eq. 1.2 or even in terms of macroscopic quantities alone as in thermodynamics (8).
S
=

approach to equilibrium We have on one side the expressions 1.10 or 1.11 involving the full N body distribution function, and on the other the Boltzmann expression 1.2. As we showed elsewhere (4), the N body expression is necessary for discussion of problems such as velocity inversion. Here we want to show that, even for simple N-body stochastic models, a microscopic expression of entropy such as Eq. 1.10 or 1.11 is needed to describe approach to equilibrium for arbitrary initial conditions. This problem is, of course, interesting independent of the dynamic derivation of Eq. 1eo or 1.11. We consider two models due to McKean (9, 10) and Kac (3, 11) for which most of the calculations -can be performed explicitly. In spite of the fact that these models are stochastic and not dynamic, we may expect them to have features in common with simple dynamic systems such as dilute gases when the transition to the nonunitary representation (Eq. 1,6) has been made. We present a rather condensed account of our results.
2. Entropy and the

3. McKean's model In this very simple probabilistic model (9, 10), one considers N "particles" with velocities e.... eN, with + 1 and -1 as allowed values for the ei. After "collision," two particles with velocities e,,e2 emerge with velocities ei*,e2*. With a probability '/2, one has: = either e,* = e, e2=* ee2orei* = ele2, e2* = e2 We then obtain the kinetic equation

[3.1]

bpo(ei,. .,eN; t)/lt

= N-1

i<j

E { po(el,.

,e, .,ecej,. ., eN;t)

The examples studied in the later sections of this paper will illustrate these general statements. (c) There are two basic

+ pO(el,. ,ee,. .,ej,. eN; t) - 2po(el,. ,e,. ,e,. ,eN;t)}

[3.2]

2620

Physics: Henin and Prigogine

Proc. Nat. Acad. Sci. USA 71

(1974)

It preserves normalization. An additional invariant po({ et = 1} ,t) has no counterpart in a gas. It corresponds to the fact that collisions between two particles with velocities +1 do not change the state of the system. As a consequence Eq. 3.2 admits an infinity of stationary solutions:

PO8({ej = 1})

= a,

PO

t(let)) = (1 - a)/(2N - 1) {et) {+i}

if [3.31

propagates. However, for p = O(N), Eq. 3.12-is not a solution of the exact equations 3.10 (except at equilibrium if, in Eq. 3.3, the invariant a equals 1 or 2-N). When we deal with the complete system, correlations play a role. Another important point is that, even for p << N, Eq. 3.12 is a solution of Eq. 3.11 only for very special initial conditions. In terms of the moments, the Liapounoff function (Eq. 1.8) becomes
N
E E= [po(et},t)12
ei..eN
=

Only if a = 1 or a = 2-N, is the stationary solution a product of one-particle distribution functions. The case a = 1 is without interest, since all the particles then have velocity + 1 and nothing happens in the course of time. For a = 2-N, the stationary distribution corresponds to microcanonical equilibrium: po"({ e4}) = 2-N for all { en}.
If we assume that, at each time t, the distribution is factorized:
N

2-N

E p=O

X (N!/p!(N - p)!) xp2 > 0 [3.13]

with
N-2

dQ/dt =-(N - 1) A, [(N - 2)!/p!(N - 2-p)!][xp+ p=O

-xp+2]2
with

0 [3.14]

po({e4et) = J~fi(eit)

f1(+1,t)
+f1(-1,t)
= 1

[3.4] we obtain from Eq. 3.2 the Boltzmann's equation (f+= f1(+1,t), 0 . f+ < 1) [3.5] df+/dt = 1 - 3f+ + 2f+2 which can be solved exactly (see ref. 10). Boltzmann's entropy and entropy production are: [3.6] sB = -f+ ln f+ + (1-f+) ln (1-f+) dsB/dt = -(1 - f+) (1 - 2f+) ln [f+/(1 - f+) ] [3.7] Close to equilibrium, with f+ = (1/2) (1 + e g), we obtain, to lowest order in e: dsB/dt = e2g2 [3.8] 8B = ln 2 - e2g2/2,
All these results depend on the chaos assumption. Two questions arise: (1) what is the generality of this condition? (2) if valid at time to, does it propagate in time? To discuss these problems, let us describe the evolution of the system in terms of moments (average values) (elae2" ... eNaN). Since eia; = 1 if a, is even and eta; = e1 if at is odd, the only independent moments are (when po is a symmetric function of all the variables): [3.9] xp = (eie2.. .e) = E ee2...eppo({ej},t)
ei. .eN

The equality sign is obtained only at equilibrium when xl = = constant (with a = 2N, this constant is X2= ... =

zero).

Inserting Eq. 3.13 in Eq. 1.10 (with pP po), assuming Eq. 3.4, and takingf+ = (1/2) (1 + eg), one recovers Boltzmann's results (Eq. 3.6) at lowest order in e. Although we have no complete proof (see also section 4 of this paper), such a procedure appears to be justified if, in the evolution equations, one keeps only terms decaying with the longest characteristic time, i.e., the smallest eigenvalue XI --1 associated with equations 3.10. Inequality 3.14 holds whatever the initial conditions. This is not so for Boltzmann's entropy. Indeed, in terms of moments Eq. 3.6 becomes:
.&B
=

(1/2) (1 + Xi) In (1 + Xi)

- (/2)(1 - xi) ln

(1

xi)

[3.15]

-('/2)(X1 X2) ln [(1 - xi)/(1 + xi)] [3.16] In general, the sign of this quantity, although positive if x2 = XI2, depends on the choice of the initial conditions. It is very easy to construct situations (e.g., x2(0) > xl(0) > 0)
=

Hence: dsB/dt

From Eq. 3.1, N): dxp/dt = [p(p -

one obtains the

evolution equation (1

<,

p <

where Boltzmann entropy decreases instead of increases. On the contrary the general expressions Eq. 1.10 or 1.11 have the correct time behavior independent of the initial conditions. Moreover, in this simple case as well as in the Kac model (see section 4) the limits for N -- o of N 1S and N -dS/dt exist and are finite.
4. Kac's model

[p(N - 1)/N]xp + [p(N - p)/N]xp+i [3.10] equivalent to the Bogoluilov, Born, Green, Kirkwood, Yvon hierarchy. For p << N, Eq. 3.10 becomes
[3.11] -P(Xp - Xp+) + O(N-1) In the limit N -- c>, Eq. 3.11 admits the factorized solution: with dxu/dt = -xi(1 -Xi) [3.12] Xp = x1p

1)/N]x,1 -

dxp/dt

In this model (3, 11), called by Kac a "caricature of the hard sphere gas," -one considers N "particles" with velocities Xi ... XN, which vary continuously from - o to + a) . After a collision between particles with initial velocities xixj, the final velocities are: xi* = xi cos O + xj sinO, Xj * = -xi sin 0 + xj cos 0 [4.1]

(Since xi = 2f+ -1, Eq. 3.12 is an alternative form of Eq. 3.5.) Thus, for a finite number of variables, molecular chaos

with 0 < 0 < 27r. The probability that a given collision takes place in the time interval dt is assumed to be adt/N. Then,

Proc. Nat. A cad. Sci. USA 71

(1974)

Entropy, Dynamics, and Molecular Chaos

2621

the kinetic equation is

bpo(xi,. . . ,XN;t)/lt

(a/N)

(m, and n, are numerical factors.) In the first term, of 0(j2/N), two of the particles 1... .j interact; in the second, of O(j(N one j)/N), the interaction occurs between N of the particles 1 ... .j and an outside particle. With B = E i, the equilibrium
i=1

-po(x*,. ,xj,. . ,j, . . ,xN;t)} [4.2] Again, the normalization is preserved. Moreover, the collision mechanism is such that:

solution of Eq. 4.9 is:

(XI". .X/,P)e_

- (

12

F(N/2)

r((k+ 2)/2)
[4.10]

fj X,2 = No, s=l

IV

constant

[4.3]
Forj << N, we have
j

Eq. 4.3 defines the microcanonical surface E and implies that pe({ xf} ,t) is different from zero only on Q. The equilibrium distribution is equal to [ (dx)N] on Q, and to zero outside a.

E E b(xift1...xji)/1t = 2Ta k= 1 8=O ns(k)

x (XI '.

(k/2

If one assumes that, at a given time to, the distribution function is factorized as far as Eq. 4.3 allows, i.e.,
N

.X28.

XjfiX6, 28)

* Audi)} *. (Xol

[4.11]

po({i} ,to) = H p(xi,to) on Q, = 0 outside a

t=1

[4.4]

one can show that, in the limit N -- co, the reduced distribution functions satisfy the chaos condition with [4.5] fi(x11to) = (2ira) `12 exp (-x12/212oX(xi,to) The entropy and entropy production are then functions of fi. For instance, lim N-1[S(to) - SeI = -(1/2) [ako + In G(ko)

The equilibrium moments are then factorized and can be obtained from Boltzmann's distribution. More generally, Eq. 4.11 admits factorized solutions, i.e., molecular chaos propagates. The moments are not all independent. Because of Eq. 4.3, moments where at least one of the Of is equal to 2 may be eliminated from the rhs of Eq. 4.11; this gives

6(xif,. .Xjiftvt
.

[Tj[0. #j) ] l(xl

..

...

vX2i

+ M/fl.

..#j) [4.12]

(1/2)

+ (1/2)

ln 2ia] [4.6]
[4.7]

with

G(ko) =

dx exp

(-kox2)[o(x)12

where ko is a solution of the equation aG(ko) + dG(ko)/dlco = 0

[4.8]

If one further assumes that (p = 1 + eg, Eq. 4.6 and the corresponding expression for N-1 dS/dt become identical to Boltzmann's results at lowest order in e. However, this is still not conclusive. Indeed, although molecular chaos in the sense of Eq. 1.3 propagates in time, the form Eq. 4.4 for po does not. If instead of Eq. 4.4, one writes down a cluster expansion, one can show that, if one assumes pair correlations to be O(N-1), they do not contribute to f, but they contribute a finite amount to S and dS/dt. It is thus necessary to elucidate the meaning of the expansion parameter e. Let us again consider the moments (xI" ...xfai). Because of the simplicity of the collision operator, if at t = 0 moments where at least one of the ai is odd vanish, this remains so later on. We then have only to consider moments where all the a1 are even (we use ,l to denote even values of ar). Their evolution equations are [X7(x) is Heaviside step function]:
j-1
j

where Mj(,%...P) is the contribution of moments different from (xi ... x1#*). The structure of these equations is such that they are easily solved for small values of P. The results, up to total # = 10, show that (a) although Eq. 4.3 implies certain conditions on the correlations [e.g., (x12x22) - 2 = O(N-1)], molecular chaos is a very special initial condition [e.g., (x14x24) - (X14)2 need not be O(N-1)1; (b) finite initial correlations contribute for t > 0 to (xlf) [when there is no chaos, fi(t) depends on fi(0) and f2(0)]; (c) the solutions are superpositions of decaying exponentials exp [-t/Trj(6. .0j)]. The longest time scale is Ta (4). Asymptotically, keeping only this time scale, one obtains as solution of Eq. 4.11:
(Xl''. .Xji)
=

(Xi*'.

Xjli)eq

X {i +
N

mj(0%. .-

) exp [-t/T1(4)]}

[4.13]

For j << N, these expressions can be obtained as averages with

Po(t)W =
fi(xi,t)

il
=

I fi(xi,t) with

(2ra)

1/2 exp

(-x2/2a){ 1 + [A(x,4)/96a2] X H4(x/V/2a) exp [-t/Tl(4)]} [4.14]

(H4 is the fourth Hermite polynomial), provided one considers exp [-t/x1(4)] as a smallness parameter (the parameter e used before) and keeps only terms linear in E.
.2s .+ x

a(Xi1'.

E E .xj-i)/at = (2Ta/N),I(j - 1) k=l =k+l

-(Xiv. .Xk-.. Xe *
..

(Pk+6)/2)

8=O

* m8(flkle)(xl.** Xk.... Xe

.X~i)4 + (2Ta(N -j)/N)

E{ I ns(n)(xI'. .xk2 '..

k=l s=O

.Xk.OL Xji)} [49]

2622

Physics: Henin and Prigogine

Proc. Nat. Acad. Sci. USA 71

(1974)

Thus, whatever the initial conditions, molecular chaos for functions of a finite number of variables is asymptotically realized. We, however, stress the fact that this asymptotic conditlbn is very strong. It is not sufficient to consider a regime in which Boltzmann's equation can be linearized (the solution of this equation contains all the characteristic times T1(I3) not only x1(4)]. One must take the asymptotic solution of the linearized equation. This is not yet sufficient to show that Eq. 1.10 becomes asymptotically identical with Boltzmann's entropy. To establish this, we have derived the asymptotic solution of the equations 4.9; this has enabled us to take correctly into account the contributions of all moments, including those with O(N) variables.
5. Concluding remarks

However, in arbitrary nonequilibrium situations we cannot expect the SC-quantity to have any simple macroscopic meaning. This is an unavoidable consequence of the very generality of the second law; whatever the type of perturbation (macroscopic or not) applied to it, the system has to react by again approaching thermodynamic equilibrium.
We thank Prof. G. Nicolis and Dr. R. Herman from General Motors Research Laboratories, for many helpful discussions. We also thank Miss Michelle M. Fleckenstein, also from General Motors, for numerical calculations concerning the McKean model. This work was started during a stay of F.H. at the Center for Statistical Mechanics and Thermodynamics at the University of Texas at Austin. The support of NATO Grant 644 is gratefully acknowledged. This paper is part of a program sponsored by the R. Welch Foundation (Houston, Texas, USA) and the Fonds de la Recherche Collective (Belgium). 1. Brush, S. G. (1966) "Irreversible processes," Kinetic Theory (Pergamon Press, Oxford, England), Vol. 2. 2. Prigogine, I. (1973) in The Boltzmann Equation, Theory and Applications, eds. Cohen, E. G. D. & Thirring, W. (Springer Verlag, Vienna, New York) pp. 401-450. 3. Kac, M. (1959) Probability and Related Topics in Physical Sciences (Interscience, London). 4. Prigogine, I., George, C., Henin, F. & Rosenfeld L. (1973) Chem. Scripta 4, 5-32. 5. Prigogine, I. (1962) Non Equilibrium Statistical Mechanics (Wiley, Interscience) (especially p. 2). 6. Prigogine, I. (1973) Nature 246, 67-71. 7. Boltzmann, L. (1877), Wien. Ber. 75, 62, reprinted in ref. 1. 8. Glansdorff, P. & Prigogine, I. (1971) Structure Stability and Fluctuations (Wiley, Interscience, New York, London; French edition: Masson, Paris). 9. McKean, H. P., Jr. (1967) J. Comb. Theor. 2, 358-382. 10. Kac, M. (1973) in The Boltzmann Equation, Theory and Applications, eds. Cohen, E. G. D. & Thirring, W. (Springer Verlag, Vienna, New York), pp. 379-400. 11. Kac, M. (1956) in Foundations of Kinetic Theory, Proc. Third Berkeley Symp. on Math. Stat. and Probl. (Berkeley: Univ. of Calif. Press), Vol. III, pp. 171-197.

The fundamental role of the XC-theorem is to describe how the dynamic system under consideration is driven to microcanonical equilibrium. This includes the evolution of all reduced distribution functions and all correlations. Therefore, a description in terms of fi will in general not be possible. The validity of Boltzmann's 3C-theorem requires strong assumptions about the initial preparation of the system and about the structure of the collision operator. It is, however, quite satisfactory that there is a connection between the generalized and Boltzmann's SC-theorems. This occurs for aged systems, in which correlations have already reached their equilibrium values. This is most transparent in Kac's model. The question whether, in more realistic systems, correlations reach equilibrium on a faster time scale than fi is still open. One can, however, expect that this will be so in the asymptotic regime in which a description in hydrodynamic modes is possible and the usual thermodynamic description applies.