# Mezić Research Group

TIME-AVERAGES OF OBSERVABLES

Marko Budišić
mbudisic@engr.ucsb.edu
SIAM Conference on Applications of Dynamical Systems Snowbird, May 2011
The minisymposium that follows focuses on time averaging in analysis and design of dynamical systems. This presentation aims to introduce the recent advances in analyzing dynamical systems by averaging functions, or observables, along the trajectories of the system. First of all, let me brieﬂy say what I will NOT be talking about. This work should not be confused with averaging of the model where the order of the model is reduced by substituting dynamics evolving on fast time scales by their averaged effect. The time-averages of observables, on the other hand, involve averaging observables, or measurements, along trajectories of the system. Common names used are: time-averages, trajectory-averages, dynamical averages, Cesàro means, ergodic averages, lagrangian averages, cumulative moving averages, statistics...

Mezić Research Group

Problem:

Trajectories are not robust descriptions of dynamics in chaotic systems, due to exponential sensitivity to errors.

Motivation:

Time-averages are robust under time-discretization and round-off error even when trajectory is not.

Time-averages provide macroscopic descriptions of geometry of dynamics: Lyapunov exponents, residence times, power spectrum,...

[Cvitanović et al., Chaos Book]

Use time-averages to extend the library of robust indicators of dynamical behaviors for analysis and design.
Monday, May 23, 2011 Marko Budišić: Time Averages of Observables
2

In chaotic systems, trajectories are non-robust. Even the smallest perturbations, common in numerical simulations, result in completely different trajectories. In Chaos Book, Cvitanović argues that trajectories are wrong objects to look at in such cases, and that the best we can do is to analyze features of the set in state space that the trajectory explores. Despite trajectories being sensitive to perturbations, simulations and numerics show that averaging over them still results in reliable time-averages, as noticed by Sigurgeirsson and Stuart. Time averages provide a macroscopic description of features of the set a trajectory is exploring in the state space. Many common dynamical indicators (Lyapunov coefﬁcients, residence times, power spectra...) can be formulated as time averages of particular observables along trajectories. This body of work aims to systematically extend the library of those indicators by a careful choice of observables that are averaged.

Mezić Research Group

Compact manifold Flow map Observable

Φt : M → M f :M→C
f [Φt (x)]dt
0

M

˜(x) = lim 1 f T →∞ T ￿

T

Birkhoff: Limits exist a.e. wrt any measure preserved by ﬂow.

Monday, May 23, 2011

Marko Budišić: Time Averages of Observables

3

Dynamics are modeled by an ﬂow map on a COMPACT manifold. Observables are scalar functions on that manifold. In most of the cases, we will focus on continuous observables. Tilde above a function will indicate that we are taking a time average of the function along the trajectory. Of course, there exist analogous formulations for maps and PDEs. Since the deﬁnition of the time averages includes taking inﬁnite-time limits, it is a valid question to ask whether these limits exist or not. A large class of dynamical systems preserve SOME measure. Birkhoffʼs theorem asserts that time averages converge almost anywhere with respect to any measure preserved. A common examples are area-preserving systems, e.g., advection in divergence-free ﬂows and Hamiltonian systems. While Birkhoffʼs result provides some comfort, it might still include cases where the exceptional set, i.e., the set where the timeaverages are ill-deﬁned, includes a big portion of the state space. <transition> However, there is a known structure that is a good example of non-existence, an attracting heteroclinic loop. As trajectories are attracted to it, they linger for longer and longer periods of time close to each ﬁxed point in the loop, which prevents the time average from settling down. Most other conﬁgurations in state space have well deﬁned time averages and they can be computed by numerical simulations.

Mezić Research Group

Compact manifold Flow map Observable

Φt : M → M f :M→C
f [Φt (x)]dt
0

M

Non-existence: Attracting heteroclinic loop

˜(x) = lim 1 f T →∞ T ￿

T

Trajectories slow down more and more as they pass equilibria, preventing convergence.

Birkhoff: Limits exist a.e. wrt any measure preserved by ﬂow.

Monday, May 23, 2011

Marko Budišić: Time Averages of Observables

3

Dynamics are modeled by an ﬂow map on a COMPACT manifold. Observables are scalar functions on that manifold. In most of the cases, we will focus on continuous observables. Tilde above a function will indicate that we are taking a time average of the function along the trajectory. Of course, there exist analogous formulations for maps and PDEs. Since the deﬁnition of the time averages includes taking inﬁnite-time limits, it is a valid question to ask whether these limits exist or not. A large class of dynamical systems preserve SOME measure. Birkhoffʼs theorem asserts that time averages converge almost anywhere with respect to any measure preserved. A common examples are area-preserving systems, e.g., advection in divergence-free ﬂows and Hamiltonian systems. While Birkhoffʼs result provides some comfort, it might still include cases where the exceptional set, i.e., the set where the timeaverages are ill-deﬁned, includes a big portion of the state space. <transition> However, there is a known structure that is a good example of non-existence, an attracting heteroclinic loop. As trajectories are attracted to it, they linger for longer and longer periods of time close to each ﬁxed point in the loop, which prevents the time average from settling down. Most other conﬁgurations in state space have well deﬁned time averages and they can be computed by numerical simulations.

Mezić Research Group

Convergence of time-averages
Orbits in regular zones Orbit in mixing zone “Sticky” orbit

images: [Levnajić, Mezić, 2010]

Standard map convergence error (log) Intermittency: trajectories get temporarily trapped around marginally stable periodic islands. [Cvitanović et al., Chaos Book] Coping with slow convergence: • cycle expansions • comp. beneﬁts for adaptive sampling • ﬁnite-time analysis
Difference in averages at 0.6×105 and 1.0×105 steps
Monday, May 23, 2011 Marko Budišić: Time Averages of Observables
4

Even when time averages are well deﬁned, the phenomenon of trajectories lingering close to marginally stable for extended period of time and then moving away can (theoretically) cause arbitrarily slow convergence rates. This is demonstrated on an example of a Chirikov standard map – a Hamiltonian iterated map with a state space that contains both regular and mixing zones that come in contact in a fractal region. Theoretically, the convergence can be arbitrarily slow even when the time averages of observables exist. The graphs on the top show log-log plots difference between ﬁnite time averages and inﬁnite time averages. Convergence of time averages in regular zones is dominated by geometric slope of -1. In mixing zones, convergence is dominated by slope of -1/2. In zones where intermittency occurs, convergence is characterized by intermittent periods of large errors as trajectories get trapped around marginally stable objects. Such zones are known to occur in KAM systems when tori break down. Predrag Cvitanovic championed an analytical method of cycle expansions to compute time-averages in intermittent zones. He proposed that, instead of averaging an observables along chaotic trajectories, we could obtain the result by averaging along periodic orbits that are embedded within the chaotic zones. This is a successful strategy, but it involves determining where unstable periodic orbits lie, which is not an easy task. Therefore cycle expansions do not provide a general-enough replacement for direct averaging. Mixing zones and intermittent zones are sufﬁciently explored by a single trajectory, due to local ergodicity with respect to measures supported on sets containing such zones. If our goal is to compute time averages on a sufﬁciently dense set of points in the state space, we might use this fact in an adaptive sampling of state space. While waiting for the time average to converge, the trajectory explores a fat set in state space, which does not have to be sampled by additional initial conditions later, as all points on the trajectory will have the same timeaverage and can be identiﬁed with the initial condition of the trajectory. Therefore, even if we have to wait for a long period of time, if our goal is coverage of state space, we get additional computational beneﬁt for waiting for time averages to converge. This beneﬁt is especially important when the space is high-dimensional Finally, we might just have to accept that ﬁnite-time averages are the best that we can do. I will revisit this topic brieﬂy at the end of this presentation, but understanding of how to harness ﬁnite-time averages is one of the burning issues that users of time averages will have to face.

Mezić Research Group

Computing averages for continuous systems

Trajectory-based numerical consistency

?

Statistics-based numerical consistency

Monday, May 23, 2011

Marko Budišić: Time Averages of Observables

5

When dynamical system is an iterative map, we have no issues of computing it on a computer. When simulating ODEs and PDEs, we use discretization schemes to approximate continuous trajectories. Most discretization schemes are validated by providing error estimates compared to trajectories of continuous systems. The work of Paul Tupper, Jonathan Mattingly, Andrew Stuart, Xiaoming Wang and others demonstrates that trajectory-consistency is not always enough to ensure that time-averages will be correctly computed when systems are high-dimensional, or have a special structure that needs to be conserved. *** slide transition *** A common observation is that a numerical scheme shouldnʼt introduce additional attractors into dynamics. Both works by Tupper and Wang show that some commonly used schemes fail to satisfy this requirement. Tupper analyzed molecular dynamics (high-DOF Hamiltonian) integrators that conserve energy. If they are not symplectic, they might stabilize periodic orbits that are otherwise unstable for continuous systems. If the continuous system was ergodic, such stable periodic islands might destroy the ergodicity of the numerical scheme resulting in incorrectly computed time-averages. Wang showed that in PDE simulations of heat convection, time-averaged indicators, such as Nusselt number, can be incorrectly computed if the scheme introduces attractors that arenʼt there for continuous system. This occurs in some commonly used numerical schemes. His theoretical result about precompactness of the set of attractors generated by numerical schemes is essentially the same as Tupperʼs, implying that the culprit for non-consistency of time averages is the possible set of attractors that numerical methods might introduce, even at small step sizes. In conclusion, while the work of Sigurgeirsson and Stuart provides hope that time-averages can be computed reliably for maps, we still need to take care in using integrators for continuous systems, paying special attention to systems with symplectic structure and inﬁnite-dimensional systems.

Mezić Research Group

Computing averages for continuous systems Trajectory-based numerical consistency

?

Statistics-based numerical consistency

[P. Tupper, 2007] Numerical ergodicity of Hamiltonian systems Time averages are correct if numerical scheme: 1. ... is ergodic, e.g., has no artiﬁcial limit cycles. 2. ... conserves energy. 3. ... conserves phase space volume (symplectic).

Symplectic schemes perform better than energy-conserving ones.

[X. Wang, 2010] Simulation of cts. inﬁnite-dimensional dissipative systems Time averages are correct if system is uniformly cts. and: 1. ... numerical global attractors for small time steps form a pre-compact set. 2. ... numerical scheme is ﬁnite-time uniformly convergent.
Monday, May 23, 2011 Marko Budišić: Time Averages of Observables

Proof-of-concept numerical scheme for computation of Nusselt no. in a convection model.
5

When dynamical system is an iterative map, we have no issues of computing it on a computer. When simulating ODEs and PDEs, we use discretization schemes to approximate continuous trajectories. Most discretization schemes are validated by providing error estimates compared to trajectories of continuous systems. The work of Paul Tupper, Jonathan Mattingly, Andrew Stuart, Xiaoming Wang and others demonstrates that trajectory-consistency is not always enough to ensure that time-averages will be correctly computed when systems are high-dimensional, or have a special structure that needs to be conserved. *** slide transition *** A common observation is that a numerical scheme shouldnʼt introduce additional attractors into dynamics. Both works by Tupper and Wang show that some commonly used schemes fail to satisfy this requirement. Tupper analyzed molecular dynamics (high-DOF Hamiltonian) integrators that conserve energy. If they are not symplectic, they might stabilize periodic orbits that are otherwise unstable for continuous systems. If the continuous system was ergodic, such stable periodic islands might destroy the ergodicity of the numerical scheme resulting in incorrectly computed time-averages. Wang showed that in PDE simulations of heat convection, time-averaged indicators, such as Nusselt number, can be incorrectly computed if the scheme introduces attractors that arenʼt there for continuous system. This occurs in some commonly used numerical schemes. His theoretical result about precompactness of the set of attractors generated by numerical schemes is essentially the same as Tupperʼs, implying that the culprit for non-consistency of time averages is the possible set of attractors that numerical methods might introduce, even at small step sizes. In conclusion, while the work of Sigurgeirsson and Stuart provides hope that time-averages can be computed reliably for maps, we still need to take care in using integrators for continuous systems, paying special attention to systems with symplectic structure and inﬁnite-dimensional systems.

Chaos 20, 033114 2010

Mezić Research Group

˜ ˜ f ◦ Φt ≡ f Time-averages are invariants of motion.
Trajectories

Φt (x)

033114-6 033114-6

An observable
Z. Levnajić and I.I.Mezić Z. Levnajić and Mezić

f (x)

Time averaged observable

Chaos 033114 2010 Chaos 20,20, 033114 2010

˜ f (x)

M = T2

Level-sets of time-averaged observables partition the state space into invariant sets.
Monday, May 23, 2011 Marko Budišić: Time Averages of Observables
6

The ﬁrst important property of time-averages is their invariance with respect to dynamics. It is common knowledge that invariants of motion, such as energy and momenta in some mechanical systems, play a strong role in understanding the dynamics of systems. However, as said before, we can compute time-averages for most dynamical systems. What role do time-averages play there? this presentation is a Chirikov standard map. It is a Hamiltonian system evolving on a 2-torus, completely regular discrete shear, to global chaos. Letʼs visualize time-averages of an observable that alternates signs on a checkerboard pattern. On the rightmost image I am coloring the level sets of the time-averaged observable.

uation 7 left and the corresponding phase space portraits right done f = Dynamical system I will be using 0.18. The grid cos 2 y . Top row: = 0, middle: = 0.09, bottom: = throughout ations. a parameter epsilon that takes it from a with

quency. In the context of standard map it is convenient to FIG. 2. Color online The graph of the wavelet W as deﬁned by Eq. 8 in a . Its time average under the dynamics of the standard map equation 7 for e a slightly bigger frequency for y coordinate—since the in b = 0.16 in c , and Grid: 800 800 sets. iterations. As a consequence of invariance, level-sets of = 0.09Color ,online The observables dareas deﬁned by and tfinal = 50 a000Its time average lookingdynamics of the standardan averaged time-averaged graph of = 0.18 in . W invariantEq. 8 inTherefore, by under the at level-sets of map equation 7 for the wavelet . nsport in x direction is much faster, the function averages FIG. 2. observable, we can immediately establish = 0.09 in b , which cregions of state space there = 50 no iterations. between = 0.16 in , and = 0.18 in d . Grid: 800 800 and tfinal is 000 transport. Just from looking at the t to zero rather quickly in this direction. Time-average and red. ots rightmostFourier basis functions with increasing fre-that there is no transport between regions colored light-blue 31 for several ﬁgure, we can immediately see obtained, but in our context it is relevant to study how quency are shown in Fig. 3. Note the relationship between the used frequency and the scale of the best-visualized phase convergence properties depend on the type of trajectory of f. obtained,31 but in our context it is relevant to study how quency are shown in Fig. 3. Note the relationship between space details. Thus, in this section we study numerically the convergence the used frequency and the scale of the best-visualized phase convergence properties depend on the type of trajectory of f. While the Fourier functions give smoothly colored plots, properties of time averages in order to estimate the precision space details. Thus, in this section we study numerically the convergence wavelets produce sharper and more detailed plots with a betof the values obtained and relate them to the different types While the Fourier functions give smoothly colored plots, properties of time averages in order to estimate the precision ter distinction between the independent invariant sets. Howof orbits. For simplicity we use only Fourier functions the wavelets produce sharper sets still happen to plots with a betof the values obtained and relate them the tth partial time ever, different invariant and more detailed be assigned the results for wavelets are similar . Consider to the different types ter distinction between thethis, we use the full power of the of orbits. a function f given by same colors. To remedy independent invariant sets. Howaverage forFor simplicity we use only Fourier functions the ever, different invariant sets Sec. happen to be assigned the results for wavelets are similar . Consider the tth partial time ergodic partition concept in still V.

same colors. To remedy this, we use the full power of the ergodic partition concept in Sec. V.
IV. THE CONVERGENCE PROPERTIES

IV. THE is well known that time averages of functions under It CONVERGENCE PROPERTIES dynamics of measure-preserving maps can converge arbiIt is well see Ref. 30 for averages of related results . trarily slowlyknown that timediscussion and functions under dynamics of measure-preserving functions bounds on rate of However, for speciﬁc continuous maps can converge arbiconvergence see Ref. 30 for discussion and related results trarily slowly are computable.31 Bounds that depend only on. the maximum absolute value of continuous function rate of However, for speciﬁc continuousafunctions bounds on can be

average for a function f given by t−1 1 t f Tk x0,y 0 , 9 f x0,y 0 = t k=0 t−1 1 f Tk x0,y , 9 f t x0,y 0 t = with limt→ f x0t, k=0 = f x0 , y 0 0 which we assume exists for y0 all grid-points x0 , y 0 . The difference with limt→ f t x0 , y 0 = f x0 , y 0 which we assume exists for all grid-points ,y 0 , y 0f t .xThe difference t = f x x − ,y 10
0 0 0 0

convergence are computable. Bounds that depend only on the maximum absolute value of a continuous function can be

31

is a sequence whose asymptotic behavior is to be studied in t = f x0,y 0 − f t x0,y 0 10

is a sequence whose asymptotic behavior is to be studied in

Mezić Research Group

Intersections of invariant partitions are again invariant partitions. Intersecting increases the resolution of information.

Mezić

˜ f g ](x) ˜˜ [f ∩(x)
Transport?

033114-7 Chaos 20, 033114 2010

˜ ˜˜(x) [f ∩ gg∩ . . . ](x)
Transport?

Ergodic theory and visualization. I

Repeating the procedure for linearly independent observables limits to the ergodic partition – collection of ergodic sets.

[Susuki, Mezić, 2009]
Monday, May 23, 2011 Marko Budišić: Time Averages of Observables

[Levnajić, Mezić, 2010]
7

There are two questions that are left by looking at a single averaged observable: 1. Is there a transport within any region of uniform color? 2. Does choice of the observables matter?

Different observables will result in different level-sets of time-averaged observables. Each of these level-set partitions will be an invariant partition. Single invariant partition does not answer whether there is transport inside of individual level sets. *** slide transition *** However, deﬁned by Eq. 8 in a . Its time partitions generated the standard map equation 7 for the wavelet W asby intersecting invariant average under the dynamics of from different observables we can increase the resolution of information and get ﬁner = 0.18 in d . Grid: 800 800 thetfinal = 50 000 iterations. information about and level sets. Continuing this process for linearly independent observables reﬁnes the invariant partition more and more, limiting to the ergodic partition. In it, every set is an ergodic set. Within any ergodic set, any two points are connected by transport. In this sense, ergodic partition extends the notion of foliation of the state space into invariant tori for completely integrable dynamical systems. obtained,31 but in our context it is relevant to study how Note the relationship between ale of the best-visualized phase convergence different time-averages of trajectory very Therefore, combining information from properties depend on the typecan provideof f. detailed information about global dynamical features of the Thus, in this section we study numerically the convergence state space. Yoshihiko Susukiʼs paper is about applications of ergodic partition visualization to analysis of continuous power systems. ns give smoothly colored plots, properties of time averages in order to estimate the precision more detailed plots with a betof the values obtained and relate them to the different types dependent invariant sets. Howof orbits. For simplicity we use only Fourier functions the still happen to be assigned the results for wavelets are similar . Consider the tth partial time , we use the full power of the average for a function f given by Sec. V.
1 f Tk x0,y 0 , f x0,y 0 = t k=0
t t−1

ROPERTIES

FIG. 3. Color online Plots for various time-averaged functions for the standard map equation 7 w 9 resonant islands are revealed when the frequency of the function is increased. Grid: 800 800 and

me averages of functions under ving maps can converge arbidiscussion and related results . ous functions bounds on rate of 31 Bounds that depend only on of a continuous function can be

with limt→ f t x0 , y 0 = f x0 , y 0 which we assume exists for all grid-points x0 , y 0 . The difference

relation to the initial point x0 , y 0 and the -value, depending on the phase space regions with different dynamical bet = f x0,y 0 − f t x0,y 0 10 haviors. We consider the function f = cos 2 y cf. Fig. 1 , is a sequence whose asymptotic behavior is to be studied in = f t for t = 108, and consider the ﬁrst 106 iterations. deﬁne f

A. The regular

For all the averages of con

Mezić Research Group

Intersections of invariant partitions are again invariant partitions. Intersecting increases the resolution of information.
033114-15 Ergodic theory and visualization. I

˜ f g ](x) ˜˜ [f ∩(x)

˜ ˜˜(x) [f ∩ gg∩ . . . ](x)

Chaos 20, 033114 2010

Transport? Transport?

coordinates u1 , v1 with the parameter 2 and an integrable 13, obtained for 4D grid 13 13 13 13 initial points as twist map in u2 , v2 . Given this structure, an interesting way follows: of varying parameters of the map equation 13 is by letting • The lattice-grid is set in full 4D phase space, and iterated =2 . We slice the 4D phase space by a 2D section with using the Froeschlé map equation 13 with = =1 for ten ﬁxed x2 , y 2 = 0,0 , and consider a grid of 500 500 initial iterations, in order to randomize the points starting from grid-points in x1 , y 1 -space, plotting the time averages of a +cos Budišić: Time Averages theObservables single function f 2 =2 cos 2 y 1 Marko 2 y 1 cos 2 y 2 Monday, May 23, 2011 of uniform lattice-grid in the case of = =0 might create 7 trajectories that strongly overlap . +cos 12 x2 +cos 12 x1 obtained by running the dynamics There are two questions thatfor 200left by looking at a single averaged Note • From thus obtained 13 13 13 13=28 561 initial 4D are 000 iterations. The results are shown in Fig. 12. observable: points, set 13 13 13=2197 to have y 1 =0, y 2 =0 , an1. Is there a transport withinthe departure from the known color? map’s phase space any region of uniform standard 1 1 due to increasing interaction between the maps. For small other 13 13 13 to have y 1 = 2 , y 2 = 2 , and so on, de2. Does choice of the observables matter? values, the phase space maintains its regular structure with pending on how many resonances are to be visualized. only small and localized chaos similar to standard map for • Run the dynamics for tfinal =100 000 iteration starting from Different observables will result in .different level-sets of two-dimensional intersmall The coloring here indicates time-averaged observables. Each of these level-set partitions will be an invariant thus created ensemble of initial 4D points. partition. Single invariant partition does not invariant sets with the selected 2D transport inside of individual account for quasipanswer whether there is • As the result, majority of points will level sets. sections of the real 4D *** slide transition *** phase space section. It appears that the global chaotic transieriodic orbits, while the selected points will visualize the tion in partitions generated from the phase space However, by intersecting invariantthe sense of vertical transport throughdifferent observables resonances increase the resolution of information and get ﬁner chosen we can occurs around =2 =0.05.process for linearly independent observables reﬁnes the invariant partition more and more, information about the level sets. Continuing this Note that in The structure Qe is an ergodic set. = =0 any limiting to the ergodic partition. In it, everyofset for the Froeschlé map forWithin is ergodic set, Fig. 13, all the mentioned connected by transport. In this any two points are phase space features are visualized as expected. The chosen resonances for clearly a product of two “rational combs” discussed earlier. sense, ergodic partition extends the notion of foliation of the state space into invariant tori for completely integrable dynamical systems. 1 1 1 1 this case are y 1 =0, y 2 =0 , y 1 = 2 , y 2 = 2 , y 1 = 3 , y 2 = 3 , This product is topologically a torus to which 2 2 and y = 3 , y 2 = 3 . • a linefrom different time-averages can 2provide very1detailed information about global dynamical features of the of length 1/q is attached at every point y 1 , y such Therefore, combining information We examine the three-function MSPs for the Froeschlé that one of is y i’s is irrational and the other rational, p/q state space. Yoshihiko Susukiʼs paper the about applications of ergodic partition with =2 0 byto analysis of continuous12 map visualization employing 4D grid of 12 12 power systems. with p/q an irreducible fraction , 12 initial grid-points and the same number of total itera• a rectangle of sides q1 ,q2 is attached at every point tions. The ensemble of initial points with selected resonance y 1 = p1 /q1 , y 2 = p2 /q2 , where p1 /q1 and p2 /q2 are irrepoints is constructed as above. We consider structural ducible fractions. changes in the MSPs by changing the -value, as shown in Fig. 14. As the coupling intensity grows, the structure from Since two rectangles generically do not intersect in a ﬁve the previous ﬁgure is destroyed, in a way similar to what dimensional 5D Euclidean space, the appropriate embedobserved for the standard map cf. Fig. 8 . In particular, note ding dimension for the Froeschlé map is ﬁve. To avoid selfthe different mechanisms of destruction of resonances and intersections in the embedding, we would thus need to conirrational orbits. The MSP’s structure reports a given level of sider at least ﬁve independent functions. This information is regularity in the phase space persisting for a certain range of important when choosing the number of functions that are coupling parameter strengths. This is a manifestation of used in clustering and approximation of the ergodic partition.

Repeating the procedure for linearly independent observables However, since we cannot graphically represent the 5D space u =x −x , v =y −y , here three-function visualized limits to the the system to two maps: a standard map in –results, we considerexample with =ergodic insets. ergodic partition 3Dcollection of =0MSPshown in Fig. which reduce embedding. An is
2 1 2 2 1 2

FIG. 11. Color online Approximations of the ergodic partition for the standard map equation 7 with =0.12. Left: three-function approximation using two functions from Fig. 10 and the function sin 4 x sin 4 y . Right: four-function approximation using these three functions in addition to sin 10 x sin 10 y . The grid 800 800 is used for all time averages and the cell division for both approximations is L=50.

[Susuki, Mezić, 2009]

[Levnajić, Mezić, 2010]

Mezić Research Group

Ergodic Quotient (EQ) and empirical measures
State space portrait

Averaging

Φt (x) ⊂ M

∀f

from cts. basis ￿

Ergodic quotient

˜ (x), f2 (x), . . . ⊂ l∞ ˜ f1 ￿

Empirical measures represent time averaging through spatial integration.

µx

˜ f (x) = ￿

f dµx
M
x Φt(x)

EQ is the set of weak-representatives of empirical measures for all initial cond.

Choice of topology on EQ deﬁnes coherent structures, e.g., discrete metric topology corresponds to ergodic partition.
Monday, May 23, 2011 Marko Budišić: Time Averages of Observables
8

Going from state space portrait to time-averages quotients the state space by ergodic partition – we lose the ability to resolve features inside of ergodic sets. That is OK, because we only need one initial condition to understand any ergodic set since the dynamics restricted to any ergodic set is a system ergodic with respect to a measure supported on the ergodic set. What do we gain? We gain the ability to compare dynamics between different parts of the systems in a robust way. Averaging is a linear, bounded functional on the space of continuous functions that can be represented by delta-like invariant measures, supported along trajectories. Empirical measures are weak-limits (in inﬁnite time) of such invariant measures. Averaging a basis of continuous functions produces a weak representation of empirical measures. Ergodic quotient is then the set of weak-representatives of empirical measures. The coherent structures that we can identify in state space are directly related to topologies imposed on the ergodic quotient. What do I mean by that? Example: assume we have sourced a lot of initial conditions on the state space and computed their time averages. A discrete metric topology is the one where objects are at distance zero only to themselves, and inﬁnity to everybody else. Taking an inﬁnite intersection of level sets of observables achieves the same thing - any element of the limiting partition contains initial conditions which had the same averages for all functions. Therefore, the ergodic partition, mentioned on the previous slide, is equivalent to ergodic quotient viewed through the lens of discrete metric topology. But we can consider other, continuous, topologies.

Mezić Research Group

Coherent structures: similarity in residence times
[MB, Mezić, 2009] State space portrait
Elliptic Dufﬁng Gyres

Φt (x) ⊂ M
Ergodic quotient ￿ ￿

˜ (x), f2 (x), . . . ⊂ l∞ ˜ f1

Negative Sobolev space norms compare residence times of trajectories. [Mathew, Mezić, 2010] Easily computed on the ergodic quotient – low pass ﬁlter! Intrinsic coordinates recovered by diffusion modes on data.
Monday, May 23, 2011 Marko Budišić: Time Averages of Observables
9

Endowing the empirical measure space with a negative-index Sobolev space norm topology (proposed by G. Mathew) results deﬁnes a particular type of coherent features. In that topology, trajectories are deemed to be a part of the same coherent structure if their residence times sets in the state space are similar. Moreover, by selecting a proper index of the Sobolev space norm we can make sure that differences across larger spatial scales matter more than differences over small scales. These types of norms amount to using Fourier basis as observables, and comparing the distance between trajectories is comparing the energy of the sequences with appropriate discounting of the higher wave numbers. In ergodic quotient, that lives in a high-dimensional space, certain state space features will be mapped to low-dimensional structures, even if the state space itself is fairly high-dimensiona. 1. The ﬁrst image depicts an elliptic well that maps to a string in the ergodic quotient. This signiﬁes that ergodic sets in an elliptic zone (periodic orbits in two-dimensional state space) can be parametrized by a single conserved quantity. 2. The second image shows a Dufﬁng-type of behavior, or a double-well, where the inside “libration” orbits each map to separate strings which end up joining when we cross the lip of wells into the “revolution” orbits. 3. Conversely, convection cells do not have revolution orbits that would join two strings together. Instead they have hyperbolic ﬁxed points in their corners that ensure that the time averages stay apart. These low-dimensional structures inhabit an inﬁnitely-dimensional sequence space. Using a single averaged observable, we are coloring the state space according to projections of such features to a one-dimensional space. In my own research, I extract the intrinsic coordinates of such structures and use them to visualize features of the state space. This provides us with low-dimensional embeddings of ergodic quotient that maximize the detail shown. *** slide transition *** If we know which trajectories mapped to certain features in EQ, we can assign colors to different features of EQ, and use these colors to color the state space portrait in order to visualize coherent structures. *** slide transition *** For example, take the Arnold-Beltrami-Childress ﬂow: steady state ﬂow characterized by prominent vortices. We visualized the vortices extracting features in the ergodic quotient and using them to color trajectories. The major elliptic region was colored by a gradient, while side vortices were picked out as disconnected components and each given its own color. However, we have done that by studying only results of simulation, which could be potentially done in a model-free setting, e.g., for experimental data, but that is consistent with “physical” properties, such as vorticity of the velocity ﬁeld.

Mezić Research Group

Coherent structures: similarity in residence times
[MB, Mezić, 2009] State space portrait
Elliptic Dufﬁng Gyres

Φt (x) ⊂ M
Ergodic quotient ￿ ￿

˜ (x), f2 (x), . . . ⊂ l∞ ˜ f1

Negative Sobolev space norms compare residence times of trajectories. [Mathew, Mezić, 2010] Easily computed on the ergodic quotient – low pass ﬁlter! Intrinsic coordinates recovered by diffusion modes on data.
Monday, May 23, 2011 Marko Budišić: Time Averages of Observables
9

Endowing the empirical measure space with a negative-index Sobolev space norm topology (proposed by G. Mathew) results deﬁnes a particular type of coherent features. In that topology, trajectories are deemed to be a part of the same coherent structure if their residence times sets in the state space are similar. Moreover, by selecting a proper index of the Sobolev space norm we can make sure that differences across larger spatial scales matter more than differences over small scales. These types of norms amount to using Fourier basis as observables, and comparing the distance between trajectories is comparing the energy of the sequences with appropriate discounting of the higher wave numbers. In ergodic quotient, that lives in a high-dimensional space, certain state space features will be mapped to low-dimensional structures, even if the state space itself is fairly high-dimensiona. 1. The ﬁrst image depicts an elliptic well that maps to a string in the ergodic quotient. This signiﬁes that ergodic sets in an elliptic zone (periodic orbits in two-dimensional state space) can be parametrized by a single conserved quantity. 2. The second image shows a Dufﬁng-type of behavior, or a double-well, where the inside “libration” orbits each map to separate strings which end up joining when we cross the lip of wells into the “revolution” orbits. 3. Conversely, convection cells do not have revolution orbits that would join two strings together. Instead they have hyperbolic ﬁxed points in their corners that ensure that the time averages stay apart. These low-dimensional structures inhabit an inﬁnitely-dimensional sequence space. Using a single averaged observable, we are coloring the state space according to projections of such features to a one-dimensional space. In my own research, I extract the intrinsic coordinates of such structures and use them to visualize features of the state space. This provides us with low-dimensional embeddings of ergodic quotient that maximize the detail shown. *** slide transition *** If we know which trajectories mapped to certain features in EQ, we can assign colors to different features of EQ, and use these colors to color the state space portrait in order to visualize coherent structures. *** slide transition *** For example, take the Arnold-Beltrami-Childress ﬂow: steady state ﬂow characterized by prominent vortices. We visualized the vortices extracting features in the ergodic quotient and using them to color trajectories. The major elliptic region was colored by a gradient, while side vortices were picked out as disconnected components and each given its own color. However, we have done that by studying only results of simulation, which could be potentially done in a model-free setting, e.g., for experimental data, but that is consistent with “physical” properties, such as vorticity of the velocity ﬁeld.

Mezić Research Group

Coherent structures: similarity in residence times
[MB, Mezić, 2009] State space portrait
Elliptic Dufﬁng Gyres

Φt (x) ⊂ M
Ergodic quotient ￿ ￿

˜ (x), f2 (x), . . . ⊂ l∞ ˜ f1

Negative Sobolev space norms compare residence times of trajectories. [Mathew, Mezić, 2010] Easily computed on the ergodic quotient – low pass ﬁlter! Intrinsic coordinates recovered by diffusion modes on data.
Monday, May 23, 2011

Primary/secondary vortices of an ABC ﬂow
9

Marko Budišić: Time Averages of Observables

Endowing the empirical measure space with a negative-index Sobolev space norm topology (proposed by G. Mathew) results deﬁnes a particular type of coherent features. In that topology, trajectories are deemed to be a part of the same coherent structure if their residence times sets in the state space are similar. Moreover, by selecting a proper index of the Sobolev space norm we can make sure that differences across larger spatial scales matter more than differences over small scales. These types of norms amount to using Fourier basis as observables, and comparing the distance between trajectories is comparing the energy of the sequences with appropriate discounting of the higher wave numbers. In ergodic quotient, that lives in a high-dimensional space, certain state space features will be mapped to low-dimensional structures, even if the state space itself is fairly high-dimensiona. 1. The ﬁrst image depicts an elliptic well that maps to a string in the ergodic quotient. This signiﬁes that ergodic sets in an elliptic zone (periodic orbits in two-dimensional state space) can be parametrized by a single conserved quantity. 2. The second image shows a Dufﬁng-type of behavior, or a double-well, where the inside “libration” orbits each map to separate strings which end up joining when we cross the lip of wells into the “revolution” orbits. 3. Conversely, convection cells do not have revolution orbits that would join two strings together. Instead they have hyperbolic ﬁxed points in their corners that ensure that the time averages stay apart. These low-dimensional structures inhabit an inﬁnitely-dimensional sequence space. Using a single averaged observable, we are coloring the state space according to projections of such features to a one-dimensional space. In my own research, I extract the intrinsic coordinates of such structures and use them to visualize features of the state space. This provides us with low-dimensional embeddings of ergodic quotient that maximize the detail shown. *** slide transition *** If we know which trajectories mapped to certain features in EQ, we can assign colors to different features of EQ, and use these colors to color the state space portrait in order to visualize coherent structures. *** slide transition *** For example, take the Arnold-Beltrami-Childress ﬂow: steady state ﬂow characterized by prominent vortices. We visualized the vortices extracting features in the ergodic quotient and using them to color trajectories. The major elliptic region was colored by a gradient, while side vortices were picked out as disconnected components and each given its own color. However, we have done that by studying only results of simulation, which could be potentially done in a model-free setting, e.g., for experimental data, but that is consistent with “physical” properties, such as vorticity of the velocity ﬁeld.

Mezić Research Group

Continuous Dynamical Indicators
Ergodicity defect
[Scott et al., 2010]

Quality of mixing and sampling
[Mathew, Mezić, 2010]

Haar wavelets as observables.
436

Fourier basis as observables.
436 G. Mathew, I. Mezić / Physica D 240 (2011) 432–442 G. Mathew, I. Mezić / Physica D 240 (2011) 432–442

Replaces “ergodic/non-ergodic” with continuous measure of area-ergodicity.

(a) Time, t = 0.0. (a) Time, t = 0.0.

(b) Time, t = (b) Time, t = 6.0. 6.0.

(c) Time, t = 12.0. Time, t = 1 (c)

Applied to design of mixing, optimal sampling of a distribution.

Monday, May 23, 2011

Marko Budišić: Time Averages of Observables

The application of analysis of topology of empirical measures goes beyond visualization of features. Sherry Scott and George Mathew have both applied this concept to convert binary descriptions – ergodic/non-ergodic as Now, µ is defined and mixing/not-mixing into continuous quantiﬁcations o generates trajectories such that the fraction Now, µ is defined as generates foliage regions close to zero. of ergodicity and mixing. Continuous quantiﬁcations can then be used as design parameters, for example to drivetheFig. 2 shows a plot ofregions is ofsuchcovera the systemistotrajectoriesclose to zert ergodicity that Ter(x) the foliage the decay the µ(x) = ￿ . (34) time. The decay is not monotonic and in decay Ter = ￿ Ter(x) . Fig. 2 shows a plot of the partic with respect to a particular measure. U µ(x) (y)dy (34) The goal will often determine the appropriate choice of observables. Sherry Scott looked at a multiscale measure of ergodicity by taking Haarʼs wavelets as observables, which allow precise tuning of scale and position of features we are interested in.

Fig. 1. Snapshots at various times of the agent trajectories generated by the SMC algorithm with first-order dynamics. One can observe the multisca Fig. 1. Snapshots at various times smaller and trajectories generated by The spacing between the trajectories becomes of the agent smaller as time proceeds. the SMC algorithm with first-order dynamics. One can obse The spacing between the trajectories becomes smaller and smaller as time proceeds.

(d) Time, t = 18.0. (d) Time, t = 18.0.

10

George Mathew, on the other hand, applied this concept to design of trajectories. In his work, he needed to plan trajectories for searchers, e.g., unmanned vehicles (represented as dots), such that they collaboratively sample a particular known prior distribution on the state space. By comparing weak representations of empirical measures of searchers and the weak representation of the target distribution, he designed optimal controllers that drove searchers to quickly sample the desired distributions. The same idea was applied to design of mixing trajectories.

the decay is time. The This is partly due to the f irregular. decay is not monotonic Ter(y)dy U The snapshots in Fig. 1 were generated with a simulation of functions used decay is irregular. This is partl the and the fixed time-stepping 3 agents and with the feedback law were generated with ais a The snapshots in Fig. 1 in (26). The domain U simulation of differential equations. used shows a plot of tim functions Fig. 2 and the fixed th unit square domain [0,with the, feedback law in (26). The domain spent a the agents outside the foliage regions 3 agents and 1] × [0 1], the total simulation time is U is by differential equations. Fig. 2 show T = 18, unit umax = 5.0. The basis ] × [0, 1]fk the total as in (9) and square domain [0, 1 functions , used are simulation timefraction approaches one, as time proceeds this is spent by the agents outside the fo 5 for K1 , K2T = 018,, and, umax The5.0. The basis functions fk used are asagents spend very little time in the foliage r = , 1 . . . 50. = initial positions of the agents are in (9) this fraction approaches one, as t chosen randomly. = 0, 1, . . . , 50.5 The initial positions of the agents areof the norm of the vectors Bj (t ). As on a plot for K1 , K2 agents spend very little time in The differential equations for the closed-loop dynamics of the ￿Bj (t )￿2 comes close to zero often, the vector chosen randomly. a plot of the norm of it never sta extended system as described by (19) for the closed-loop dynamics of the The differential equations are solved using a fixed ￿Bj (t )￿2 comes close to zero ofte time-step 4th order system as described Note that thesolved using a fixed 3.2. Second-order dynamics extended Runge–Kutta method. by (19) are Neumann boundary condition of the basis functions fk guarantees that the time-step 4th order Runge–Kutta method. Note that the Neumann 3.2. Second-order dynamics velocity component normal to the boundary at the boundaries of boundary condition of the basis functions fk guarantees that The dynamics of the extended system tha the the domain is always zero and therefore the agents never escape tions and of the agents velocity component normal to the boundary at the boundaries of velocities dynamics of and the varia The the extende the domain. is described as: the domain is always zero and therefore the agents never escape tions and velocities of the agents From the snapshots in Fig. 1, one can see the multiscale nature the domain. ˙ xj (τ ) = vj (τis described as: ) of the algorithm. The spacing between the trajectories becomes From the snapshots in Fig. One shouldsee the multiscale nature u (τ ) 1, one can note that the vj (τ ) = j x (τ ) = v (τ ) ˙ smaller and smaller as time proceeds. ˙j j of the algorithm. The spacing between the going over becomes trajectories trajectories are not such that they completely avoid ˙k (τ ) = Wk (τ(τ ) = u (τ ) S the vj ) ˙ smaller and smaller as time proceeds. One algorithm j the foliage regions as in collision avoidance. Rather, theshould note that N trajectories are not such that they completely avoid going over ￿ (τ ) = W (τ ) ˙ Sk k ˙ Wk (τ ) ∇ fk (xj (τ )) · vj (τ ). the foliage regions as in collision avoidance. Rather, the algorithm =

N ￿ j=1 ˙ 5 For numerical purposes, we need to have a cutoff for the number of Fourier Wk (τ ) = ∇ f (xj (τ )) · v (τ ). The forces on the agents areksubject to jthe co efficients used. The exact effect of this cutoff on the performance of the algorithm j=1 max . We are going to use the same model p is a challengingFor numerical purposes, Roughly speaking, a cutoff forcutoff, the 5 and interesting problem. we need to have for a lower the number ofFFourier The forces on the agents are sub proach as with first-order dynamics. The cost gaps left between the trajectories of the agents will be bigger than those obtained efficients used. The exact effect of this cutoff on the performance of the algorithm with a higher cutoff. ing to use here .is a weighted sumusethe firs Fmax We are going to of the sa is a challenging and interesting problem. Roughly speaking, for a lower cutoff, the proach as with first-order dynam gaps left between the trajectories of the agents will be bigger than those obtained with a higher cutoff. ing to use here is a weighted su

Mezić Research Group

˜ f ￿→ f
[Mezić, Banaszuk, 2004.]
8
1

Focus on dynamics of T : M → M, T ≡ Φτ “on the attractor”, systems with recurrent dynamics.

Koopman Operator

Uf = f ◦ T
400

linear, unitary on attractor captures full nonlinear dynamics
U=
n ￿ ω ei2πωk PT k + Uc

C.W. Rowley, I. Mezi´, S. Bagheri, P. Schlatter, and D.S. Henningson c k=1
(a)
0.5

(b)
300

Im{λj } ￿

vj ￿
0

˜ ˜ Uf = f

200

Time-averaging projects onto eigenspace at 1 (invariant functions). Complex eigenspaces reveal periodicity of dynamics.

!0.5

100

˜ ˜ U fω = ei2πω fω
St

!1

!1

!0.5

Re{λj }

0

0.5

1

0 0

0.03 0.06 0.09 0.12 0.15 0.18 0.21 0.24

Figure 2. (a) The empirical Ritz values λj . The value corresponding to the ﬁrst Koopman mode is shown with the blue symbol. (b) The magnitudes of the Koopman modes (expect the ﬁrst Koopman each frequency. In bothin the crossﬂow. [Rowley etfrom 2009] white, depending one) at spectrum of a jet ﬁgures, the colors vary smoothly al., red to on the magnitude of the corresponding mode. Budišić: Time Averages of Observables Monday, May 23, 2011 Marko

11

So far I have talked about the relationship between state space features and the time-averaged observables. What is the relationship between spectrumtime-average? frequency content u1 (ω) of u1 (t).we are composing an observable with the evolution of the shows the At every step of the averaging process, The peak frequency corresponds ˆ observable and its system.to a vortex shedding ofto that operation iswith the Strouhal number St ≡ f D/Vjet = 0.0174. Operator that corresponds wake vortices the Koopman operator. It is a linear, and it is dual to the Perron-Frobenius operator that evolves densities of initial conditions. On the attractor, Koopman operator is unitary with respectjet the L2 function space over any In ﬁgure 1(d,f), a second probe located a few jet diameters along the to trajectory invariantx2 = (12, 6, 2), shows a second oscillation that can be identiﬁed with the shedding of measure on the attractor. Let me stress that the operator itself is LINEAR in observables but that it captures full dynamics of a NONLINEAR dynamical system. order the operator enables us than the low-frequency mode. Note that the peak frequencies The unitarity of of magniture larger to study its spectrum in an effort to understand its behavior. The image shows a numerically computed spectrum of a Koopman spectra for a ﬂuid ﬂow. depending on the location of the probe. of the power operator vary slightly Time-averaging projects the observable4.1. Koopman modes for the Koopan operator, which contains all invariant functions for dynamics. onto the eigenspace at 1 and frequencies The complex eigenspaces reveal periodicity in dynamics of the system corresponding to the frequency of the eigenvalue associated with the In this section we compute the Koopman modes and show that they directly allow an eigenspace. We can access the projection to periodic eigenspaces by harmonic averages, which are introduced on the next slide.

the shear-layer vortices. The peak frequency beats with St = 0.141 which is nearly one

P

identiﬁcation of the various shedding frequencies. The empirical Ritz values λj and the Koopman operator is an instantaneous object; its of ﬂow-ﬁelds {uonly 1 , ﬁnite um−1 } = {u(t = 200), u(t = inﬁnite-time empirical vectors vj of a sequence deﬁnition contains 0 , u a . . . , time advancement of dynamics. The averaging is just , u(t = through with m = 251 are computed at 1 – the space of invariant functions.earlier. to access other 202), . . . a TOOL 700)} which we analyze the eigenspace using the algorithm described If we want eigenspaces, we can modify simple time averages to form harmonic averages. Thus, the transient time (t < 200) is not sampled and only the asymptotic motion in phase space is considered. Figure 2(a) shows that nearly all the Ritz values are on the unit circle |λj | = 1 indicating that the sample points ui lie on or near an attracting set. The Koopman eigenvalue corresponding to the ﬁrst Koopman mode is the time-averaged ﬂow and is depicted with blue symbol in ﬁgure 2(a). This mode, shown in ﬁgure 1(b), captures the steady ﬂow structures as discussed previously. In ﬁgure 2(a), the other (unsteady) Ritz values vary smoothly in color from red to white, depending on the magnitude of the corresponding Koopman mode. The magnitudes deﬁned by the global energy norm ￿vj ￿, and are shown in ﬁgure 2(b) with the same coloring as the spectrum. In ﬁgure 2(b) each mode is displayed with a vertical line scaled with its magnitude at its corresponding frequency ωj = Im{log(λj )}/∆t (with ∆t = 2 in our case). Only the ωj ≥ 0 are shown, since the eigenvalues come in complex conjugate pairs. Ordering the modes with respect to their magnitude, the ﬁrst (2-3) and second (4-5) pair of modes oscillate with St2 = 0.141 and St4 = 0.136 respectively, whereas the third pair of modes (6-7) oscillate with St6 = 0.017. All linear combinations of the frequencies excite higher modes, for instance, the nonlinear interaction of the ﬁrst and third pair results in the fourth pair, i.e. St8 = 0.157 and so on. In ﬁgures 1(e) and (f) the power spectra of the two DNS time signals (black lines)

Mezić Research Group

Harmonic averages
˜ (x) = lim 1 fω N →∞ N
N −1 ￿ n=0

e−i2πωn f [T n (x)]

˜ ˜ U fω = ei2πω fω

[Mezić, Banaszuk, 2004.]

Harmonic Quotient at ω 3

˜ fω1 ˜ fω2 ˜ fω3 . . .

g ω1 ˜ g ω2 ˜ g ω3 ˜ . . .
Discrete Fourier Transform

˜ hω 1 ˜ hω 2 ˜ hω 3 . . .

... ... ... .. .

R. Mohr, I Mezić: Applications to analysis of computer networks (in progress)
Monday, May 23, 2011 Marko Budišić: Time Averages of Observables
12

Harmonic averages are formed from time averages by adding a complex term that rotates with frequency of the eigenvalue we are interested in. Harmonic averages span the complex eigenspaces in the same fashion that time averaged spanning. Harmonic averaging projects an observable to the complex eigenspace. For any initial condition studies, there are two different parameters to choose: an observable and the frequency of averaging. While this might seem daunting, ﬁxing any of those parameters results in familiar objects: 1. Fixing an observable, and computing all frequencies, results in a DFT of an observable. 2. Fixing a frequency, and computing averages of all continuous observables results in harmonic quotients: analogues to ergodic quotients discussed earlier, except oscillating at a certain frequency. Ryan Mohr, also at UC Santa Barbara, is working on applications of these methods to analysis of computer TCP networks. *** slide transition *** To demonstrate what harmonic averages look like, Iʼll turn again to the standard map. Looking at harmonic averages at period 5, we see that averages over features that arenʼt periodic with period 5 are equal to zero. The level sets at non-zero values correspond to periodic sets which return to themseves after 5 time steps. As harmonic averages are complex functions, we can use both real and imaginary part to distinguish the phase within these sets.

Mezić Research Group

Harmonic averages
˜ (x) = lim 1 fω N →∞ N
N −1 ￿ n=0

Standard map

ε = 0.12 f (x, y) = sin(8πx + 2πy)

e−i2πωn f [T n (x)]

˜ ˜ U fω = ei2πω fω

[Mezić, Banaszuk, 2004.]

Harmonic Quotient at ω 3

˜ fω1 ˜ fω2 ˜ fω3 . . .

g ω1 ˜ g ω2 ˜ g ω3 ˜ . . .
Discrete Fourier Transform

˜ hω 1 ˜ hω 2 ˜ hω 3 . . .

˜ ￿fω (x, y)

˜ ￿fω (x, y), ω = 1/5

... ... ... .. .

FIG. 1. Color online Single-function plots of time averages for the standard map equation 7 left and the corresponding phase space portraits right done as 100 iterations of 11 11 random trajectories picked from 0 , 1 2 for the function f = cos 2 y . Top row: = 0, middle: = 0.09, bottom: = 0.18. The grid of 800 800 initial points was used, and the dynamics was run for tfinal = 30 000 iterations.

R. Mohr, I Mezić: Applications to analysis of computer networks (in progress)
Monday, May 23, 2011 Marko Budišić: Time Averages of Observables
12

curve, a different coloration occurs there. This type of chaotic transport in Hamiltonian maps can be approximated by the Markov tree model.27–29 In order to investigate the phase space structure at larger smaller scale, one takes smaller bigger Fourier/wavelet

frequency. In the context of standard map it is convenient to use a slightly bigger frequency for y coordinate—since the transport in x direction is much faster, the function averages out to zero rather quickly in this direction. Time-average plots for several Fourier basis functions with increasing fre-

Harmonic averages are formed from time averages by adding a complex term that rotates with frequency of the eigenvalue we are interested in. Harmonic averages span the complex eigenspaces in the same fashion that time averaged spanning. Harmonic averaging projects an observable to the complex eigenspace. For any initial condition studies, there are two different parameters to choose: an observable and the frequency of averaging. While this might seem daunting, ﬁxing any of those parameters results in familiar objects: 1. Fixing an observable, and computing all frequencies, results in a DFT of an observable. 2. Fixing a frequency, and computing averages of all continuous observables results in harmonic quotients: analogues to ergodic quotients discussed earlier, except oscillating at a certain frequency. Ryan Mohr, also at UC Santa Barbara, is working on applications of these methods to analysis of computer TCP networks. *** slide transition *** To demonstrate what harmonic averages look like, Iʼll turn again to the standard map. Looking at harmonic averages at period 5, we see that averages over features that arenʼt periodic with period 5 are equal to zero. The level sets at non-zero values correspond to periodic sets which return to themseves after 5 time steps. As harmonic averages are complex functions, we can use both real and imaginary part to distinguish the phase within these sets.

Mezić Research Group

Finite-time averages Koopman Operator is an instantaneous object. Do we really need the inﬁnite-time formalism to analyze it?

• ﬁnite-time Lyapunov exponents [Haller, 2010] and more... • ﬁnite-time turbulence quantiﬁers [Foias, Jolly, 2005] • time-averaged velocity [Poje, Haller, Mezić, 1999], [Mezić et al. 2010]

1 T ￿

T 0

1 f [Φt (x)]dt = lim τ →∞ τ ￿

τ

f [Φ(t mod T ) (x)]dt
0

Finite time averaging is equivalent to periodizing the dynamics.
Monday, May 23, 2011 Marko Budišić: Time Averages of Observables
13

As mentioned, Koopman operator is instantaneous object, in case of iterated maps, it composes observable with a single step dynamics. This gives hope that analyzing it through inﬁnite time averages is not necessary. There have been several research efforts to convert inﬁnite time indicators to ﬁnite time. As a glimpse of one path to understand ﬁnite time averages, this identity shows that ﬁnite-time averages are equivalent to inﬁnite time averages for modiﬁed ﬂow map. The modiﬁcation consists of periodizing dynamics of the original time map with the period equal to the time over which ﬁnite-time average was computed. While this might seem artiﬁcial and useless at ﬁrst, this approach is readily used in signal processing, when Discrete Fourier Transform is used to analyze and ﬁlter measured signals that are not necessarily periodic. However Koopman operator frames this approach within the context of full nonlinear dynamics.

Mezić Research Group

Conclusions
Numerical methods should be designed with time-averaging in mind. Trajectory-based consistency does not always imply consistency of time-averages for Hamiltonian and inﬁnite-dimensional systems. Wealth of information is waiting to be managed. Fully nonlinear data can be analyzed by linear tools. Choosing observables: can we do better than the entire function basis? Compressed sensing? Empirical measure reconstruction? Finite-time averages are here to stay. Inﬁnite averaging is just a tool to access the instantaneous Koopman op. What can ﬁnite-time averages tell us about it? Physical data, nonrecurrent systems require ﬁnite-time averages.
Monday, May 23, 2011 Marko Budišić: Time Averages of Observables
14

These are the main take-aways from this talk: 1. Numerical integrators that were built around trajectories of ODEs might not be adequate for computation of statistics. On the other hand, since time averages show increased robustness to noise and numerical errors, as compared to trajectories, we might be able to build averaging-integrators that would be more efﬁcient at computing time averages of trajectories, while ensuring the statistical consistency. 2. Moving the full nonlinear data to the linear world results in a wealth of information stored in averages of observables. At this point, unless we are physically motivated to choose a particular observable, we store the information in a large number of observables. A more selective approach to choosing observable could enable us to reduce computational requirements of the method. It might be possible to use ideas from compressed sensing to do this. A step further would involve full empirical measure reconstruction, effectively viewing the ergodic quotient as input data to the inverse problem. Due to singular nature of a lot of empirical measures, with respect to Lebesgue, makes it difﬁcult to use current inverse problem solutions without encountering large reconstruction artifacts. 3. A requirement for viable methods is the ability to handle experimental data, and truly aperiodic data. It seems that to do this, we have to abandon the inﬁnity and embrace ﬁnite-time averages. Understanding how ﬁnite-time features relate to properties of the system is an important step in acceptance of time-averaging as a standard tool in analysis and design.

Mezić Research Group

Main references
• • • • • • • • • • •
Budišić and Mezić. An approximate parametrization of the ergodic partition using time averaged observables. Proceedings of the 48th IEEE Conference on Decision and Control, Shanghai, China (2010) pp. 3162-3168 Cvitanović et al. Chaos: Classical and Quantum, ChaosBook.org, Niels Bohr Institute, Copenhagen, 2011 Levnajić and Mezić. Ergodic theory and visualization. I. Mesochronic plots for visualization of ergodic partition and invariant sets. Chaos (2010) vol. 20 (3) pp. 19 Mathew and Mezić. Metrics for ergodicity and design of ergodic dynamics for multi-agent systems. Physica D (2010) vol. 240 pp. 432-442 Mattingly et al. Convergence of numerical time-averaging and stationary measures via Poisson equations. SIAM J. Numer. Anal. (2010) vol. 48 (2) pp. 552-577 Mezić and Banaszuk. Comparison of systems with complex behavior. Physica D (2004) vol. 197 (1-2) pp. 101-133 Rowley et al. Spectral analysis of nonlinear ﬂows. J. Fluid Mech. (2009) Scott et al. Capturing deviation from ergodicity at different scales. Physica D (2009) vol. 238 (16) pp. 1668-1679 Sigurgeirsson and Stuart. Statistics From Computations. Foundations of Computational Mathematics, LMS Lecture Note Series, Cambridge University Press (2001) vol. 284 pp. 323-344 Tupper. Computing statistics for Hamiltonian systems: A case study. Journal of Computational and Applied Mathematics (2007) Wang. Approximation of stationary statistical properties of dissipative dynamical systems: time discretization. Math. Comp (2010)
Marko Budišić: Time Averages of Observables
15

Monday, May 23, 2011