# The Nature of Mathematical Modeling

Neil Gershenfeld

CAMBRIDGE
UNIVERSITY PRESS

3 Laplace Transforms 2.1.8 Problems 3 Partial Differential Equations 3.1 Selected References page x xi 1 3 Part One: Analytical Models 2 Ordinary Differential and Difference Equations 2.1 Linear Differential Equations 2.5 Discrete Time Equations 2.3.1 Rectangular Coordinates 3.4 Perturbation Expansions 2.1 The Origin of Partial Differential Equations 3.2 Cylindrical Coordinates 3.4 Transform Techniques 3.r Contents Preface 1 Introduction 1.6 Problems 4 Variational Principles 4.3 Separation of Variables 3.3.1.2 Linear Partial Differential Equations 3.1.5 Selected References 3.2 Integrals and Missing Variables 4.7 Selected References 2.1 Euler's Equation 4.3 Constraints and Lagrange Multipliers 5 9 9 12 13 17 18 19 21 22 24 24 26 27 28 29 31 32 33 33 34 34 34 36 37 .6 z-Transforms 2.3.1 Variational Calculus 4.3 Spherical Coordinates 3.2 Systems of Differential Equations and Normal Modes 2.

5 Problems 8 Finite Elements 8.1 Linear Congruential 5.2 Stochastic Processes 5.3 Selected References 8.4 Selected References 6.2 Runge-Kutta Methods 6.1 Optics: Fermat's Principle 4.3 Beyond Runge—Kutta 6.4 Problems 9 Cellular Automata and Lattice Gases 9.2.4 Selected References 7.5 Problems 38 38 38 39 40 43 43 44 44 46 48 50 51 55 56 57 59 60 60 Part Two: Numerical Models 6 Finite Differences: Ordinary Differential Equations 6.5 Problems 5 Random Systems 5.1 Hyperbolic Equations: Waves 7.2 Analytical Mechanics: Hamilton's Principle 4.4 Selected References 5.2.3 Random Number Generators 5.1.1.1 Weighted Residuals 8.1 Numerical Approximations 6.3.2 Variational Problems 4.2.1 Lattice Gases and Fluids 9.3 Symmetry: Noether's Theorem 4.2 Stochastic Differential Equations 5.2.2 Linear Feedback 5.4 Selected References 4.2 Characteristic Functions 5.2 Parabolic Equations: Diffusion 7.1 Joint Distributions 5.2.5 Problems 7 Finite Differences: Partial Differential Equations 7.3 Rigid Body Motion 4.1 Distribution Evolution Equations 5.2 Cellular Automata and Computing 63 67 67 70 72 76 76 78 79 81 84 91 91 93 93 99 100 101 102 103 107 .2 Rayleigh-Ritz Variational Methods 8.Contents 4.1 Random Variables 5.3 Elliptic Equations: Boundary Values 7.3.

6.8 Selected References 12.6 Selected References 10.7 Regularization 12. .1.7 Problems 11 T r a n s f o r m s 11.4 Genetic Algorithms 13.2 Splines 12.4 11.6 Neural Networks 12.4 Overrating 12. 125 127 127 128 128 11.1 Levenberg-Marquardt Method 10. .5 Estimation.3.1 Model Estimation 10.4.5 The Blessing of Dimensionality .5 Curse of Dimensionality 12.1 Multidimensional Search 13.1 Orthogonal Transforms 111 115 116 117 118 119 122 124 .2 11. Fisher Information.1 Singular Value Decomposition 10.2 Local Minima 13.1 Back Propagation 12.5 11.4 Nonlinear Least Squares 10.4 Problems 109 110 Part Three: Observational Models 10 Function Fitting 10.1.3 Simulated Annealing 13.1 Pade Approximants 12.9 Problems 13 Optimization and Search 13.6 Fourier Transforms Wavelets Principal Components Selected References Problems 129 131 136 138 138 139 139 139 141 142 145 147 148 150 152 153 155 155 156 157 161 162 164 166 12 Architectures 12.3 Linear Least Squares 10.2 Orthogonal Functions 12.3 11.3 Radial Basis Functions 12. and the Cramer-Rao Inequality 10.1 Polynomials 12.Contents 9.2 Least Squares 10.3 Selected References 9. .

2.4 Java A1.l.5 Selected References 14.1.Contents 13.4.7 Problems 16 Linear and Nonlinear Time Series 16.3 OpenGL Al.7 Problems 167 168 169 169 172 174 178 185 185 186 186 187 189 195 197 203 203 204 205 207 208 213 214 216 217 220 224 224 Appendix 1 Graphical and Mathematical Software Al.1 Dimensions 16.6 Problems 15 Filtering and State Estimation 15.3 Problems 225 226 226 228 230 230 234 240 244 249 Appendix 2 Network Programming A2. Sorting.2.1 Programming Environments Al.4 Characterization 16.4.2 X Windows Al.5 Forecasting 16.1 Linear Time Series 16.2.5 Hidden Markov Models 15.2 Fitting Densities 14.2. and All That 250 250 .2 Wiener Filters 15.1 Postscript Al.3 Mixture Density Estimation and Expectation-Maximization 14.6 Selected References 15.4 Cluster-Weighted Modeling 14.3 Kalman Filters 15.l Math Packages Al.1 Matched Filters 15.3 Entropies 16. and Trees 14.2 Graphics Tools Al.2 Interactive Environments A1.6 Selected References 16.1 OSI.2 The Breakdown of Linear Systems Theory 16.6 Selected References 13.1 Histogramming.7 Problems 14 Clustering and Density Estimation 14.3 State-Space Reconstruction 16.4. TCP/IP.4 Nonlinearity and Entrainment 15.2 Lyapunov Exponents 16.

5 Random Systems A4.1 Introduction A4.Contents A2.9 Cellular Automata and Lattice Gases A4.6 Finite Differences: Ordinary Differential Equations A4.13 Optimization and Search A4.8 Finite Elements A4.ll Transforms A4.12 Architectures A4.16 Linear and Nonlinear Time Series Bibliography Index .15 Filtering and State Estimation A4.2 Socket I / O A2.3 Partial Differential Equations A4.3 Parallel Programming 251 254 Appendix 3 Benchmarking 257 259 259 259 266 269 271 276 281 289 292 302 305 309 315 319 323 325 330 340 Appendix 4 Problem Solutions A4.7 Finite Differences: Partial Differential Equations A4.2 Ordinary Differential and Difference Equations A4.10 Function Fitting A4.4 Variational Principles A4.14 Clustering and Density Estimation A4.