¨ t Mu

¨nchen
Ludwigs-Maximilians-Universita
— Departement for Physics —
University Observatory

Computational Methods in Astrophysics
Monte Carlo Simulations
and
Radiative Transfer

first edition (2005)

Joachim Puls
winter semester 2005/2006

ii

Contents
0 Introduction

1

1 Theory
1.1 Some basic definitions and facts . . . . . . . . . . . . . . . . . . . . .
1.1.1 The concept of probability . . . . . . . . . . . . . . . . . . . .
1.1.2 Random variables und related functions . . . . . . . . . . . .
1.1.3 Continuous random variables . . . . . . . . . . . . . . . . . .
1.2 Some random remarks about random numbers . . . . . . . . . . . .
1.2.1 (Multiplicative) linear congruential RNGs . . . . . . . . . . .
1.2.2 The minimum-standard RNG . . . . . . . . . . . . . . . . . .
1.2.3 Tests of RNGs . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Monte Carlo integration . . . . . . . . . . . . . . . . . . . . . . . . .
1.3.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3.2 The method . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3.3 When shall we use the Monte Carlo integration? . . . . . . .
1.3.4 Complex boundaries: “hit or miss” . . . . . . . . . . . . . . .
1.3.5 Advanced reading: Variance reduction – Importance sampling
1.4 Monte Carlo simulations . . . . . . . . . . . . . . . . . . . . . . . . .
1.4.1 Basic philosophy and a first example . . . . . . . . . . . . . .
1.4.2 Variates with arbitrary distribution . . . . . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

1-1
1-1
1-1
1-1
1-2
1-6
1-6
1-8
1-10
1-18
1-18
1-19
1-22
1-23
1-24
1-26
1-26
1-29

2 Praxis: Monte Carlo simulations and radiative transfer
2-1
2.1 Exercise 1: Simple RNGs and graphical tests . . . . . . . . . . . . . . . . . . . . 2-1
2.2 Exercise 2: Monte Carlo integration - the planck-function . . . . . . . . . . . . 2-2
2.3 Exercise 3: Monte Carlo simulation - limb darkening in stellar atmospheres . . . 2-3

iii

CONTENTS

iv

1). 1 . 1. In the next chapter we will give an overview of the underlying theoretical concept.3. If you are a beginner in Monte Carlo techniques. particularly in high dimensions • differential equations • physical processes. 1. In Sect. The essential approach is to approximate theoretical expectation values by averages over suitable samples. At first we will summarize few definitions and facts regarding probabilistic approaches (Sect.Chapter 0 Introduction This part of our practical course in computational techniques deals with Monte Carlo methods. An inevitable prerequesite for this approch is the availability of a (reliable) random number generator (RNG). 1. parameter estimation) • optimization • integration. 1. Most interestingly. you might leave out those sections denoted by “advanced reading” (which consider certain topics which might be helpful for future applications). which allow to solve mathematical/scientific/economic problems using random numbers. The theoretical section is closed by an introduction into Monte Carlo simulations themselves (Sect. Plan. all results have to be analyzed and interpreted by statistical methods. The plan to carry out this part of our computational course is as follows: Prepare your work by carefully reading the theoretical part of this manual.4). Outline. The fluctuations of these averages induce statistical variances which have to be accounted for in the analysis of the results. Some typical areas of application comprise • statistics (tests of hypothesis. An important application of Monte Carlo techniques constitutes the Monte Carlo integration which is presented in Sect. Monte Carlo methods can be applied for both stochastic and deterministic problems. The outline of this manual is as follows.2 we will consider the “construction” of simple RNGs and provide some possibilities to test them. culminating in the solution of one (rather simple) astrophysical radiative transfer problem by means of a Monte Carlo simulation. In Chap 2 we will outline the specific exercises to be solved during this practical work. Moreover.

plots etc. in order to allow for an appropriate preparation. 2 . whereas the second day covers exercise 3.CHAPTER 0. exercise 1 and 2 should be finished. INTRODUCTION On the first day of the lab-work then. Don’t forget to include all programs. into your final elaboration. Please have a look into these exercises before you will accomplish them.

. . with a real number xi . e. E. any other association. P (Ek ) = pk . where the probability for the occurence of such an event.v. x6 = 1.g.2 Random variables und related functions We now associate each discrete event. Of course. These events shall be denoted by E k . x6 = 6. . is allowed as well. and these events are complete concerning all possible outcomes of the experiment. which can result in a number of different and discrete outcomes (events). If Fi and Gj are independent of each other.1 1. Ei . the event E6 that the result of throwing a dice is a “six” might be associated with the r.1) i=1 For a two-stage experiment with events Fi and Gj .1. called random variable (r. (1. then P (Ei and Ej ) = 0.2) 1.g. then pk = 0. i = 1.. rolling a dice. must satisfy the following constraints: (a) 0 ≤ pk ≤ 1. N . e. (c) If two events. (b) If Ek cannot be realized.1. Ei and Ej .1 Some basic definitions and facts The concept of probability Let’s assume that we are performing an experiment.. If Ek is the definite outcome. then pk = 1.) or variate .v. the probability for the combined event is given by p(Eij ) = pij = p(Gj ) · p(Fi ). 1-1 .g. (d) If there are N mutually exclusive events. P (Ei or Ej ) = pi + pj . (1.Chapter 1 Theory 1. Ei .. are mutually exclusive. then N X pi = 1. Eij is called the combined event (Gj has occured after Fi ). .

g(x). The probability that the result of the experiment will lie inside an interval [a. xi .9) a 1-2 . Zb P (a ≤ x ≤ b) = f (x0 )dx0 .4) i=1 Note that the expectation value is a linear operator. on the other hand.1. gh = E(gh) = E(g) · E(h) = g · h.8) where f (x) is the so-called probability density function (pdf). σ(ag) = aσ(g). the standard deviation has the following scaling property. i.6) of a r.3) E(x) = x = i=1 if the events Ei . . the variance is no linear operator (prove by yourself!).g. If g and h are statistically independent. Any arbitrary function of this random variable. .3 Continuous random variables So far. σ. the probability that the event lies inside a specific (infinitesimal) interval is well defined. describes the “mean quadratic deviation from the mean”. that an exactly specified scattering angle will be realized) is zero. (1. How do we have to proceed in those cases where different events can be no longer enumerated (example: scattering angles in a scattering experiment)? The probability for a distinct event (e. (1.e. 1. N are complete and mutually exclusive.. (1. . b] can be calculated from this pdf. one can show that Var (ag + bh) = a2 Var (g) + b2 Var (h) . (1. . (1. with associated discrete random variables. i = 1. The variance E(ag(x) + bh(x)) = aE(g) + bE(h).CHAPTER 1. (1.5) Var (x) = (x − x)2 = x2 − 2x · x + x2 = x2 − x2 . is also a random variable. and The standard deviation. P (x ≤ x0 ≤ x + dx) = f (x)dx. (1. and we can construct the corresponding expectation value. we have considered only discrete events. N X E(g(x)) = g = g(xi )pi .v. is defined as σ(x) = (Var (x)) 1/2 .. In contrast to the expectation value. THEORY The expectation value of a random variable (sometimes sloppyly denoted by “average” or “mean” value) is defined by N X x i pi . Ei .7) Thus..

on the other hand. x0 The probability. and we find 1 for all x ∈ [a.2) is given by f (x) = 1. 1. we have f (x) = 0. THEORY In analogy to the discrete probabilities. that any value inside x ≤ ≤ x + dx is realized. A uniform distribution between [a. b]. b]. 1.11) −∞ Consequently. ∞). this means that F (x) = x for x ∈ [0. 1. (1. it follows immediately that F (∞) = Z∞ f (x)dx = −∞ −∞ = Zb a Za | f (x)dx + Zb f (x)dx + a {z =0 } ! Z∞ f (x)dx b | {z =0 } f (x)dx = C · (b − a) = 1.2 display the situation graphically. Since for the remaining intervals.10b) −∞ the combined probability that all possible values of x will be realized is unity! The cumulative probability distribution (cdf). denotes the probability that all events up to a certain threshold.10a) (1. (1. b] is defined via a constant pdf: f (x) = C ∀ x ∈ [a. b = 1 (cf. pi . 0 P (x ≤ x) ≡ F (x) = Zx f (x0 )dx0 . −∞ < x < ∞ Z∞ f (x0 )dx0 = 1 : (b) (1. x. drawn by a random number generator (RNG) with a = 0. • F (−∞) = 0 and • F (∞) = 1. F (x). is identical for all x ∈ [a. Sect. Figs.CHAPTER 1.e. The corresponding cumulative probability distribution results in f (x) = F (x) = Zx a f (x)dx = Zx a F (x) = 0 for x < a F (x) = 1 for x > b dx x−a = b−a b−a ∀ x ∈ [a. a) and (b.13) Regarding RNGs. (1. will be realized. b]. b].12) b−a Thus.1 and 1. 1-3 . (−∞. also the pdf has to satisfy certain constraints: (a) f (x) ≥ 0. 1]. the pdf for uniformly distributed random numbers.1 Example. i. f (x)dx. • F (x) is a monotonic increasing function (because of f (x) ≥ 0)..

15b) −∞ 2 Var (g) ≡ σ (g) = 2 (g(x0 ) − g) f (x0 )dx0 −∞ 1-4 . of a uniform distribution within the interval [a. THEORY f (x) PSfrag replacements 1 b−a x 0 a b Figure 1.15a) Z∞ (1.1: Probability density function (pdf). In analogy to the discrete case.14b) −∞ 2 Var (x) ≡ σ = Z∞ (x0 − µ)2 f (x0 )dx0 (1. F (x).CHAPTER 1.2: Corresponding cumulative probability distribution.b]. F (x) 1 PSfrag replacements 0 x a b Figure 1.14a) −∞ Z∞ E(g) ≡ g = g(x0 )f (x0 )dx0 (1. expectation value and variance of a continous random variable are defined by Z∞ E(x) ≡ x ≡ µ = x0 f (x0 )dx0 (1. f (x).

or normal distribution).3: « „ 1 (x − µ)2 .6826. see Fig. Var (x) = σ 2 E(x) = µ and the so-called “1σ-deviation” is given by P (µ − σ ≤ x ≤ µ + σ) ≈ 0.2 Example (The Gauss. The pdf of a Gauss-distribution is defined as follows. 1. THEORY h σ PSfrag replacements µ Figure 1. 1.61 · f (µ)). 1-5 .3: The Gauss-distribution (h = f (µ)e−1/2 ≈ 0. f (x) = √ exp − 2σ 2 2πσ Expectation value and variance can be easily calculated.CHAPTER 1.

however.1 (Multiplicative) linear congruential RNGs For most “standard” tasks such as integrations and (astro-)physical simulations. THEORY 1. C. The first three of these requirements are a “must”. SEED. mostly based on encryption methods. • The maximum value possible for this algorithm is M − 1. a new SEED-value (“1st random number”) is calculated. we will concentrate on pseudo random numbers. which will be met by most of the RNGs discussed in this section. the best choice is to use so-called (multiplicative) linear congruential RNGs. e. M integer-numbers) x = SEED/M normalization. At first. • calculated by a fast algorithm.2. x ∈ [0.. 1. SEED = (A · SEED + C) mod M (SEED. i. Strictly speaking. Sub-random numbers are particularly used for Monte Carlo integrations.g. particularly by the “minimum standard” RNG (see below). we will briefly discuss how to construct simple random number generators (RNGs) and how to test them. which must/should satisfy the following requirements: • uniformly distributed • statistically independent • not or at least long-periodically (most algorithms produce random numbers with a certain period) • reproducable (tests of programs!) • portable: it should be possible to obtain identical results using different computers and different programming languages. which..e. which have an “optimal” uniform distribution. and • sub-random numbers. Since SEED is always (integer mod M ). there are three different classes of random numbers. whereas the latter ones are a “should”. obtained from radio-active decays (laborious. and so on. conversion to real-number (1. because of their speed.CHAPTER 1.2 Some random remarks about random numbers In this section. a 2nd random number is created. In the remainder of this section. From this value. A.16) • For any given initial value. namely • real random numbers. calculated from a deterministic algorithm. few) • pseudo random numbers. we have 0 ≤ SEED < M. are much slower. let us consider the requirements for useful random numbers. 1-6 . but are not completely independent (for an introduction. though other methods are availabe. 1). Basic algorithm. see Numerical Recipes).

the calculation of SEED requires the evaluation of intermediate results being larger than M − 1 (“overflows”) so that the algorithm would fail (e. (M − 1) should correspond to the maximum integer which can be represented on a specific machine (231 − 1 for 4-byte integers). . if SEED(actual) = SEED(initial) • Since this period should be as long as possible ⇒ M should roughly correspond to the maximum integer value which can be represented on the specific system. In the following we consider the cycle with n = 5. Because of (1. i. • n ≤ 31 for 4-byte integers (note: only positive/zero integers are used) • As an example. if SEED becomes of order M ). Since we are aiming at a rather long cycle.1: Random number sequence SEED = (5 · SEED + 1) mod 32 with initial value 9. which is actually the case. In the above cycle. A way out of this dilemma was shown by Schrage (1979). etc. the maximum possible cycle (→ 32 different numbers) should be produced by using these values. THEORY • By construction.e. . the algorithm implies a periodic behaviour: a new cycle starts. M . 1. M − 1.CHAPTER 1. 46 mod 32. A and M are satisfied. A = 5 and C = 1 satisfy the required conditions if n ≥ 2 (other combinations are possible as well).. however. 1. (1. 0 .3 Example (Algorithms with C > 0 and M = 2n ).g. ↓ 9 14 7 4 21 10 19 0 1 6 31 28 13 2 11 24 25 30 23 20 5 26 3 16 17 22 15 12 29 18 27 8 9 14 7 · · · (← new cycle) Table 1. He devised a method by which multiplications of type (A · SEED) mod M. • According to the above statement. this problem would already occur for the first number calculated. At least for linear congruential RNGs with C > 0 it can be proven that the maximum possible period. which. however. In this case. The problem of portability.1 SEED (start) = 9 SEED (1) = 5 · 9 + 1 mod 32 = 46 mod 32 = 14.17) 1-7 .. is not always the case. A possible solution would comprise an assembly language implementation of our algorithm. with 8-byte product registers. the algorithm would be no longer portable (implementation is machine dependent).16). M = 32. • An optimum algorithm would produce all possible integers. as shown in Tab. can be actually created if certain (simple) conditions relating C.

1 With this value of M . can be calculated without overflow. but most of these cylces inhibit certain problems.g. one obtains roughly 2. on the one hand. (A · SEED) mod M At first.2. SEED and M consisting of a certain length (e. 1).18) where square brackets denotes an integer-operation. uniform distribution and almost no correlation problem.g. In this way then. again only certain values of A are “allowed” to maintain the required properties of the RNG. In dependence of M . This “trick”.  5  16807 = 7 A = 48271   69621. A (1. is also a prime number. most simple RNG for the case C ≡ 0. and x ∈ (0. e. long cycles are possible (though being at least a factor of 4 shorter than above). 4 byte). is the largest representable 4-byte integer. when A is given by one of the following numbers. this number is called a Mersenne prime number. For M = 2n . In such a case.CHAPTER 1. 1.. Let M = and thus 231 − 1 and A = 16807. called “minimum-standard” generator.15 · 109 different random numbers ∈ (0. THEORY with A. The corresponding sequence has a long period. SEED = 0 is prohibited. This lucky situation is not a pure coincidence. for C = 0 we strongly advise to avoid M = 2n and to use alternative cycles with M being a prime number. Then » – M q = = 127773 A r = M mod A = 2836 M = 16807 · 127773 + 2836 = 231 − 1. a maximum period of M − 1 (no “zero”) can be created if A is chosen in a specific. then in many but not all cases 2p − 1 is also prime. 1.. and. however. 1).4 Example. All three values comply to both the complex requirement for allowing the maximum period and one additional condition which arises if Schrage’s “trick” shall be applied. as discussed below. particularly concerning the independence of the created numbers. An impressive example of such problems is given in example 1. In particular. Thus.11. on the other.2 The minimum-standard RNG Based on these considerations. since the above number is a so-called Mersenne prime number. 1-8 . M must be decomposed in the following way: M =A·q+r · ¸ M with q = . M = 231 − 1 = 2147483647. Parkand Miller (1988) suggested to use the following. which. rather complex way. In this case then. As a first consequence. implies that our algorithm has to get along with C ≡ 0. r = M mod A. 1 If p is prime. Advanced reading: Schrage’s multiplication.

1.19) it is also ensured that |DIFF| < M − 1 can be calculated without overflow as well. SEED = 1147483647.19) can be calculated without overflow! We are now looking for SEEDnew = (A · SEED) mod M.6 Example (simple). 1-9 – . DIFF > 0 DIFF < 0. the following theorem can be proven: 1. Direct calculation yields SEEDnew = 79888958987787 mod M = 419835740.20) Because of (1. DIFF + M.21) 1. without overflow DIFF = A(SEED mod M) − r » SEED q = 1309014042 −889178302 {z } | <M −1 ! = 419835740 = SEEDnew . THEORY With these definitions of q and r. A = 69621. Let DIFF := A · (SEED mod q) − r · · ¸ SEED . |{z} |{z} <M−1 <M−1 1. (1. (1. {z } | >M(∗) where (∗) cannot be represented with 4 bytes. and with (1. The requirements are satisfied.20.21) we find (A · SEED) mod M = A · (SEED mod q) − r » SEED q – = 3 · 2 − 1 · 1 = 5. Using Schrage’s trick.5 Theorem. (ii) The random number SEEDnew finally results via SEED new = ( if if DIFF. we obtain.7 Example (typical).CHAPTER 1. If r < q (this is the additional condition) and 0 < SEED < M − 1. Calculation of (3 · 5) mod 10 without overflow corresponding to an intermediate value > (M − 1) = 9 (A = 3 and SEED = 5): ff ˆ ˜ =3 q = 10 3 r = 10 mod 3 = 1 thus r < q and SEED < M − 1 = 9. q (1. then (i) A · (SEED mod h q)i r · SEED q ) < M − 1. M = 231 − 1 = 2147483647.

16) suffer from a certain deficiency. In particular. Let’s roll the dice and define N1 number of events with outcome 1. 1). . a number of tests should be performed. 1. and a graphical test. Let Ni . . namely the calculation of Pearson’s χ2p (which asymptotically corresponds to the “usual” χ2 .. N2 number of events with outcome 5 and 6. N = N 1 + N2 .1.K * q) . 1.2. one has to confirm that the random numbers provided by the generator are uniformly distributed and statistically independent from each other. more general problem. Sect. . A = 69621.1 Advanced reading: Test of uniform distribution At first. From then on. with 0 < ISEED < M − 1. We like to test the probability that a particular value of Ni corresponds to a given number ni . k with k X Ni =: N i=1 the number of outcomes regarding a certain event i during an “experiment”. q = 30845. p(x)dx = dx.8 Example (Test of dices). We will see that all RNGs based on the linear congruential algorithm (1.3 Tests of RNGs In order to check for the reliability of any unknown RNG. . only four statements are required. Two rather simple possibilities will be described in the following. the variable ISEED has to be initalized. . Parameter:: M1 = 1. i = 1.3. the rest are declarations! Before the first call of RAN. To calculate a random number with this generator. i. r = 23902 Real. 2 corresponding to a globale constant in C++ 1-10 . . K K = ISEED / q ISEED = A * (ISEED . THEORY Algorithm tion: for the Park-Miller minimum-standard generator and Schrage’s multiplica- RAN(ISEED) Real RAN Integer (4 Byte).CHAPTER 1. one has to check whether the created random numbers are uniformly distributed in (0./Float(M) Integer (4 Byte):: ISEED.r * K IF (ISEED < 0) ISEED = ISEED + M RAN = ISEED * M1 END Note that (ISEED . At first let us consider the following. Parameter2 :: M = 2147483647.e. Remember that a uniform distribution implies a constant pdf within this interval. 1.K * q) corresponds to the operation SEED mod q. which can be cured in a simple and fast way by the so-called “reshuffling” method. . cf. 4. ISEED must not be changed except by RAN itself! 1.3.2..

The quantity Q gives the probability that by chance a larger value of χ2 (larger than actually found) could have been present. ni . n1 = the dice has not been manipulated. Let’s test now four different dices. THEORY We will test whether N1 and N2 are distributed according to the expectation. After 15 throws each we obtained the results given in Tab. 6. chap. • if m additional constraints are present. = 2 1 ni N N 3 3 Because of the requirement that one has to expect at least five events in each channel.8 1. 1. √ see Numerical Recipes.27 Table 1.24 ·10−7 10−3 0. within 0. Method. throw the dice once more.22) is a statistical function. Because of this theorem.3 10. Note that for an unmanipulated dice. Too good (χ2 = 0)) means that the result is “exactly” as one expects. If Q has a reasonable value (typically. 0. i = 1. 1. “asymptotically” means that in each “channel” i at least ni = 5 events are to be expected. d4ce 4: The result is OK. P P • if ni is normalized via ni = Ni = N . with expectation value 0 and variance 1. 2 as given above. we have f = k − m − 1. 3 i. 1. however still possible. < χ 2 >≈ 1 ± 2 is to be expected. such a result is only obtained if the experiment has been faked. With Ni .2. whereas a low value indicates that the actual χ2 is too large for the given degrees of freedom (dice faked!). which asymptotically (i. 1-11 . we have f = k − 1 ( one constraint). everything should be OK. In praxis.95). we have f = k. well-known from optimization/regression methods. whether To perform such a test.2.2 Q(χ2p . Pearson’s χ2p (1.22) For this quantity.e. are defined by • if ni is fixed. dice 1 2 3 4 N1 10 1 4 8 N2 5 14 11 7 χ2p 0 24.05. this proof is rather complex). Throw the dice once more. Pearson’s χ2p results in χ2p = X (Ni − ni )2 (N2 − 31 N )2 (N1 − 32 N )2 + . the following theorem can be proven (actually. The degrees of freedom. Usually. Remember that the expectation value of χ 2 p 2 is f and that the standard deviation of χ is (2f ).2: Test of four different dices The results of our “experiment” can be interpreted as follows: d1ce 1: The outcome is “too good”.e. .. d2ce 2: The outcome is very unprobable.10 Example (Continuation: Test of dices). f . Pearson’s χ2p has to be calculated χ2p = k X (Ni − ni )2 ni i=1 . 3 n2 = 1 N. (1.9 Theorem. f = 1) 1 8. 2 N.. Thus. . and the dice seems to be ummanipulated.CHAPTER 1. d3ce 3: The outcome is unprobable. 10 5 and the degrees of freedom are f = 2 − 1 = 1. we have to throw the dice at least N = 15 times. and the dice can be suspected to be manipulated. which are normally distributed. for Ni À 1 ∀ i) corresponds to the sum of squares of f independent random variables. we have χ2p = (N2 − 5)2 (N1 − 10)2 + . Concerning the calculation of Q. Pearson’s χ2p asymptotically corresponds to the “usual” χ2 -distribution.

We calculate N random numbers and distribute them according to their values into the corresponding channels.g. An alternative formulation of Pearson’s χ2p χ2p = k X (Ni − ni )2 i=1 where ni is given by = k X (Ni − pi N )2 i=1 pi N . chap. of course. at least with respect to the defined channels. To answer this question. Remember the original question whether the provided random numbers are uniformly distributed within (0.e. the experiment should be performed a couple of times (say.CHAPTER 1. X that pi = 1. 14. i.3 1-12 . • At last. 3 e. k − 1) has to be calculated. (1. a minimum of N = 5k random numbers have to be drawn. that the initial SEED value has to be different for different l. If the 1 random numbers were uniformly distributed. N/k i=1 such that in each channel the expected value of ni = N/k is at least 5. 3 • If the indivual Qj values are “reasonable” and the Kolmogorroff-Smirnov-test has yielded a “reasonable” probability as well. Then. .e. Note. This can be obtained by the so-called Kolmogoroff-Smirnov-test. i Application to random numbers. l) within the series are distributed according to χ2 with k − 1 degrees of freedom. at least 10 times). i. . we should have pi = for all i. 1) into k equally wide sub-intervals. .j (j = 1. and • the test has to be repeated l times. we “bin” the interval (0. • Ni the number of events in channel i with i • pi the probability for the event in channel i. we test whether the individual χ2p. To obtain a larger significance regarding our findings. we calculate k X (Ni − Nk )2 χ2p = . the RNG provides indeed uniformly distributed random numbers. if the dices were unmanipulated (see below). we count the occurence Ni in each channel. with typically l ≈ 10. k Thus. . The channels must be selected in such a way that all possible events can be accounted for. Numerical Recipes. • the probability Q(χ2p . The specific results have then to be checked versus the expectation that the individual χ2 must follow the χ2 -distribution for f degrees of freedom. THEORY Comment...23) • k is the number of “channels” (possible events) X Ni = N . 1). After the required number of random numbers has been drawn (see below).

Though we do know now (hopefully!) that the number are uniformly distributed. Fig. we bin the unit square (0.. a) Two-dimensional tests. 1-13 . x2 = 0. and p = 1/k 2 random numbers in each “quadratic” channel have to be expected (independent events. and both the individual values of χ2p. For this purpose. THEORY x3 . independent of xi . we are not yet finished. x 2 PSfrag replacements       1 0 Figure 1. The corresponding Pearson’s χ2p is calculated via 2 χ2p = k X (Ni − N/k 2 )2 i=1 N/k 2 . ! P (xi+1 |xi ) = P (xi+1 ) ∀i. (1.5.3. As a minimum. To test this independence. x3 = 0.4 for the random numbers x1 = 0. N = 5k 2 numbers have to be drawn now.e. cf. more-dimensional tests have to be applied. Again. more than one series j has to be checked. we still have to check whether they are statistically independent.9). create pairs of random numbers and sort them into corresponding “quadratic” channels (cf.j as well as their distribution (via Kolmogoroff-Smirnov) have to be inspected. which check for the correlation of random numbers.2)).1.4: 2-D test (see text).CHAPTER 1.3.2 Advanced reading: Test of statistical independence Having accomplished this test. 1. x4 = 0. if Ni is the number of drawn pairs in channel i.2. For k channels in one dimension one obtains k 2 channels in two dimensions. 1)2 in two dimensions. 1. i. ! P (xi+1 |xi ) = P (xi+1 ) checks whether two consecutive numbers are statistically independent. x 4       1                 x1 . In other words: The probability that after a number xi the number xi+1 will be drawn must be equal to the probability for xi+1 itself.

0000000E+00 0. 000.8 454074.7373354 0.66 26666. RANDU (by IBM!).2 452882.8 454412. ! P (xi+2 |xi+1 . xi ) = P (xi+2 ) checks for the independence of a third random number from the previously drawn first and second one. we carry out such a 3-D test with 303 = 27 000 channels and 270 000 random numbers. 1. as it is also evident from the individual values of χ2p alone. up to 8 dimensions should be checked for a rigorous test of a given RNG.0000000E+00 0.0000000E+00 (in single precision) (note that each individual value of χ2p correspond to ≈ 17 · f ) and the KS-test yields KS-TEST: prob = 5.77 26667. Triples of three consecutive random numbers are sorted into this cube.7 455033.9247663 0. RANDU.0000000E+00 0.1 452412.0 453038.0000000E+00 0. The further proceeding is as above.0000000E+00 0.2668213 0.and 2-D are passed without any problem.0000000E+00 0. (What problem might arise?) Check the web for published results.0000000E+00 0.0000000E+00 0. Park/Miller Minimum-Standard ---------------------------chi2 chi2 chi2 chi2 chi2 chi2 chi2 chi2 chi2 chi2 = = = = = = = = = = 26851.2786968 The probability that the values for χ2p do follow the χ2 -distribution is given (via Kolmogoroff-Smirnov) by KS-TEST: prob = 0. i.98 27006.7 456254.82 27135. an algorithm which has been discussed as being potentially “dangerous” (remember that the minimum-standard generator uses a prime number for M ). The corresponding results are given by RANDU ----chi2 chi2 chi2 chi2 chi2 chi2 chi2 chi2 chi2 chi2 = = = = = = = = = = 452505. we consider the unit cube (0.2070863 0. A = 65539 and C = 0.24 Q Q Q Q Q Q Q Q Q Q = = = = = = = = = = 0.2369688 0. being actually of the order of ≈ 27.0000000E+00 0. we conclude that the minimum-standard generator passes the 3-D test.29 27187. We have calculated 10 test series each for the Park/Miller minimum-standard generator and for RANDU. the situation is different from the 3-D case on (it seems that IBM did not consider it as necessary to perform this test). N/(303 ) = 10 events per channel are to be expected. c) Typically.546628E-10 (!!!) 1-14 .CHAPTER 1.3661267 0. and the corresponding probability is given by pi = 1/k 3 . Thus..4 453992. In the following. One of the first RNGs which have been implemented on a computer. and a value of χ2 of the order of f = (27 000 − 1) should result for an appropriate RNG. Thus.7192582 0. THEORY b) Three-dimensional tests. Though the above tests for 1.2 453478.17) with M = 231 .20 26865.e.9249226 0. 1)3 .2 Q Q Q Q Q Q Q Q Q Q = = = = = = = = = = 0. Below we compare the outcome of our test.38 27079. In analogy to a) and b).11 Example (RANDU: A famous failure).7328884.4844202 0.37 27142. In contrast. used the algoritm (1.37 27167.

they do not fill this space uniformely. THEORY Figure 1. these numbers are not statistically independent and therfore no pseude-random numbers! 1. this number becomes reduced by a factor of 41/3 . With respect to a 2-D representation. i. 1. For RANDU (see example 1. a deficiency becomes obvious which affects all random numbers craeated by linar congruential algorithms: If one plots k consecutive random numbers in a kdimensional space. at least 31 of those hyper-surfaces are realized. It can be shown that the maximum number of these hyper-surfaces is given by M 1/k . the probability is VERY low that the individual χ2p follow a χ2 distribution with f ≈ 27. 000. With our choice of A = 65 and C = 1. where only a careful choice of A and C ensures that this maximum is actually realized. this generator delivers only 15! surfaces (see Fig. thus reducing the maximum number of surfaces to 812.e. If M = 231 . a 3-D representation should yield a maximum number of (231 )1/3 ≈ 1290 surfaces. The numbers are located on 31 hyper-surfaces. because only even numbers and additionally only half of them can be created. Though RANDU delivers numbers which are uniformely distributed (not shown here).3. In Fig.3 Graphical representation One additional/alternative test is given by the graphical representation of the drawn random numbers. 1. Consequently. In contrast however. i.5: 2-D representation of random numbers.2. 1-15 .. which again shows the failure of RANDU. each third number is strongly correlated with the two previous ones. the available space is by no means uniformly filled..CHAPTER 1. the maximum number of hyper-surfaces (here: straight lines) is given by 20481/2 ≈ 45. From such tests.11). created via SEED = (65 ∗ SEED + 1) mod 2048.6).e. but are located on (k − 1)-dimensional hyper-surfaces.5 we display this deficiency by means of a (short-period) generator with algorithm SEED = (65 ∗ SEED + 1) mod 2048.

which must be filled during a first “warm-up” phase (actually. .4 0. if N registers are available) (2) OUTPUT = V(I) (deliver random number) Y = OUTPUT (prepare for next random number) (3) V(I) = RAN (fill register with new random number) The RNG recommended by Numerical recipes.8 the result.5 0. Let y be a random number just drawn.2 0.6 0. Reshuffling To destroy the correlation described above (which will be always present for linear congruential RNGs). THEORY ’out’ 1 0.9 1 1 0.6 0.3 0. A new random number follows from (1) I = 1 + INTEGER(y · N) (calculate register index. 1-16 . 1.CHAPTER 1.4 0. .4 0.7 0. 1.1 0. N). The algorithm can be formulated as follows.3 0. the very first 8 numbers are not used at all).6 0. In this method.2 0. only 15 of them are realized.8 0.9 0.7 0.3 0.5 0.5 0. uses N = 32 registers V(1.8 0.8 0.2 0. “ran1”. the drawn random number itself is used to destroy the inherent sequence of consecutive numbers and thus the correlation. before the first number can be drawn. Though 812 hyper-surfaces are possible.1 0 0 0.7 displays the algorithm and Fig.6: 3-D representation of “random” numbers created by RANDU.9 0. Fig. called reshuffling. a fairly cheap and easy method is available.1 0 Figure 1.7 0.

1-17 . THEORY y (1) V(1) PSfrag replacements (3) RAN (2) V(I) OUTPUT V(N) N registers (first filling during “warm up”) Figure 1. Any previous correlation has been destroyed.5.CHAPTER 1.7: Reshuffling algorithm Figure 1. 1. but with reshuffling.8: As Fig.

Note that the variance of G is a factor of N lower than the variance of g. 1. . 1.25) states that the expectation value of the arithmetic mean of N random variables g(x) is nothing else than the expectation value of g itself.g. we calculate a new random variable (remember that functions of random variables are random variables themselves) by means of G= N X 1 g(xn ). Impatient readers might immediately switch to Sect.24) is also the arithmetical mean with respect to the drawn random variables. Simpson etc. 1.2). we will present the corresponding procedure in Sect. For arbitrary pdfs.3. is the expectation value of g itself. • This will be the basis of the Monte Carlo integration. Such an integration can replace more common methods based on well-known integration formulae (Trapez. • Let us begin by drawing N variates (= random variables) x1 . as a first and important application of Monte Carlo methods.24) where g is an arbitrary function of x.1 Preparation Before we can start with the “real” stuff.2.26 can be interpreted similarly: The variance of the arithmetical mean of N (independent!) random variables g(x) is lower than the variance of g(x) itself.. 2 N N N n=1 n=1 where we have assumed that the variates are statistically independent (cf.3.3. E(G) = G = N n=1 (1. the expectation value of G is given by N X 1 E(g(xn )) = g. we will discuss the Monte Carlo integration in this section. . . • From these variates. 1. we need some preparatory work in order to understand the basic ideas.e. 1. from the uniform distribution of pseudo random numbers as created by a RNG. xN from a given pdf f (x).3. independent of N . 1-18 . G (1.26) Var (G) = Var Var (g(xn )) = Var (g) . by a factor of 1/N . 1.3. 1. N n=1 (1. which justifies our effort concerning the corresponding tests of RNGs (Sect. g(xn ). as outlined in Sect.2. e.3 Monte Carlo integration After having discussed the generation of random numbers. In so far. The variance of G.CHAPTER 1.7). • Besides being a random variable. on the other hand.) and should be used under certain conditions. THEORY 1. Note that statistically independent variates are essential for this statement.4. (1.25) i. . Since the expectation value is a linear operator. which can be defined as follows: Approximate the expectation value of a function by the arithmetic mean over the corresponding sample!!! g=G≈G • Eq. is given by ! ÃN N X 1 X 1 1 g(xn ) = (1.

i. . . n = 1. THEORY Implication: The larger the sample. G= N 1 X g(xn ). (Remember that Var (G) is the mean square deviation of G from G) Summary. Before applying this knowledge within the Monte Carlo integration.2 The method The reader might question now in how far these statements concerning expectation values and variances have to do with real integration problems.. G → G = g for N → ∞. the smaller the variance of the arithmetical mean.CHAPTER 1. N according to f (x) and calculate the arithmetical mean.3. This will become clear in a second. ¾ ½ pdf f (x) E(g) true expectation value random variable g(x) Var (g) true variance Z E(g) = g = g(x)f (x)dx Z 2 Var (g) = (g(x) − g) f (x)dx Draw N variates. Note in particular that G≈G± p Var (G). we like to summarize the derived relations. and the better the approximation g ≈ G.e. N n=1 Then we have G = E(G) = E(g) = g 1 Var (G) = Var (g) N 1 1/2 σ(G) = (Var (G)) = √ σ(g) N √ The last identity is known as the “1/ N -law” of the Monte Carlo simulation. .27) . xn . . The actual “trick” now is the Monte Carlo approximation: Approximate g = G by G 1. by a (clever) re-formulation of the integral I to be evaluated: I= Zb a g(x)dx = Zb g(x) f (x)dx = f (x) Zb a a 1-19 h(x)f (x)dx (1.

see below).30) g − g 2 ≈ (J − G2 ). (1.CHAPTER 1.. N n=1 Finally. . The “new” fuction h(x) is thus given by g(x) = (b − a)g(x). calculate g(xn ) and evaluate the corresponding arithmetical mean. the integral I can be easily estimated by means of the Monte Carlo approximation.e. (1.29) Thus. and we just have learnt how to approximate expectation values! Until further notification. In a second step then. we will use a constant pdf. with the standard deviation resulting from Var (G)4 (which can be proven from the “central limit theorem” of statistics). when the variates xn are uniformly drawn from the interval [a. cf. b]. If we identify now the error of the estimated integral. the established Monte Carlo integration is the (arithmetical) mean with respect to N “observations” of the integrand. it is obvious √ that this error follows the “1/ N -law”. multiplied by the width of the interval.28) We like to stress that this relation is still exact. i.. √ δI ∝ 1/ N . ≈ (b − a)G. any integral can be re-interpreted as the expectation value of h(x) with respect to f (x). Var (G) = Var (g) = N N k N k J if J= G2 N 1 X 2 g (xn ) N n=1 2 is the Monte Carlo estimator for E(J) = J = g 2 and if we approximate g 2 = G by G2 . exact I = (b − a)g = (b − a)G Monte Carlo approx. consider a uniform distribution over the interval [a.e.12). (1. . G= N 1 X g(xn ). b]. f (x) h(x) = and the integral becomes proportional to the expectation value of g: I = (b − a) Zb a exact! g(x)f (x)dx = (b − a)g. THEORY The actual trick is now to demand that f (x) shall be a probability density function (pdf). ´ 1³ 2 1 1 (1. i. The error introduced by the Monte Carlo approximation can be easily estimated as well. N according to the pdf (which is almost nothing else than drawing N random numbers. δI.   1 a≤x≤b f (x) = b − a 0 else. we draw N variates xn . In this way then. 4 multiplied by (b − a) 1-20 . . n = 1. .

5865013 1. a) N = 5 and b) N = 25. 1. Assume that the following variates have been provided. Eq.8341565 0.0119059 (2) We calculate 1 X −Xn e N ˙ ¸ 1 X “ −X ”2 J = g2 = e n . we will estimate the integral Z2 e−x dx = 1 − e−2 = 0. 2] and “integrate” for the two cases.6981865E-02 1.5346164 1. we draw N random numbers uniformly distributed ∈ [0.5391896 0.4704462 0.8350208 1.86467 0 via Monte Carlow integration. we have to multiply them by a factor of 2.1895919 0.12 Example.5560532 0.6684224 0.3276724 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Case b) 0. (1) At first.7886472 0.1748597 0.3096535 0. THEORY By generalizing the integration element dx to an arbitrary-dimensional volume element dV .3469290 1.2970020 0.31) N V if 1 X g(xn ) N n ­ 2® 1 X 2 g =J = g (xn ) N n hgi = G = and V is the volume of the integration region considered. 1). we find the following cooking recipe for the Monte Carlo Integration: Ã ! r ³ Z ´ 1 2 2 hg i − hgi gdV ≈ V hgi ± (1. Since the RNG delivers numbers distributed ∈ (0.1599936 1.3153778 1.7929640 1.7114938 7.9196489 0. N G = hgi = 1-21 . 1.3457211 0. 1.8263693E-05 1.8897665 1. [Monte Carlo integration of the exponential function] As a simple example.41. Sect. 1 2 3 4 5 Case a) 7.1941637 0.CHAPTER 1.8677271 1. cf.3043649 0.4.2692878 0.4616689 1.

Stroud. we have to draw 250 000 random numbers. the sample size has to be increased by a factor of 1002 .9202501 ± 0. (1. Before concluding that the MC integration is useless. √ √ Note that the “1/ N -law” predicts an error reduction by 1/ 5 = 0.. As a preliminary conclusion then. whereas the MC integration always scales with ∼ N −1/2 . 1-22 . e.091613322. the integral is approximated by Case a) I ≈ 0..4904828.2453225. 1971.9676 · 10−3 .3 When shall we use the Monte Carlo integration? Because of this large number of evaluations (and the corresponding computational time.3.CHAPTER 1. Only from d = 5 (Trapez) or d = 9 (Simpson) on. not completely met by our simulation.8644224 ± 0.H.001 (factor 100). in contrast to errors resulting from standard quadrature methods. the MC error decreases faster than the standard quadrature error. and a comparison of exact (I(exact) = 0.2645783 Case b) I ≈ 0. The error for N = 25 is of the order of ±0..86476) and Monte Carlo result shows that it is also of the correct order. the error of the standard quadrature (with equidistant step size) increases from ∼ N −2(4) 2 4 for 1-D over ∼ N −1(2) for 2-D to ∼ N − 3 ( 3 ) for 3-D integrals. thus I = 0. The result of such a simulation ˙ ¸ (note that the RNG has to be fast) for N = 250 000 is given by G = hgi = 0. Thus.4601251.g.1. however. even if the RNG itself is fast). 1. to reduce the error to ±0.e. At first let us remind that the “1/ N -law” of the MC integration error is independent of the dimensionality of the problem.5 In the latter case (and assuming simple boundaries. the result given for case b) represents “reality” only within a 2-sigma error. J = g 2 = 0. one might question whether and when the Monte Carlo integration will be advantageous to standard quadrature √ methods such as exploiting the Simpson. Approximate Calculation of Multiple Integrals. Morever. Thus.447.2930297 (3) With I≈V hgi ± r ! ” 1 “ 2 hg i − hgi2 N and “volume” V = 2. one should consider the following arguments: 5 though a rigorous error analyis for more-D integration is very complex. J = 0. i.32) for N equidistant sampling points and dimensionality d. we might state that the MC integration can be quickly programmed but that a large number of evalutions of the integrand are necessary to obtain a reasonable accuracy. A. since case a) was estimated based on a very small sample.2992171 b) G = 0.9809657 ± 0. Prentice-Hall. We see that the (estimated) error has decreased as predicted.or Trapezrule.4322112. the integration errors scales via δI δI ∝ ∝ 2 N−d 4 N−d Trapez Simpson. simple integration formulae and equidistant step-sizes). J = 0. THEORY with the following results a) G = 0.

1. the integral can be easily approximated from Z n I = f (x)dx ≈ A · . • If the required precision is low. and the MC integration should be preferred. again the MC solution might be advantegous. since the longer computational time is compensated by the greatly reduced programming effort. (1.33a) If n hits have been counted in total. because of its simplicity. At least the principal procedure shallR be highlighted here. 1.4 Complex boundaries: “hit or miss” To evaluate more-D integrals with complex boundaries. an alternative version of the MC integration is usually employed. THEORY area A with N points PSfrag replacements Z f (x)dx Figure 1. • N points with co-ordinates (xi .9.33b) N 1-23 . To evaluate the integral I = f (x)dx.CHAPTER 1. by means of random numbers. yi ) are uniformely distributed across the rectangle with area A. knows as “hit or miss”-method.9: “Hit or miss”-method for the evaluation of 1-D integrals. • Just for more-D integrals complex boundaries are the rule (difficult to program). for the 1-D case and by means of Fig. • All those points are counted as “hits” that have a y-co-ordinate with yi ≤ f (xi ) (1. as long as the integrand is not locally concentrated. • If the specific integral must be calculated only once. the MC integration should be preferred as well. and particularly for first estimates of unknown integrals.3.

• The generalization of this method is fairly obvious.35) where < g/p > is the arithmetic mean of g(xn )/p(xn ). (1. for given radius n . The former approach relies on dividing the integration volume into sub-volumes. chosen in such a way as to decrease the variance. calculating the MC estimates for each individual sub-volume and adding up the results finally. THEORY • A disadvantage of this method is of course that all (N − n) points being no “hit” (i. which are located above f (x)) are somewhat wasted (note that f (x) has to be evaluated for each point). we will now use a different pdf p.. i.e. From this equation.8. For this reason. for typical examples see “Numerical recipes”. being the consequence of a rather slow decrease in error (∝ N −1/2 ) and giving rise to relatively large computational times. if g ≈ constant p over the integration region. r. It is this factor which can be reduced by importance sampling. 7.. the (original) variance can be diminished signficantly if the pdf p is chosen in such a way that it is very close to g. More information can be obtained from Numerical Recipes. can be somewhat decreased by different methods. (1.34) For arbitrary pdf’s p then.e.g. one proceeds as above. but depends also on the difference (< g 2 > − < g >2 ). the generalization of (1. Remember that the integration error does not only contain the scaling factor N −1/2 . Since any pdf has to be normalized.31) is straightforward. I = gdV = p Instead of using a uniform distribution. however now counting those points as hits on a square A which satisfy the relation x2i + yi2 ≤ r2 . the individual variances can add up to a total variance which is smaller than the variance obtained for an integration of the total volume. i. The basic idea is very simple.. If the sub-volumes are chosen in a proper way. E. if g is the function to be integrated and angle brackets denote arithmetic means. the circumscribing area A has to be chosen as small as possible. Remember that the integral I had been re-interpreted as an expectation value regarding a specified pdf p. sometimes considerably. The most important ones make use of the so-called stratified sampling and importance sampling.3. and the Monte Carlo estimator of the integral is given by g 1 I =h i± √ p N µ g g h( )2 i − h i2 p p ¶ 12 . Chap. to calculate the area of a circle.e. a constant pdf. Z Z g pdV. the area of the circle results from the fraction A N 1..CHAPTER 1. Again. one always has to consider the constraint Z pdV = 1. we will have a closer look into the importance sampling approach.5 Advanced reading: Variance reduction – Importance sampling The large number of required sampling points. respectively (mixed strategies are possible as well). In the following. and the variates xn have been drawn according to the pdf p! Note in particular that any volume factor has vanished. 1-24 .

and we would have gained nothing.34). for p = g. the variance would become zero. however. under the restriction that p is constant. . THEORY If p = g. Because of the normalization (1. Eq. Finally. we immediately find in this case that p= 1 .35. 1.d. Instead of using p = g then. which. N h 1-25 q.e. is just the integral we actually like to solve. and one might think that this solves the problem immediately. As we will see in the next section. we will show that our MC cooking recipe (1.CHAPTER 1. the best compromise is to use a pdf p which is fairly similar to g.31) is just a special case for the above generalization. the calculation of variates according to R a pdf p requires the calculation of pdV . V and I= Z gdV ≈ = µ ¶1 g 2 1 g 2 2 g h( i± √ ) i−h i 1/V 1/V 1/V N ¶ µ 1 1 V < g > ± √ (< g 2 > − < g >2 ) 2 . but also allows for a quick an easy calculation of the corresponding variates.

and depends on the particular cross-section(s) of the involved interactions(s).in a different perspective . One can show that this problem can be described by Milne’s integral equation. one has to perform a completely new simulation if only one of the parameters might change. 1. t and τ the “optical depth”. in a (plane-parallel) atmospheric layer. in the outer regions of (very) hot stars. and the result (plus noise) is obtained from collecting the output of the individual runs. This problem is met. THEORY 1. particularly since the physics of the subprocesses (i.e. if photons are scattered by free electrons only. and combined in a proper way as to describe the complete problem. advantages: Usually. such a method can be programmed in a fast and easy way. the solution of the equation of radiative transfer (RT) is non-trivial. The optical depth is the relevant depth scale for problems involving photon transfer. and in more complex situations (more-D. 1. (1. We like to calculate the spatial radiation energy density distribution E(τ ). however. perpendicular to the atmosphere). Monte Carlo simulations are employed to obtain corresponding results. and results of complicated problems can be obtained on relatively short time-scales.4. which . in dependence of the provided pdf’s (which are calculated from random numbers). Moreover. As an example we consider the following (seemingly trivial) problem. the interaction between photons and matter) is well understood. nonhomogenous.4 Monte Carlo simulations We are now (almost) prepared to perform Monte Carlo simulations (in the literal sense of the word). for a given 1-26 .) problem. disadvantages: On the other hand. E1 (x) = Z∞ 1 e−x·t dt.. • Instead.. n.CHAPTER 1. Because of this reason. σ. no symmetry. let us outline the basic “philosophy” of such simulations. we assume that the multitude of outcomes has provided a fair mapping of “reality”.36) where E1 (x) is the so-called first exponential integral. • These individual sub-processes are described by using pdf’s. here with respect to the radial direction (i. and on the absorber densities. 1 E(τ ) = 2 Z∞ 0 E(t)E1 |t − τ | dt.13 Example (Radiative transfer in stellar atmospheres). clumpy medium) mostly impossible. to a good approximation. which cannot (or only with enormous effort) be solved in a closed way. a deeper understanding of the results becomes rather difficult. Even in a simple geometry. the reliability of the results depends crucially on a “perfect” knowledge of the sub-processes and a proper implementation of all possible paths which can be realized during the simulation. At first. In particular. Since. • After having followed a sufficient number of runs. we do know the “physics” of the constituating sub-processes.will be reconsidered in the practical work to be performed.1 Basic philosophy and a first example • Assume that we have a compley physical (or economical etc.e. The final result is then obtained from multiple runs through the various possible process-chains. and no analytical approximation can be developed throughout the process of solution.

usually different interactions are possible. e. Mihalas. however.710446.g. Stellar atmospheres. 2. San Francisco. for more details see Sect.38) ≤ q(τ ) < 0. on the other hand. 1-27 . Fig. independent of τ (see.html. Freeman. τ = 0) is given by √ E(τ ) = 3 (τ + q(τ )) . p(µ)dµ ∼ dµ.CHAPTER 1. 1978. D. and the spatial radiative energy density distribution (normalized to its value at the outer boundary. 1. but only in a very complex way.de/people/puls/stellar at/stellar at.. (1.6 √1 3 (1. Chap. In the considered case (pure thomson-scattering). defined by τmax . the optical depth is easily calculated and moreover almost frequency independent.unimuenchen. θ (with respect to the radial direction).. which constitutes the real complicate part of An adequate solution by a Monte Carlo simulation.10: Monte Carlo Simulation: radiative transfer (electron scattering only) in stellar atmospheres. THEORY outside τ3 PSfrag replacements θ τ2 τ1 inside Figure 1. with the problem. can be realized as follows (cf. 6 Note that an approximate solution of Milne’s equation can be obtained in a much simpler way.36) can be exactly solved. • The probability for a certain emission angle. all these possibilites have to be accounted for in the calculation of the optical depth. (this is a general result for absorption of light) and • the scattering angle (at low energies) can be approximated to be isotropic. frequency. s2 ) = σ n(s)ds. or http://www.. is given by p(µ)dµ ∼ µdµ.37) r2 where the absorber density corresponds to the (free) electron density and the optical depth is defined between to spatial points with “co-ordinates” s1 and s2 and s is the geometrical length of the photon’s path. Analytic and MC solution of Milne’s integral equation Milne’s equation (1. E(0) q(τ ) is the so-called “Hopf-function”. Zr1 τ (s1 . 3 and 4).usm. µ = cos θ.3): • Photons have to be “created” emerging from the deepest layers. • The optical depth passed until the next scattering event can be described by the probability p(τ )dτ ∼ e−τ dτ. if we consider only frequencies being lower than X-ray energies. where the only difference compared to the exact solution regards the fact that q(τ ) results in 2/3.10.

the analytical result is well reproduced. Fig. cf. THEORY Figure 1.CHAPTER 1. From this example and accounting for our knowledge accumulated so far. optical depth lengths τ in Example 1. and the analytical one (Eq. 1) p(x) = 0 else. exponentially instead of uniformly? 1-28 . 1. • If the simulation is performed for a large number of photons.38) is given by the bold line. respectively) via a RNG. which are distributed according to a pdf e−τ dτ . 1) (or (0. i.11: Solution of Milne’s integral equation (Example 1. 1. • All photons have to be followed until they leave the atmosphere (τ ≤ 0) or are scattered back into the stellar interior (τ > τmax ). for example.11. we still do not know how to obtain random variables (variates) y which are distributed according to an arbitrary (normalized!) pdf.e. The MC solution is indicated by asterisks..13) by means of a Monte Carlo simulation using 106 photons.13. and the corresponding energies have to be summed up as a function of optical depth. ( 1 for x ∈ [0. an immediate problem becomes obvious: Though we are able to create uniform distributions on the interval [0. 1). f (y). How to create.

y + dy must be the same! 1.2 must be equal to the probability that y is in 0.39a) .14 Example.2 .4 (cf.3. Indeed.5. 1. this problem is central to Monte Carlo simulations. From this argumentation. 0.12. The probability that x is in 0.2] (instead of [0. and p(x)dx is the probability that x is in dx.CHAPTER 1. x + dx and y is a function of x. Example 1.12: The transformation method.4. . where we were forced to consider a uniform distribution on the interval [0. f (y)dy is the probability that y is in dy. the probability that x ∈ [0.12). then the probability that y is in the range y. 0. must be equal: If p(x)dx is the probability that x is within the range x. . where we required the variates to be drawn from a specially designed pdf in order to reduce the variance. and y = 2x. THEORY y y(x) dy PSfrag replacements x dx Figure 1. 1] (= 1) must correspond to the probability that y ∈ [0. dy 1-29 (1. x shall be distributed uniformely in [0. we encountered a similar problem already in example 1.1 The inversion. see text. Similarly.or transformation method Let p(x) and f (y) be different probability density functions (pdf’s) p(x)dx = P (x ≤ x0 ≤ x + dx) f (y)dy = P (y ≤ y 0 ≤ y + dy) with y = y(x): The random variable y shall be a function of the random variable x.2 Variates with arbitrary distribution Actually. and in Sect. P . .4. 2] (also = 1). 1]. A physical transformation means that corresponding probabilities.1]).2.1 . 1. 1. and will be covered in the next section. we thus have |p(x)dx| = |f (y)dy| or alternatively ¯ ¯ ¯ dx ¯ f (y) = p(x) ¯¯ ¯¯ . .

Then.. To create a variate which shall be distributed according to a given pdf. and y has to be drawn according to the relation b−a y = a + (b − a) · x. 1). y2 )dy1 dy2 | . 1). x has been calculated by a RNG ⇒ p(x) = 1. if x is uniformly distributed. x ∈ [0. step 2: We derive F (y). a random number.e.CHAPTER 1. ¯ ¯ ¯ ∂(x1 . we have to perform the following steps: step 1: If f (y) is not normalized. the inversion-(or transformation-) method requires that F (y) • can be calculated analytically and • can be inverted.41) .39b) Let now p(x) be uniformly distributed on [0. b] (cf. x2 ) ¯ ¯ f (y1 .g. 1-30 x ∈ [0. f (y) = 1 b−a ⇒ F (y) = Zy a 1 y−a dy = b−a b−a y−a (test: F (a) = 0. 1. Then ¯ ¯ ¯ y(x) ¯ Zx ¯ Z ¯ ¯ ¯ 0 f (y)dy ¯ dx = |f (y)dy| → dx = ¯ ¯ ¯ ¯ ymin ¯ 0 ¯ ¯ Zy ¯ ¯ ¯ 0 0¯ x = F (y) = ¯ f (y )dy ¯ ¯ ¯ ymin {z } | ⇒ with F (ymin ) = 0. distribution Thus we find that y = F −1 (x). f (y). The more-D generalization of this relation involves the Jacobian of the transformation.12). ok). we have to normalized it at first.40) In summary. by replacing f (y) → C · f (y) with C = F (ymax )−1 . |p(x1 . Example 1. y shall be drawn from a uniform distribution on [a. THEORY We have to use the absolute value because y(x) can be a (monotonically) increasing or decreasing function. has to be calculated. F (b) = 1.15 Example (1). y2 ) = p(x1 . whereas probabilities have to be larger than zero by definition. = x. step 3: By means of a RNG. F (ymax = y(1)) = 1. x2 )dx1 dx2 | = |f (y1 . e. (1. 1]. step 4: We then equalize F (y) =: x and step 5: solve for y = F −1 (x). y2 ) ¯ (1.. i. (1. cumulative prob. x2 ) ¯¯ ∂(y1 .

y2 ) is the product of two functions which depend only on y1 and y2 . let’s consider then p (1. uniformly distributed on (0. (1. 1). the generalization of the inversion method is straightforward.43b) i.17 Example (3). x2 . and the inversion can be performed by using interpolations on this grid.e. standard deviation = 1). y ≥ 0. y has to be drawn from an exponential distribution.. Recipe: Draw two randon numbers x1 . y2 ) = f (y1 ) · f (y2 ). 1−x 1−x 1. 2 1 f (y)dy = √ e−y /2 2π (here: mean 0.13. we like to draw y according to a normal distribution.43a) y1 = −2 ln x1 cos(2πx2 ) p y2 = −2 ln x1 sin(2πx2 ). f (y1 .3. then y is exponentially distributed! Test: f (y)dy = e−(− ln(1−x)) dy(x) and dy = because (1 − x) is distributed as x! x ∈ [0. Z f (y)dy cannot be calculated analytically or F (y) cannot be inverted and a tabulation method (see above) is discarded.g. (1. called 1-31 . x2 from a RNG. x2 ) ˛ ˛˛ ∂x ˛ ˛ ∂y2 ˛ ˛ = ˛ ∂y1 ˛− √1 e−y12 /2 · √1 e−y22 /2 ˛ . 1. Finally. The integral F (y) becomes tabulated as well. To solve exercise 2. which define the emission angle from the lower boundary in Example 1. as a function of y (e.. F (y) = Zy 0 ⇒ If one calculates y = − ln(1 − x) → 0 e−y dy 0 = 1 − e−y =: x y = − ln x. calculate variates according to p(µ)dµ ∼ µdµ. from a numerical integration on a certain grid with sufficient resolution). y2 ) = ˛˛ = ˛ ∂x2 ∂x2 ˛ ˛ ˛ ˛ ˛ ∂(y1 . Indeed. y = − ln x.39b) and p(x1 .16 Example (2). x2 ) = 1 we find ˛ ˛ ˛ ˛ ˛ 1 ˛ ∂x1 ˛ ˛ ∂(x1 . and both variables are normally distributed. there is no need to give up. « „ 1 x1 = exp − (y12 + y22 ) 2 „ « y2 2πx2 = arctan y1 y12 + y22 = −2 ln x1 y2 = tan(2πx2 ) y1 With the transformation (1. y2 ) 2π 2π ∂y1 ∂y2 Thus. then y1 and y2 as calculated by (1. f (y) = e −y .e.. f (y1 .19 Example (5). f (y1 . In case that the pdf is tabulated.CHAPTER 1. there exists a fairly simple procedure to calculate corresponding variates.43) are normally distribued! 1. respectively. µ = cos θ ≥ 0. Even if the inversion method cannot be applied (i. For this purpose.18 Example (4). 1).42) 1 1−x ! dx ⇒ f (y)dy = dx = p(x)dx. x1 . THEORY 1. we use two random variables.

x2 is of highest imporance here!!! For this pair then. and so on.13: The most simple implementation of von Neumann’s rejection method (see text). in Numerical Recipes. e. and we enclose it by a rectangle which has a length corresponding to the definition interval.2.g. At first. THEORY Figure 1.4. (1. at least regarding its most simplistic implementation.2 von Neumann’s rejection method (advanced reading) which will be outlined briefly. 1. another pair is drawn.45) if x1 . distributed uniformly according to the rectangle chosen (similar as in the “hit and miss” approach. must be normalized. As always..41). i. 1. we accept xi to be a variate distributed according to w. [x min . wmax . we draw a pair of random numbers.. (xi .44) (1. xi yi = = xmin + (xmax − xmin ) · x1 wmax · x2 . yi ). depends on the ratio of the area of rectangle and the area below w. xmax ].e. which uses an additional comparison function to minimize this ratio is presented. Again.. The optimization of this method.13 shows the principle of this method. reject xi : yi > w(xi ). have to comply with the maximum extent in x and y (Eq. of rejected variates. Fig. 7.CHAPTER 1.g. e. Chap. of course.4). Sect. The statistical independence of x1 . and a height which is larger/equal to the maximum of w(x). 1. The random co-ordinates. the pdf. x2 are pairs of consecutive random numbers drawn from a RNG.3.3. whereas otherwise we reject it! If a value was rejected. 1-32 . the number of “misses”. if accept xi : yi ≤ w(xi ). w(x). 1.

you will find the program rng simple ex1. This program calculates five random numbers by means of the linear congruential RNG as applied in Fig. Plot the distribution and compare with the manual (ps-figures can be created with the keyword \ps).e. plot the corresponding results obtained from the “minumumstandard generator” ran1.f90. Of course. i. a) At first.e.. 1. . Display the results for different A. i = 1. you will investigate the 2-D correlation of random numbers obtained from this representative (low cycle!) RNG. Copy this program to a working-file of your convenience and use this file for your future work. as a function of A. Plot typical cases for large and small correlation. yi for a sufficiently large sample.1 Exercise 1: Simple RNGs and graphical tests On the homepage of our course. .N . you should check the results from Fig. which can be read and plotted with the idl procedure test rng. create an output-file (to be called ran out) with entries x i . you have to modify the main program accordingly and to re-compile both programs together. . i.Chapter 2 Praxis: Monte Carlo simulations and radiative transfer 2. Note that such RNGs with M being a power of two should create the full cycle if C is odd and (A − 1) is a multiple of 4. A = 65 and C = 1. For this purpose. . What do you conclude? c) Finally. In the following. with M = 2048. for the same number of N .. A = 5. perform the appropriate modifications of the program.5.5. which is contained in the working directory as program file ran1. 9. b) After success. Have a look into this procedure as well! Convince yourself that all possible numbers are created (how do you do this?). modify the program in such a way that arbitrary values for A can be read in from the input (modifying C will not change the principal result).pro. .37. 1. Compare with the results from above and discuss the origin of the difference.f90. 2-1 .

. Compare with the exact value as given above.2 Exercise 2: Monte Carlo integration .1) with (2.the planckfunction One of the most important functions in astronomy is the planck-fucntion. and check the 1/ N -law of the MC integration. Z∞ Bν (T )dν = σB 4 T . For this purpose. From much simpler considerations (splitting the integral into two parts connected at x = 1). using random numbers as generated from ran1. resulting in the well-known stefan-boltzmann-law. PRAXIS: MONTE CARLO SIMULATIONS AND RADIATIVE TRANSFER 2. Use also different sample sizes.2) and the given value of σB . (Hint: Use a suitable substitution for the integration variable in (2. b) We will now determine the value of the integral (2. written in terms of the so-called specific intensity (which is a quantity closely related to the density of photons propagating into a certain direction and solid angle). it should be clear that the integral must be of order unity anyway. use different maximum-values (from the input). If you have made no error in your calculation (consistent units!). this constant is no natural constant but a product of different natural constants and the value of the dimensionless integral. which is π 4 /15 and can be derived. Z∞ 0 x3 dx.f90 and perform appropriate modifications. Strictly speaking.1) with frequency ν and temperature T . All other symbols have their usual meaning. is given by Bν (T ) = 1 2hν 3 hν c2 e kT −1 (2. As a most important result then.1)).CHAPTER 2. erg cm−2 s−1 grad−4 ) is the boltzmann-constant. from an analytic integration over the complex plane.2) 0 where σB ≈ 5. √ to examine which value is sufficient. −1 ex (2.g. use a copy of the program mcint ex2. π (2.3) a) Determine the value (three significant digits) of this integral by comparing (2.67 · 10−5 (in cgs-units. 2-2 . The total intensity radiated from a black body can be calculated by integrating over frequency. e. N . which. this exercise shows how simply the T 4 -dependence arising in the stefan-boltzman law can be understood. Since the integral extends to infinity which cannot be simulated. you should find a value close to the exact one.3) from a Monte Carlo integration.

3 Exercise 3: Monte Carlo simulation .. At first. Stellar atmospheres. in this exercise we will perform a Monte Carlo simulation to confirm the above result.13. i.e. the interested reader may have a look into Mihalas. optical path length and scattering angle. Eq. In order to avoid almost all subtleties of radiative transfer and corresponding approximate solutions2 . according to the comments given in the program. with absorption and emission processes assumed to be grey. Update always the radial optical depth of the photon. update the corresponding array counting the number of photons which have escaped under a certain range of angles.f90..uni-muenchen.4) where I1 is the specific intensity for the radial direction. 2. . which has been developed (long ago!) by Eddingtion and Barbier. on the other hand. If your program runs successfully.limb darkening in plane-parallel. frequency independent.18). and the corresponding region appears to be fainter. 2. develope a sketch of the program flow including all possible branches (a so-called flow-chart).4 is the consequence of an approximate solution of the equation of radiative transfer in plane-parallel symmetry1 . and follow the photons until they have left the atmosphere. as in Example 1. This can be clearly observed for the sun.105 photons and an optical depth of the atmosphere.de/people/puls/stellar at/stellar at.. The “only” part missing is the actual path of the photons. Just one additional comment: The above relation (2. contrasted to the sun’s radius of 700 000 km. In the latter case then.pro. e. D. one can approximiate the angular dependence of the specific intensity (see below) by the expression I(µ) ≈ I1 (0. 3 and 4). San Francisco... the direction cosine (i.g.4 carefully.4 + 0. Chap. but is the consequence of a large number of absorption and emission processes in an atmosphere of finite (optical) depth. by a factor of roughly 0. Freeman. discuss your flow-chart with your supervisor. the cosine of angle between radius vector and photon direction) is denoted by µ. PRAXIS: MONTE CARLO SIMULATIONS AND RADIATIVE TRANSFER 2.e. 2-3 .6 µ) (2. If. under the condition that the stellar photosphere is very thin compared to the stellar radius: the solar photosphere.1.4. For µ = 0. the intensity observed at the center of a star. τrmmax = 10. according to the comments in the program. Before implementing this program. but we will sort the photon’s now according to the direction they leave the atmosphere.e. 1 i. you should be able to program this path within few statements. i.4) has nothing to do with the presence of any temperature stratification (though it can be influenced by it). The angular distribution created can be displayed via the idl procedure limb test. If you have studied Sect. grey atmospheres Limb darkening describes that fact that a (normal) star seems to be darker at the edge than at its center.e. which has to be copied to a working file and then complemented. 2 Again. .usm. N = 104 . µ = 1. accounting for appropriate distributions of “emission angle” (Example 1. or http://www.. is only a few hundred kilometers thick.13) is based on the same approach. and not according to their energy. 1. you should have obtained a plot as given in Fig. b) Implement the algorithm developed in a) into your program and test it using 20 chanels for the angular distribution. 1978.html. as we will see below.CHAPTER 2. The principle strategy is very close to the simulation as described in Example 1.13. the angle between radial and photon’s direction is 90o (this situation is met at the limb of the star). a) The principle structure of the simulation (to be used again with ran1) is given in program limb ex3. Note that the approximate solution of Milne’s integral equation (Example 1.

so that we have to draw from the pdf dµ.pro in such a way as to display the specific intensity and compare with the prediction (2.CHAPTER 2. Thus. in order to convert to photon numbers. we have calculated the number of photons leaving the atmosphere with respect to a surface perpendicular to the radial direction. this number is proportional to the specific intensity weighted with the projection angle µ. N (µ)dµ ∝ I(µ)µdµ.5) and the intensity within dµ is obtained from the number of photons. Modify the plot routine limb test. PRAXIS: MONTE CARLO SIMULATIONS AND RADIATIVE TRANSFER Figure 2. c) In our simulation. is defined with respect to unit projected area. The total optical depth of the atmosphere is 10. I(µ). Hint: use the idl-procedure poly fit. described in idlhelp. i. which is assumed to be present at the lowermost boundary. Without going into details. it is the specific intensity and not the photon number which is uniformly distributed with respect to µ! Thus.4).4) from a linear regression to your results. centered at the mid of the corresponding channel (already implemented into the program output). 2. (2. Why is a certain discrepancy between simulation and theory to be expected? 2-4 . divided by an appropriate average of µ. on the other hand. and derive the limb-darkening coefficients (in analogy to Eq. This relation is also the reason why the distribution of the photons’ “emission angles” at the lower boundary follows the pdf µdµ: for an isotropic radiation field. the emission angle refers to the photons themselves and thus is (almost) isotropic. we have to draw the emission angles at the lower boundary from the pdf µdµ instead of dµ! Inside the atmosphere. since the specific intensity. Use N = 106 photons for τmax = 10 now..e.1: Monte Carlo simulation of limb darkening: angular distribution of 10 5 photons sorted into 20 channels.

Which angular distribution do you expect now for the specific intensity? Does your simulation verify your expectation? 2-5 .CHAPTER 2. PRAXIS: MONTE CARLO SIMULATIONS AND RADIATIVE TRANSFER d) To convince yourself that limb-darkening is the consequence of a multitude of scattering effects. reduce τmax to a very small value.

1-26 Pearson’s χ2 . 1-1 rejection method. 1-6 cumulative probability distribution. 2-2 standard deviation. 1-1 probability density function. 1-14 method. 1-19 Monte Carlo integration. 1-15 hit or miss method. 1-29 specific intensity. 1-21 hit or miss. 1-19 intensity specific. 1-27 Kolmogoroff-Smirnov-test. 1-30 random number generator linear congruential. 1-12 Mersenne prime number. 1-8 random numbers. 1-6 real. 1-1 expectation value. 1-22 Monte Carlo simulations. 1-11. 1-3. 1-31 optical depth. 1-3 pseudo random numbers. 1-6 limb darkening. 1-31 graphical test of random numbers. 1-31 normal. 2-2 inversion method. 1-13 normal distribution. 1-6 sub-. 1-31 uniform. 1-27 von Neumann’s rejection method. 1-23 hyper-surfaces. 1-30 2-6 . 1-12 Schrage’s multiplication. 1-1 continuous. 1-19 scaling law for errors. 1-5 Hopf-function. 1-12 transformation method. 2-3 Monte Carlo approximation. 1-5. 1-6 tests. 2-2 stefan-boltzmann-law. 1-24 sub-random numbers. 1-8 planck-function. 1-3 distribution exponential. 1-32 reshuffling. 1-23 tests of random numbers. 1-32 RANDU . 1-2 probability distribution cumulative. 1-15 more-dimensional tests. 1-6 graphical test. 1-2 stratified sampling. 1-2 exponential distribution. 1-13 pseudo-. 1-8 Milne’s integral equation. 1-29 uniform distribution. 1-24 integration Monte Carlo.Index Gauss-distribution. 1-19 cooking recipe. 1-6 minimum-standard. 1-26 probability. 1-16 event. 1-26 more-dimensional tests of random numbers. 1-3. 1-12 random variables. 1-15 importance sampling. 1-2 discrete. 2-2 thomson-scattering.

1-2 variance reduction.INDEX variance. 1-1 2-7 . 1-24 variate.