You are on page 1of 15



Probability & Statistics
• The theory of probability is a branch of pure mathematics. From a certain set of axioms and definitions mathematicians build up the theory by deduction. • In contrast, for a physicist, statistics is a branch of applied mathematics which is essentially inductive. Nevertheless statistics is intimately connected with the theory of probability. Example: Suppose that it is known that when a coin a has 1 an a priori probability p = 2 of landing on one specific side (head) and a probability (1 − p) of landing on the other sides (tail). • What is the probability of observing r heads out of n tosses??? • This is a question in probability theory and the answer is provided by the binomial distribution law [Chap. 5]: n! pr (1 − p)n−r . r!(n − r)!

P (r; n) = &


Computational Physics - Phys 75.487/502 c Alain Bellerive - Carleton University


• A reasonable answer to this question is to say that the most likely value of the parameter is given by: r (2) p=p= . ˜ n The experiment has then given a point estimate p = p for ˜ the unknown parameter. Computational Physics . However from the nature of the experiment.Phys 75.' $ Probability & Statistics (con’t) • A complete different situation exists if one has NO a priori knowledge of the probability p and decides to perform an experiment to determined this parameter. r heads were observed.487/502 c Alain Bellerive . given that in n tosses.Carleton University & % 2 . It would then be a question of statistics to ask what the parameters p is like. we are not completely sure that the value p ˜ obtained is identical to the true value of the parameter p. • A simple experiment would consist in tossing a coin repeatedly and counting how many times the outcome heads would occur.

then one should obtain a different estimate for p. p2 ] really includes the true value of p. • Thus. we shall state (not prove) the rules that govern the theory of probability.Carleton University % 3 .487/502 c Alain Bellerive . & Computational Physics .' $ Experiment • Intuitively one expects that if the coin experiment was repeated with new sequences tosses. THIS IS STATISTICAL INFERENCE! • Generality: larger(smaller) the interval. • The faith that p lies within [p1 . n): to make statistical inference it is necessary to know the functional form of the binomial distribution. • Since the actual calculation of a confidence interval require assumption about the probability P (r. larger(smaller) the certitude that [p1 .Phys 75. instead of stating the result of the experiment in terms of a single number p one could give an interval ˜ estimate [p1 . • Hence. p2 ] could then be expressed by a confident level. p2 ] for the parameter p.

where P (E) = 0 if the EVENT never occurs and P (E) = 1 if it always occurs.Carleton University % 4 . • In this course we will use the frequency approach.487/502 c Alain Bellerive . & Computational Physics .Phys 75.' $ Definition of Probabilities • Physicist do not agree about the best way to define probability! • Frequentist versus Bayesian. • Frequentist assumption: converge to the ’true’ value as n → ∞. in a sequence of n trials of an experiment the outcome of a specified EVENT (or class) occurs r times: r in the limit n → ∞ . ADOPT THE DEFINITION OF PROBABILITY IN TERM OF THE LIMIT OF RELATIVE FREQUENCY OCCURRENCE! • Thus. (3) P (E) = n • From this definition 0 ≤ P (E) ≤ 1.

• The probability P (x ≤ X ≤ x + dx) is associated to the event of getting a measurement within the interval [x. • We define the probability density function f (x) for the continuous random variable X by the equation: f (x) dx = P (x ≤ X ≤ x + dx) . The exhaustive requirement is then f (x) dx = 1 Ω (5) Computational Physics .Carleton University & % 5 .487/502 c Alain Bellerive .' $ Random Variables and Sample Space • Random variable: prior to an experiment the outcome cannot be predicted with complete certainty. • Associate each outcome xi of the experiment with the probability Pi such that: n P (X = xi ) = Pi with i=1 Pi = 1 (exhaustive).Phys 75. All the outcome are said to be exclusive. x + dx]. (4) • Continuous: Random variable can have a continuum of values within any finite interval Ω.

Element An object which belongs to a set A is said to be an element of A. Subset If every element of the set B is also an element of the set A. ¯ Complement In a sample space Ω. Intersection A ∩ B is the set of element that belong to A and B.Phys 75. A 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 B Venn Diagram Computational Physics .Carleton University & % 6 . we say the B is a subset of A. Use P (AB) ≡ P (A ∩ B). Union A ∪ B is the set of element that belong to A or B. the complement A is the set of all elements in Ω that do not belong to A.487/502 c Alain Bellerive . or both.' $ Definitions Set The concept of a set is used to denote a collection of objects with some common properties.

Q: How can we express the probability of the subset B relative to the new sample space A? A: This new probability is called the conditional probability of B relative to A: P (B|A).' $ Rules Axioms • P (E)Ω = 1 • P (A) ≥ 0 ¯ • P (A + A) = 1 • P (A) = N (A) N (E) P (B) = N (B) N (E) P (AB) = N (AB) N (E) Addition P (A ∪ B) = P (A) + P (B) − P (A ∩ B) Mutually exclusive P (A + B) ≡ P (A ∪ B) = P (A) + P (B) Conditional Probability Consider A and B two subsets of the sample space Ω.487/502 c Alain Bellerive .Carleton University % 7 . & Computational Physics . Suppose that we are interested only in the subset A. P (B|A) It should read: “probability of B given A”.Phys 75.

487/502 c Alain Bellerive .Carleton University & % 8 .Phys 75. N (EA) N (A) N (A)/N (E) (6) $ P (B|A) = P (B|A) = P (AB) P (A) • Assume E= n Ai i=1 • Then n P (B) = P (EB) = P (AB) = i=1 P (Ai )P (B|Ai ) E A2 A3 A4 A5 B A1 An Computational Physics .' Conditional Probability N (AB) N (AB) N (AB)/N (E) = = .

Phys 75.487/502 c Alain Bellerive . This is a fundamental theorem of probability theory.Carleton University % 9 . • Then for 1 ≤ k ≤ n fixed P (Ak |B) = Then P (Ak |B) = Finally P (Ak B) P (B) P (B|Ak )P (Ak ) P (B) P (Ak |B) = P (B|Ak )P (Ak ) n P (B|A )P (A ) i i i=1 Bayes theorem is concerned with conditional probabilities where P (B) and P (A) are the unconditional (or a priori) probabilities of B and A. & Computational Physics .' $ Bayes Theorem • Consider 1) A1 · · · An a partition of E and 2) B an event of A. but its use in statistics is a subject of some controversy (Bayesian Statistics). respectively.

• P (d) is the prior probability of obtaining data d. What is called a Bayesian viewpoint is the application of the laws of probability to non-repeatable events: H is a hypothesis or proposition. % 10 . in which statistics is derived from a probability interpretation that includes the degree of belief in an hypothesis. • It can be rewritten using the other terms as: P (d) = i P (d|Hi )P (Hi ). after the experiment which produced data d.487/502 c Alain Bellerive . It thus refers not only to repeatable measurements (as does the frequentist interpretation).Phys 75. Computational Physics . either true or untrue.' Bayesian Approach $ An important school of statistical theory. The interpretation of data can be described by Bayes Theorem for an hypothesis H and experimental data d. • P (H) is the prior probability of H being true.Carleton University & Remark: We will mainly use the frequentist approach in this course. where summation runs over all hypotheses. P (H|d) = P (d|H)P (H) P (d) (7) Bayesian meaning of the different terms: • P (H|d) is the degree of belief in the hypothesis H. • P (d|H) is the ordinary likelihood function used also by non-Bayesian. and P (H) is interpreted as the degree of belief in the proposition.

Phys 75. then we say that A and B are dependent: P (B|A) = P (AB) = P (B) P (A) (8) • But if A and B are independent if and only if: P (B|A) = P (B) (9) INDEPENDENCE IMPLIES: P (AB) = P (A) P (B) Computational Physics .' $ Independence: Multiplication Rule • The realization of A influenced the probability of B.487/502 c Alain Bellerive .Carleton University & % 11 .

Assuming all relays act independantly.Carleton University % 12 . Answer: P (E) = P (E1 ) P (E2 ) P (E3 ) = α3 .e. find the probability for the flow of a current between the terminal. Rules: P (E) = P (E1 ∩ E2 ∩ E3 ).and.487/502 c Alain Bellerive .Phys 75.' $ Example: Relay Networks I 1 2 3 Relay Network Question: The probability for the closing of each relay in the circuit is some given number α.and. & Computational Physics .E3 . Hint: The current will flow (event E) if it can go in the branch: i.E2 . E = E1 .

find the probability for the flow of a current between the terminal.and. Rules: The current will flow if the condition E = E1 . Answer: Since P (A ∪ B) = P (A) + P (B) − P (A ∩ B).487/502 c Alain Bellerive .Phys 75. Computational Physics .(E2 .E3 ) is satistied. P (E) = P (E1 ) + P (E2 ∩ E3 ) − P (E1 ∩ [E2 ∩ E3 ]) P (E) = P (E1 )+P (E2 ) P (E3 )−P (E1 ) P (E2 ) P (E3 ) = α+α2 −α3 .Carleton University & % 13 . i. Hint: The current will flow if it can go in one of the branch. Assuming all relays act independantly.e.' Example: Relay Networks II 1 $ 2 3 Relay Network Question: The probability for the closing of each relay in the circuit is some given number α.or. P (E) = P (E1 ∪ [E2 ∩ E3 ]).

is detected by a spherical arrangement of phototubes as illustrated in Figure 1.93. from particles traversing a Cerenkov counter along their path. Assume all phototubes to act independently. In order to discriminate against accidental triggering of the system.' $ Example: Coincidence Suppose the Cerenkov light.f.f. If the event E of having a signal from one phototube corresponds to a probability P (E) = ǫ = 0. (c) Which experimental set-up gives the largest detector efficiency? Ref: Probability and Statistics in Partcile Physics (Frodesen et al.487/502 c Alain Bellerive . Figure 1a)? (b) What is the probability for the detection of the Cerenkov light when the tubes are grouped together three by three (c. Figure 1b)? Each group is activated if at least one of the tubes in the group has a signal. (a) What is the probability for the detection of the Cerenkov light by all nine independent phototubes (c.Phys 75.Carleton University & % 14 .) Computational Physics . it is desirable to observe coincidences between the signals from several phototubes.

' $ Example: Coincidence (con’t) OR OR OR (a) 9−fold coincidence (b) 3−fold coincidence Figure 1: Arrangements of nine phototubes in a Cerenkov counter.Carleton University & % 15 .Phys 75. Computational Physics .487/502 c Alain Bellerive .