Professional Documents
Culture Documents
Contents
• Basic concepts
• Discrete random variables
• Discrete distributions (nbr distributions)
• Continuous random variables
• Continuous distributions (time distributions)
• Other random variables
2
4. Basic probability theory
3
4. Basic probability theory
Combination of events
• Union “A or B”: A ∪ B = {ω ∈ Ω | ω ∈ A or ω ∈ B}
• Intersection “A and B”: A ∩ B = {ω ∈ Ω | ω ∈ A and ω ∈
• Complement “not A”: B} Ac = {ω ∈ Ω | ω ∉ A}
• Events A and B are disjoint if
– A∩B=∅
• A set of events {B1, B2, …} is a partition of event A if
– (i) Bi ∩ Bj = ∅ for all i ≠ j
A
– (ii) ∪ i Bi = A
B1
B3
B2
4
4. Basic probability theory
Probability
5
4. Basic probability theory
Conditional probability
P( A | B) = P(P(B)
A∩B)
• It follows that
P( A ∩ B) = P(B)P( A | B) = P( A)P(B | A)
6
4. Basic probability theory
• Assume further that P(Bi) > 0 for all i. Then (by slide 6)
P( A) = ∑i P(Bi )P( A | Bi )
• This is the theorem of total probability Ω
B1 B3
A
B4
B2
7
4. Basic probability theory
Bayes’ theorem
P( A)P(Bi )P( A| i
P(Bi | A) =
∑ j P(B j )P( A|B j )
• B )theorem of total probability (slide 7), we get
Furthermore, by the
• This is Bayes’ theorem
– Probabilities P(Bi) are called a priori probabilities of events Bi
– Probabilities P(Bi | A) are called a posteriori probabilities of events Bi
8
(given that the event A occured)
4. Basic probability theory
P( A ∩ B) = P( A)P(B)
• It follows that
P( A | B) = P( A∩B) = P( A)P(B) = P( A)
P(B)
P(B)
• Correspondingly:
P( A)
4. Basic probability theory
Random variables
{X ≤ x} ∈
Example
X(ω) 0 1 1 1 2 2 2 3
11
4. Basic probability theory
Indicators of events
P{1A = 0} = P( Ac ) = 1− P( A)
12
4. Basic probability theory
FX (x) = P{X ≤ x}
• Cdf determines the distribution of the random variable,
– that is: the probabilities P{X ∈ B}, where B ⊂ ℜ and {X ∈
B} ∈
• Properties:
– (i) FX is non-decreasing
1
– (ii) F is continuous from the right FX(x)
X
– (iii) FX (−∞) = 0
– (iv) FX (∞) = 1 x
0
13
4. Basic probability theory
14
4. Basic probability theory
15
4. Basic probability theory
Contents
• Basic concepts
• Discrete random variables
• Discrete distributions (nbr distributions)
• Continuous random variables
• Continuous distributions (time distributions)
• Other random variables
16
4. Basic probability theory
P{X ∈ S X } = 1
• It follows that
– P{X = x} ≥ 0 for all x ∈ SX
– P{X = x} = 0 for all x ∉ SX
• The set SX is called the value set
17
4. Basic probability theory
Point probabilities
pi := P{X = xi}, xi ∈ S X
• Definition: The probability mass function (pmf) of X is a function
pX: ℜ → [0,1] defined as follows:
pi , x = xi ∈ S X
pX (x) := P{X = x} =
• Cdf is in this case a step function: 0, x ∉ SX
FX (x) = P{X ≤ x} = ∑ pi
i:xi ≤ x
18
4. Basic probability theory
Example
pX(x) FX(x)
1 1
x x
x1 x2 x3 x4 x1 x2 x3 x4
20
4. Basic probability theory
Expectation
µ X := E[ X ] := ∑ P{X = x}⋅ x =
∑ p X (x)x = ∑ pi xi x∈SX
x∈SX i
– Note 1: The expectation exists only if Σi pi|xi| < ∞
– Note 2: If Σi pi xi = ∞, then we may denote E[X] = ∞
• Properties:
– (i) c ∈ ℜ ⇒ E[cX] = cE[X]
– (ii) E[X + Y] = E[X] + E[Y] 21
– (iii) X and Y independent ⇒ E[XY] = E[X]E[Y]
4. Basic probability theory
Variance
σ X2 := D2 [ X ] := Var[ X ] := E[( X −
E[ X ])2 ]
• Useful formula (prove!):
D2 [ X ] = E[ X 2 ] − E[ X
2
]
• Properties:
– (i) c ∈ ℜ ⇒ D2[cX] = c2D2[X]
– (ii) X and Y independent ⇒ D2[X + Y] =
D2[X] + D2[Y]
22
4. Basic probability theory
Covariance
Cov[ X ,Y ] = E[ XY ] − E[ X ]E[Y ]
• Properties:
– (i) Cov[X,X] = Var[X]
– (ii) Cov[X,Y] = Cov[Y,X]
– (iii) Cov[X+Y,Z] = Cov[X,Z] + Cov[Y,Z]
– (iv) X and Y independent ⇒ Cov[X,Y] = 0
23
4. Basic probability theory
σ X := D[ X ] := D2 [ X ] = Var
[X]
• Definition: The coefficient of variation of X is defined by
c X := C[ X ] :=E[ X
D[ X ] ]
• Definition: The kth moment, k=1,2,…, of X is defined by
µ X(k ) := E[ X
k]
24
4. Basic probability theory
E[ X n ] = µ
2
D2 [ X n ] = σn
D[ X n ] = σn
25
4. Basic probability theory
Xn → µ
26
4. Basic probability theory
Contents
• Basic concepts
• Discrete random variables
• Discrete distributions (nbr distributions)
• Continuous random variables
• Continuous distributions (time distributions)
• Other random variables
27
4. Basic probability theory
Bernoulli distribution
Binomial distribution
(in ) i!(n−i)!
– n = total number of experiments
–
n!
p = probability of success in any single experiment
• Value set: SX = {0,1,…,n} =
n!=n⋅(n−1)2⋅1
• Point probabilities:
()
P{X = i} = in ip (1− n−i
• p)
Mean value: E[X] = E[X ] + … + E[X ] = np
1 n
• (independence!)
Variance: D2[X] = D2[X1] + … + D2[Xn] = np(1 − p)
29
4. Basic probability theory
Geometric distribution
P{X = i} = pi (1− p)
• Mean value: E[X] = ∑i ipi(1 − p) = p/(1 − p)
• Second moment: E[X2] = ∑i i2pi(1 − p) = 2(p/(1 − p))2 + p/(1 − p)
• Variance: D2[X] = E[X2] − E[X]2 = p/(1 − p)2
30
4. Basic probability theory
P{X ≥ i + j | X ≥ i} = P{X ≥ j}
• Prove!
– Tip: Prove first that P{X ≥ i}
= pi
31
4. Basic probability theory
and
1 − pi , i
P{X min = X i} =
1 − p1 ∈{1,2}
• Prove! p2
– Tip: See
slide 15
32
4. Basic probability theory
Poisson distribution
X ∼ Poisson(a), a > 0
– limit of binomial distribution as n → ∞ and p → 0 in such a way that np
→a
• Value set: SX = {0,1,…}
• Point probabilities: i −a
P{X = i} = i! e a
33
4. Basic probability theory
Example
• Assume that
– 200 subscribers are connected to a local exchange
– each subscriber’s characteristic traffic is 0.01 erlang
– subscribers behave independently
• Then the number of active calls X ∼ Bin(200,0.01)
• Corresponding Poisson-approximation X ≈ Poisson(2.0)
• Point probabilities:
0 1 2 3 4 5
34
4. Basic probability theory
Properties
X1 + X 2 ∼ Poisson(a1 + a2 )
• (ii) Random sample: Let X ∼ Poisson(a) denote the number of
elements in a set, and Y denote the size of a random sample of this set
(each element taken independently with probability p). Then
Y ∼ Poisson( pa)
• (iii) Random sorting: Let X and Y be as in (ii), and Z = X − Y. Then
Y and Z are independent (given that X is unknown) and
Z ∼ Poisson((1− p)a)
35
4. Basic probability theory
Contents
• Basic concepts
• Discrete random variables
• Discrete distributions (nbr distributions)
• Continuous random variables
• Continuous distributions (time distributions)
• Other random variables
36
4. Basic probability theory
x
FX (x) := P{X ≤ x} = ∫ f X ( y)
dy
• The function fX is called the probability −∞
density function (pdf)
– The set SX, where fX > 0, is called the value set
• Properties:
– (i) P{X = x} = 0 for all x ∈ ℜ
– (ii) P{a < X < b} = P{a ≤ X ≤ b} = ∫ b f (x) dx
a
X
= ∫∫-∞
–– (iv) P{X ∈ A}
P{X ∈ ℜ} = ∞ f (x) dx = ∫ fX(x) dx = 1
(iii) A X(x) dx
f X S
X 37
4. Basic probability theory
Example
fX(x) FX(x)
1
x x
x1 x2 x3 x1 x2 x3
SX = [x1, x3]
38
4. Basic probability theory
39
4. Basic probability theory
Contents
• Basic concepts
• Discrete random variables
• Discrete distributions (nbr distributions)
• Continuous random variables
• Continuous distributions (time distributions)
• Other random variables
40
4. Basic probability theory
Uniform distribution
Exponential distribution
X ∼ Exp(λ), λ > 0
– continuous counterpart of geometric distribution (“failure” prob. ≈
λdt)
• Value set: SX = (0,∞)
• Probability density function (pdf): −λx
f X (x) = λe , x >
• 0 function (cdf):
Cumulative distribution
– Prove!
• Tip: Prove first that P{X > x} = e−λx
• Application:
– Assume that the call holding time is exponentially distributed with mean h
(min).
– Consider a call that has already lasted for x minutes.
Due to memoryless property,
this gives no information about the length of the remaining holding time: it is
distributed as the original holding time and, on average, lasts still h minutes!
43
4. Basic probability theory
λi , i
P{X min = X i } =
λ1 +λ2 ∈{1,2}
• Prove!
– Tip: See
slide 15
44
4. Basic probability theory
X ∼ N(0,1)
– limit of the “normalized” sum of IID r.v.s with mean 0 and variance 1 (cf.
slide 48)
• Value set: SX = (−∞,∞)
• Probability density function (pdf):
− 12 x2
f X (x) = ϕ(x) := e
1
• Cumulative distribution function (cdf):
2π x
FX (x) := P{X ≤ x} = ∫−∞ ϕ( y)
• Φ(x)
Mean value: :== 0 (symmetric pdf)
E[X] dy
• Variance: D2[X] = 1
45
4. Basic probability theory
{
FX (x) := P{X ≤ x} = P σX − µ ≤ σx − µ }= σ
( )
• Mean value: E[X] = µ + σE[(X − µ)/σ] = µ (symmetric pdf around µ)
x−µ
• Φ
Variance: D2[X] = σ2D2[(X − µ)/σ] = σ2
46
4. Basic probability theory
47
4. Basic probability theory
X n ≈ N(µ, n1 σ 2 )
48
4. Basic probability theory
Contents
• Basic concepts
• Discrete random variables
• Discrete distributions (nbr distributions)
• Continuous random variables
• Continuous distributions (time distributions)
• Other random variables
49
4. Basic probability theory
FW(x) 1
1−ρ
x
0
0
50