Week2 Annotated Final

Independence of two random variables
Definition (Independence (two random variables))

Let X and Y be two random variables defined in a probability space with
ranges TX and TY , respectively. X and Y are said to be independent if any
event defined using X alone is independent of any event defined using Y
alone. Equivalently, if the joint PMF of X and Y is fXY , X and Y are
independent if
fXY (t1 , t2 ) = fX (t1 )fY (t2 )
for t1 ∈ TX , t2 ∈ TY .
General case vs independent case

I General: fXY (t1 , t2 ) = fX (t1 )fY |X =t1 (t2 ) (always true)
I Independent: fY |X =t1 (t2 ) = fY (t2 )
If X and Y are independent
I Joint PMF equals product of marginal PMFs
I Conditional PMF equals marginal PMF
Week 3: Multiple Random Variables 43 / 48

Simple examples
t2 \ t1 0 1 fY
0 1/4 1/4
1 1/4 1/4
fX

Simple examples
t2 \ t1 0 1 fY t2 \ t1 0 1 fY
0 1/4 1/4 0 1/2 1/4
1 1/4 1/4 1 1/8 1/8
fX fX

Simple examples
t2 \ t1 0 1 fY t2 \ t1 0 1 fY
0 1/4 1/4 0 1/2 1/4
1 1/4 1/4 1 1/8 1/8
fX fX
t2 \ t1 0 1 fY
0 x 1/2 − x
1 1/2 − x x
fX

More examples
t2 \ t1 0 1 2 fY
0 1/9 1/9 1/9
1 1/9 1/9 1/9
2 1/9 1/9 1/9
fX

More examples
t2 \ t1 0 1 2 fY t2 \ t1 0 1 2 fY
0 1/9 1/9 1/9 0 1/6 1/12 1/12
1 1/9 1/9 1/9 1 1/4 1/8 1/8
2 1/9 1/9 1/9 2 1/12 1/24 1/24
fX fX

More examples
t2 \ t1 0 1 2 fY t2 \ t1 0 1 2 fY
0 1/9 1/9 1/9 0 1/6 1/12 1/12
1 1/9 1/9 1/9 1 1/4 1/8 1/8
2 1/9 1/9 1/9 2 1/12 1/24 1/24
fX fX
t2 \ t1 0 1 2 fY
0 0 1/8 1/8
1 1/8 1/8 1/8
2 1/8 1/8 1/8
fX

More examples
t2 \ t1 0 1 2 fY t2 \ t1 0 1 2 fY
0 1/9 1/9 1/9 0 1/6 1/12 1/12
1 1/9 1/9 1/9 1 1/4 1/8 1/8
2 1/9 1/9 1/9 2 1/12 1/24 1/24
fX fX
t2 \ t1 0 1 2 fY t2 \ t1 0 1 2 fY
0 0 1/8 1/8 0 1/6 1/12 1/8
1 1/8 1/8 1/8 1 1/4 1/8 1/8
2 1/8 1/8 1/8 2 1/24 1/24 1/24
fX fX

Independent vs dependent random variables
To show X and Y are independent, verify
fXY (t1 , t2 ) = fX (t1 )fY (t2 )
for all t1 ∈ TX , t2 ∈ TY .
To show X and Y are dependent, verify
fXY (t1 , t2 ) 6= fX (t1 )fY (t2 )
for some t1 ∈ TX and t2 ∈ TY .

I Special case: fXY (t1 , t2 ) = 0 when fX (t1 ) 6= 0, fY (t2 ) 6= 0

Example: Random 2-digit number
A 2-digit number from 00 to 99 is selected at random. Partial information is
available about the number as two random variables. Let X be the digit in
units place. Let Y be the reminder obtained when the number is divided by
4.

Example: IPL powerplay over
Let X = number of runs in the over. Let Y = number of wickets in the
over.

Independence of multiple random variables
Definition (Independence of random variables)

Let X1 , . . . , Xn be random variables defined in a probability space with
range of Xi denoted TXi . X1 , . . . , Xn are said to be independent if events
defined using different Xi are mutually independent. Equivalently,
X1 , . . . , Xn are independent iff
fX1 ···Xn (t1 , . . . , t2 ) = fX1 (t1 )fX2 (t2 ) · · · fXn (tn )
for all ti ∈ TXi .
If X and Y are independent

I Joint PMF equals product of marginal PMFs
I Conditional PMF equals unconditioned PMF
All subsets of independent random variables are independent

Example: Toss a fair coin thrice
Let Xi = 1 if i-th toss is heads and Xi = 0 if i-th toss is tails, i = 1, 2, 3.

Joint PMF: fX1 X2 X3 (t1 , t2 , t3 ) = 1/8 for ti ∈ {0, 1}


Xi ∼ Uniform{0, 1}
fX1 X2 X3 (t1 , t2 , t3 ) = fX1 (t1 )fX2 (t2 )fX3 (t3 )


fX1 X2 X3 (t1 , t2 , t3 ) = fX1 (t1 )fX2 (t2 )fX3 (t3 )

X1 , X2 , X3 : Independent random variables

Example: Random 3-digit number 000 to 999
X = first digit from left, Y = number modulo 2, Z = first digit from right
X ∼ Uniform{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
Y ∼ Uniform{0, 1}
Z ∼ Uniform{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}

X ∼ Uniform{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
Y ∼ Uniform{0, 1}
Z ∼ Uniform{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
X, Z
I fXZ (t1 , t3 ) = 1/100 = fX (t1 )fZ (t3 )
I Independent
X, Y
I fXY (t1 , t2 ) = 1/20 = fX (t1 )fY (t2 )
I Independent
Y,Z
I fYZ (1, 2) = 0 6= fY (1)fZ (2)
I Not independent

X ∼ Uniform{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
Y ∼ Uniform{0, 1}
Z ∼ Uniform{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
X, Z
I fXZ (t1 , t3 ) = 1/100 = fX (t1 )fZ (t3 )
I Independent
X, Y
I fXY (t1 , t2 ) = 1/20 = fX (t1 )fY (t2 )
I Independent
Y,Z
I fYZ (1, 2) = 0 6= fY (1)fZ (2)
I Not independent
X , Y , Z : Dependent random variables

Example: Even parity
t1 t2 t3 fX1 X2 X3 (t1 , t2 , t3 )
0 0 0 1/4
0 1 1 1/4
1 0 1 1/4
1 1 1 1/4
Number of 1s in (X1 , X2 , X3 ) is even (hence, the name)

t1 t2 t3 fX1 X2 X3 (t1 , t2 , t3 )
0 0 0 1/4
0 1 1 1/4
1 0 1 1/4
1 1 1 1/4

All pairs are independent
fX1 X2 X3 (0, 0, 1) = 0 6= fX1 (0)fX2 (0)fX3 (1)

t1 t2 t3 fX1 X2 X3 (t1 , t2 , t3 )
0 0 0 1/4
0 1 1 1/4
1 0 1 1/4
1 1 1 1/4

All pairs are independent
fX1 X2 X3 (0, 0, 1) = 0 6= fX1 (0)fX2 (0)fX3 (1)
X1 , X2 , X3 : Dependent random variables

Independent and Identically Distributed (i.i.d.)
Definition (i.i.d.)
Random variables X1 , . . . , Xn are said to be independent and identically
distributed (i.i.d.), if
1 they are independent,
2 the marginal PMFs f

Xi are identical.

Definition (i.i.d.)

Xi are identical.
Repeated trials of an experiment creates i.i.d. sequence of random

variables
I Toss a coin multiple times
I Throw a die multiple times

Definition (i.i.d.)

Xi are identical.
Repeated trials of an experiment creates i.i.d. sequence of random

variables
I Toss a coin multiple times
I Throw a die multiple times
Notation: X1 , X2 , . . . , Xn are i.i.d. with distribution X means that fXi is the

same as fX

Problem: i.i.d. Geometric
Let X1 , . . . , Xn be i.i.d. with a Geometric(p) distribution. What is the
probabilty that all of these random variables are larger than some positive
integer j?

Problem: i.i.d. samples
1/2 1/4 1/8 1/16 1/16
Let X ∼ { 0 , 1 , 2 , 3 , 4 }, and let X1 , . . . , Xn be i.i.d. samples with
distribution X .
1 What is the probability that 4 is missing in the samples?
2 What is the probability that 4 appears exactly once in the samples?
3 What is the probability that 3 and 4 appear at least once in the
samples?

Problem: Memoryless property of Geometric
Let X ∼ Geometric(p). Find the following.
1 P(X > n)
2 P(X > m + n|X > m)

Visualizing functions of random variables
Andrew Thangaraj
IIT Madras
Andrew Thangaraj (IIT Madras) Visualizing functions of random variables 1 / 28

Section 1
One random variable, one-to-one functions

Uniform{0, 1, . . . , 10} and Binomial(10,0.5): PMF plots
Uniform{0, 1, , 10}
0.05
0.00
0 2 4 6 8 10
Binomial(10,0.5)
0.2
0.1
0.0
0 2 4 6 8 10
PMF of Y = X − 5 (table method)
Uniform{0, 1, . . . , 10} Binomial(10,0.5)
x P(X = x ) y =x −5 x P(X = x ) y =x −5
0 1/11 -5 0 0.00097656 -5
1 1/11 -4 1 0.00976563 -4
2 1/11 -3 2 0.04394531 -3
3 1/11 -2 3 0.1171875 -2
4 1/11 -1 4 0.20507813 -1
5 1/11 0 5 0.24609375 0
6 1/11 1 6 0.20507813 1
7 1/11 2 7 0.1171875 2
8 1/11 3 8 0.04394531 3
9 1/11 4 9 0.00976563 4
10 1/11 5 10 0.00097656 5

PMF plots of X ∼ Uniform{0, 1, . . . , 10} and Y = X − 5
X Uniform{0, 1, , 10}
0.05
0.00
0 2 4 6 8 10
Y=X 5
0.05
0.00
6 4 2 0 2 4 6
PMF plots of X ∼ Binomial(10,0.5) and Y = X − 5
X Binomial(10,0.5)
0.2
0.1
0.0
0 2 4 6 8 10
Y=X 5
0.2
0.1
0.0
6 4 2 0 2 4 6
PMF of Y = 2X (table method)
Uniform{0, 1, . . . , 10} Binomial(10,0.5)
x P(X = x ) y = 2x x P(X = x ) y = 2x
0 1/11 1 0 0.00097656 1
1 1/11 2 1 0.00976563 2
2 1/11 4 2 0.04394531 4
3 1/11 8 3 0.1171875 8
4 1/11 16 4 0.20507813 16
5 1/11 32 5 0.24609375 32
6 1/11 64 6 0.20507813 64
7 1/11 128 7 0.1171875 128
8 1/11 256 8 0.04394531 256
9 1/11 512 9 0.00976563 512
10 1/11 1024 10 0.00097656 1024

PMF plots of X ∼ Uniform{0, 1, . . . , 10} and Y = 2X
X Uniform{0, 1, , 10}
0.05
0.00
0 2 4 6 8 10
Y = 2X
0.05
0.00
0 200 400 600 800 1000
PMF plots of X ∼ Binomial(10,0.5) and Y = 2X
X Binomial(10,0.5)
0.2
0.1
0.0
0 2 4 6 8 10
Y = 2X
0.2
0.1
0.0
0 200 400 600 800 1000
One-to-one functions
One-to-one: For different input, the function results in different outputs

I In the plot of y = f (x ) versus x , every horizontal line intersects the
curve only at one point.
I Monotonic functions are one-to-one
I Example: x − 5, 2x
If f is one-to-one
I Easy to find PMF of f (X ) using the table method
I Easy to visualize the plot of the PMF
P(Y = f (x )) = P(X = x )

Section 2
One random variable, many-to-one functions

PMF of Y = (X − 5)2 (table method)
X ∼ Uniform{0, 1, . . . , 10}
x P(X = x ) y = (x − 5)2
0 1/11 25
1 1/11 16
2 1/11 9
3 1/11 4
4 1/11 1
5 1/11 0
6 1/11 1
7 1/11 4
8 1/11 9
9 1/11 16
10 1/11 25

X ∼ Uniform{0, 1, . . . , 10} Y = (X − 5)2
x P(X = x ) y = (x − 5)2 y P(Y = y )

0 1/11 25 0 1/11
1 1/11 16 1 2/11
2 1/11 9 4 2/11
3 1/11 4 9 2/11
4 1/11 1 16 2/11
5 1/11 0 25 2/11
6 1/11 1
7 1/11 4
8 1/11 9
9 1/11 16
10 1/11 25

X ∼ Uniform{0, 1, . . . , 10} Y = (X − 5)2
x P(X = x ) y = (x − 5)2 y P(Y = y )

0 1/11 25 0 1/11
1 1/11 16 1 2/11
2 1/11 9 4 2/11
3 1/11 4 9 2/11
4 1/11 1 16 2/11
5 1/11 0 25 2/11
6 1/11 1
7 1/11 4
(x − 5)2 is not one-to-one
8 1/11 9
9 1/11 16
10 1/11 25

PMF plots: X ∼ Uniform{0, 1, . . . , 10}, Y = (X − 5)2
X Uniform{0, 1, , 10}
0.1
0.0
0 2 4 6 8 10
Y = (X 5)2
0.1
0.0
0 5 10 15 20 25
PMF plots: X ∼ Binomial(10,0.5), Y = (X − 5)2
X Binomial(10,0.5)
0.4
0.2
0.0
0 2 4 6 8 10
Y = 2X
0.4
0.2
0.0
0 5 10 15 20 25
Many-to-one functions
Many-to-one: For two or more different input, the function should
result in the same output
I In the plot of y = f (x ) versus x , some horizontal line intersects the
curve at more than one point.
I Non-monotonic functions are many-to-one
I Example: y = x 2 , y = x (1 − x ), y = xe −x
If f is many-to-one
I The table method will generally work. . . .
I Let y0 be a value taken by f at the points x1 , x2 , . . . , xm and nowhere
else. That is, y0 = f (x1 ) = · · · = f (xm ).
P(Y = y0 ) = P(X = x1 ) + · · · + P(X = xm )

Problem
Let X ∼ Uniform{−5, −4, . . . , 5}. Let
(
x, x > 0
f (x ) =
0, x ≤ 0.
Find the distribution of Y = f (X ).

Problem
Let X ∼ Uniform{−500, −499, . . . , 500}. Let f (x ) = max(x , 5). Find the
distribution of Y = f (X ).

Section 3
Two random variables

Small examples and table method: Sum
X , Y ∼ iid Unif{0, 1}, Z = X + Y
x y fXY (x , y ) z
0 0 1/4 0
0 1 1/4 1
1 0 1/4 1
1 1 1/4 2

X , Y ∼ iid Unif{0, 1}, Z = X + Y
x y fXY (x , y ) z
0 0 1/4 0
0 1 1/4 1
1 0 1/4 1
1 1 1/4 2
Z ∈ {0, 1, 2}

X , Y ∼ iid Unif{0, 1}, Z = X + Y
x y fXY (x , y ) z
0 0 1/4 0
0 1 1/4 1
1 0 1/4 1
1 1 1/4 2
Z ∈ {0, 1, 2}
P(Z = 0) = 1/4, P(Z = 1) = 1/2, P(Z = 2) = 1/4

Small examples and table method: Max
Z = max(X , Y )
x y fXY (x , y ) z
0 0 1/2 0
0 1 1/4 1
0 2 1/8 2
1 0 1/16 1
1 1 1/32 1
1 2 1/32 2

Small examples and table method: Max
Z = max(X , Y )
x y fXY (x , y ) z
0 0 1/2 0
0 1 1/4 1
0 2 1/8 2
1 0 1/16 1
1 1 1/32 1
1 2 1/32 2
P(Z = 0) = 1/2, P(Z = 1) = 11/32, P(Z = 2) = 5/32

Moderate size examples: Too cumbersome
Pair of fair dice are thrown. What is the distribution of the sum or max
or min?

or min?
Table method
I 36 possible values
I Prone to errors

or min?
Table method
I 36 possible values
I Prone to errors
What if size becomes 100?

Visualizing functions of 2 variables
g(x , y ): function
I 3D plot is one option, but often not very useful
Contours I: values of (x , y ) that result in g(x , y ) = c

I Make a plot of those (x , y ) for different c
Contours II: values of (x , y ) that result in g(x , y ) ≤ c

I Make a plot of those (x , y ) for different c
Contours are quite useful in many problems

Examples
Sum function, g(x , y ) = x + y

Examples
Sum function, g(x , y ) = x + y
2 x+
y=
2
0 x
4 2 0 +y= 2 4
0
2 x+
y=
2
4
Click here for desmos illustration and other examples

Function of two random variables
X , Y ∼ fXY , X ∈ X , Y ∈ Y
Let Z = g(X , Y ) be a function of X and Y

What is the PMF of Z ?

X , Y ∼ fXY , X ∈ X , Y ∈ Y

Step 1: Find range of Z
I All possible values for g(x , y ), x ∈ X , y ∈ Y

X , Y ∼ fXY , X ∈ X , Y ∈ Y

I All possible values for g(x , y ), x ∈ X , y ∈ Y
Step 2: Add over the contours

I Suppose z is a possible value taken by Z
X
P(Z = z) = fXY (x , y )
(x ,y ):g(x ,y )=z

Example: Pair of dice, Sum
X , Y ∼ iid Unif{1, 2, 3, 4, 5, 6}, Z = X + Y

X , Y ∼ iid Unif{1, 2, 3, 4, 5, 6}, Z = X + Y

I Z ∈ {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}

X , Y ∼ iid Unif{1, 2, 3, 4, 5, 6}, Z = X + Y

I Z ∈ {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}
z 2 3 4 5 6 7
P(Z = z) 1/36 2/36 3/36 4/36 5/36 6/36
z 8 9 10 11 12
P(Z = z) 5/36 4/36 3/36 2/36 1/36

Example: Pair of dice, Max
X , Y ∼ iid Unif{1, 2, 3, 4, 5, 6}, Z = max(X , Y )


I Z ∈ {1, 2, 3, 4, 5, 6}


I Z ∈ {1, 2, 3, 4, 5, 6}
z 1 2 3 4 5 6
P(Z = z) 1/36 3/36 5/36 7/36 9/36 11/36

iid Uniform{1, . . . , n}: Sum
X , Y ∼ iid Unif{1, 2, . . . , n}, W = X + Y

X , Y ∼ iid Unif{1, 2, . . . , n}, W = X + Y
W ∈ {2, . . . , 2n}
w −1
(
n2
, 2 ≤ w ≤ n + 1,
P(W = w ) = 2n−w +1
n2
, n + 2 ≤ w ≤ 2n.

X , Y ∼ iid Unif{1, 2, . . . , n}, W = X + Y
W ∈ {2, . . . , 2n}
w −1
(
n2
, 2 ≤ w ≤ n + 1,
P(W = w ) = 2n−w +1
n2
, n + 2 ≤ w ≤ 2n.
P(X = x) P(Y = y) P(W = w)

0.10
0.08
0.06
0.04
0.02
0.00
0 5 10 0 5 10 0 10 20
Figure 1: P(W = w ) vs w , n = 10

iid Uniform{1, . . . , n}: Max
X , Y ∼ iid Unif{1, 2, . . . , n}, Z = max(X , Y )

Z ∈ {1, . . . , n}
2z − 1
P(Z = z) =
n2

Z ∈ {1, . . . , n}
2z − 1
P(Z = z) =
n2
P(X = x) P(Y = y) P(Z = z)
0.15
0.10
0.05
0.00
0 5 10 0 5 10 0 5 10
Figure 2: P(Z = z) vs z, n = 10

Section 4
Functions of random variables

Example: Throw a die twice
A fair die is thrown twice. What is the probability that the sum of the two
numbers seen is 6? What is the PMF of the sum?

Example: Area of a random rectangle
The length of a rectangle L ∼ Uniform{5, 7, 9, 11}. Given L = l, the
breadth B ∼ Uniform{l − 1, l − 2, l − 3}.

PMF of g(X1 , . . . , Xn )
Definition (PMF of a g(X1 , . . . , Xn ))

Suppose X1 , . . . , Xn have joint PMF fX1 ···Xn with TXi denoting the range of
Xi . Let g : TX1 × · · · × TXn → R be a function with range Tg . The PMF of
X = g(X1 , . . . , Xn ) is given by
X
fX (t) = P(g(X1 , . . . , Xn ) = t) = fX1 ···Xn (t1 , . . . , tn ).
(t1 ,...,tn ):g(t1 ,...,tn )=t
Proof: write the event and use definition of joint PMF

Directly useful for small problems
Can be extended for joint PMF of two functions g and h

Example: Given a small table for joint PMF
t1 t2 t3 fX1 X2 X3 (t1 , t2 , t3 )
0 0 0 1/9
0 0 1 1/9
0 0 2 1/9
0 1 1 1/9
0 1 2 1/9
1 0 0 1/9
1 0 2 1/9
1 1 0 1/9
1 1 1 1/9

Working

Example: Binomial from Bernoulli trials
Let X1 , . . . , Xn be the results of n i.i.d. Bernoulli(p) trials. The sum of the
n random variables X1 + · · · + Xn is Binomial(n, p).

Example: Sum of two uniforms
Let X ∼ Uniform{0, 1, 2, 3} and Y ∼ Uniform{0, 1, 2, 3} be independent.
Find the PMF of Z = X + Y .

Sums of two random variables taking integer values
Suppose X and Y take integer values and let their joint PMF be fXY . Let
Z = X +Y.

Z = X +Y.
Let z be some integer.
P(Z = z) = P(X + Y = z)
∞
X
= P(X = x , Y = z − x )
x =−∞
X∞
= fXY (x , z − x )
x =−∞
X∞
= fXY (z − y , y )
y =−∞

Z = X +Y.
Let z be some integer.
P(Z = z) = P(X + Y = z)
∞
X
= P(X = x , Y = z − x )
x =−∞
X∞
= fXY (x , z − x )
x =−∞
X∞
= fXY (z − y , y )
y =−∞
Convolution: If X and Y are independent,

∞
X
fX +Y (z) = fX (x )fY (z − x )
x =−∞
Sum of two independent Poissons
Let X ∼ Poisson(λ1 ) and Y ∼ Poisson(λ2 ) be independent.
1 Find the PMF of Z = X + Y .
2 Find the conditional distribution of X |Z .

Functions and independence
If X and Y are independent, g(X ) and h(Y ) are independent for any
two functions g and h

If X1 , X2 , X3 , X4 are mutually independent,

I g(X1 , X2 ) is independent of h(X3 , X4 )

I g(X1 , X2 , X3 ) is independent of h(X4 )

I g(X1 , X2 , X3 ) is independent of h(X4 )
Functions of non-overlapping sets of independent random variables are

also independent.

Exercises
Sum of independent Binomial(m, p) and Binomial(n, p)

Sum of independent Geometric(p) and Geometric(q)
Sum of r i.i.d. Geometric(p)
Sum of independent Negative-Binomial(r , p) and
Negative-Binomial(s, p)

Minimum of two random variables
X , Y ∼ fXY
Z = min(X , Y ): function of X and Y
Ex: Throw a die twice. Z = minimum of the two numbers seen.

X , Y ∼ fXY
fZ (z) = P(min(X , Y ) = z)
= P((X = z and Y = z) or (X = z and Y > z)
or (X > z and Y = z))
X X
= fXY (z, z) + fXY (z, t2 ) + fXY (t1 , z)
t2 >z t1 >z

X , Y ∼ fXY
fZ (z) = P(min(X , Y ) = z)
= P((X = z and Y = z) or (X = z and Y > z)
or (X > z and Y = z))
X X
= fXY (z, z) + fXY (z, t2 ) + fXY (t1 , z)
t2 >z t1 >z
What about maximum?

Example: Throw a die twice
Throw a die twice. Z = minimum of the two numbers seen.
1 2 3 4 5 6
1 1/36 1/36 1/36 1/36 1/36 1/36
2 1/36 1/36 1/36 1/36 1/36 1/36
3 1/36 1/36 1/36 1/36 1/36 1/36
4 1/36 1/36 1/36 1/36 1/36 1/36
5 1/36 1/36 1/36 1/36 1/36 1/36
6 1/36 1/36 1/36 1/36 1/36 1/36

Independent case: cumulative distribution function (CDF)
of maximum
Definition (CDF of a random variable)
Cumulative distribution function of a random variable X is a function
FX : R → [0, 1] defined as
FX (x ) = P(X ≤ x ).

of maximum
FX (x ) = P(X ≤ x ).
Suppose X and Y are independent and Z = max(X , Y ).
FZ (z) = P(max(X , Y ) ≤ z)
= P((X ≤ z) and (Y ≤ z))
= P(X ≤ z)P(Y ≤ y )
= FX (z)FY (z)

of maximum
FX (x ) = P(X ≤ x ).
Suppose X and Y are independent and Z = max(X , Y ).
FZ (z) = P(max(X , Y ) ≤ z)
= P((X ≤ z) and (Y ≤ z))
= P(X ≤ z)P(Y ≤ y )
= FX (z)FY (z)
Independent: CDF of maximum is product of CDFs. What about min?

Problem: Min and max of i.i.d. sequences
Let X1 , . . . , Xn ∼ i.i.d. X . Find the distribution of the following:
1 min(X1 , . . . , Xn )
2 max(X1 , . . . , Xn )

Problem: Min of two independent Geometrics
Let X ∼ Geometric(p) and Y ∼ Geometric(p) be independent. Find the
distribution of min(X , Y ).

Week2 Annotated Final

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Week2 Annotated Final

Uploaded by

Copyright:

Available Formats

Independence of two random variables

Definition (Independence (two random variables))

General case vs independent case

Week 3: Multiple Random Variables 43 / 48

Week 3: Multiple Random Variables 44 / 48

Week 3: Multiple Random Variables 44 / 48

Week 3: Multiple Random Variables 44 / 48

Week 3: Multiple Random Variables 45 / 48

Week 3: Multiple Random Variables 45 / 48

Week 3: Multiple Random Variables 45 / 48

Week 3: Multiple Random Variables 45 / 48

To show X and Y are independent, verify

fXY (t1 , t2 ) = fX (t1 )fY (t2 )

fXY (t1 , t2 ) 6= fX (t1 )fY (t2 )

for some t1 ∈ TX and t2 ∈ TY .

Week 3: Multiple Random Variables 46 / 48

Week 3: Multiple Random Variables 47 / 48

Week 3: Multiple Random Variables 48 / 48

Definition (Independence of random variables)

fX1 ···Xn (t1 , . . . , t2 ) = fX1 (t1 )fX2 (t2 ) · · · fXn (tn )

for all ti ∈ TXi .

If X and Y are independent

Week 3: Multiple Random Variables 50 / 74

Let Xi = 1 if i-th toss is heads and Xi = 0 if i-th toss is tails, i = 1, 2, 3.

Week 3: Multiple Random Variables 51 / 74

Let Xi = 1 if i-th toss is heads and Xi = 0 if i-th toss is tails, i = 1, 2, 3.

fX1 X2 X3 (t1 , t2 , t3 ) = fX1 (t1 )fX2 (t2 )fX3 (t3 )

Week 3: Multiple Random Variables 51 / 74

Let Xi = 1 if i-th toss is heads and Xi = 0 if i-th toss is tails, i = 1, 2, 3.

fX1 X2 X3 (t1 , t2 , t3 ) = fX1 (t1 )fX2 (t2 )fX3 (t3 )

Week 3: Multiple Random Variables 51 / 74

Week 3: Multiple Random Variables 52 / 74

Week 3: Multiple Random Variables 52 / 74

X , Y , Z : Dependent random variables

Week 3: Multiple Random Variables 52 / 74

Number of 1s in (X1 , X2 , X3 ) is even (hence, the name)

Week 3: Multiple Random Variables 53 / 74

Number of 1s in (X1 , X2 , X3 ) is even (hence, the name)

Week 3: Multiple Random Variables 53 / 74

Number of 1s in (X1 , X2 , X3 ) is even (hence, the name)

Week 3: Multiple Random Variables 53 / 74

2 the marginal PMFs f

Week 3: Multiple Random Variables 54 / 74

2 the marginal PMFs f

Repeated trials of an experiment creates i.i.d. sequence of random

Week 3: Multiple Random Variables 54 / 74

2 the marginal PMFs f

Repeated trials of an experiment creates i.i.d. sequence of random

Notation: X1 , X2 , . . . , Xn are i.i.d. with distribution X means that fXi is the

Week 3: Multiple Random Variables 54 / 74

Week 3: Multiple Random Variables 55 / 74

Week 3: Multiple Random Variables 56 / 74

Week 3: Multiple Random Variables 57 / 74

Andrew Thangaraj (IIT Madras) Visualizing functions of random variables 1 / 28

One random variable, one-to-one functions

Andrew Thangaraj (IIT Madras) Visualizing functions of random variables 2 / 28

Andrew Thangaraj (IIT Madras) Visualizing functions of random variables 4 / 28

Andrew Thangaraj (IIT Madras) Visualizing functions of random variables 7 / 28

One-to-one: For different input, the function results in different outputs

Andrew Thangaraj (IIT Madras) Visualizing functions of random variables 10 / 28

One random variable, many-to-one functions

Andrew Thangaraj (IIT Madras) Visualizing functions of random variables 11 / 28

Andrew Thangaraj (IIT Madras) Visualizing functions of random variables 12 / 28