Outline

1 Preliminaries in Probability

2 Discrete Random Variables

3 Joint Distribution

4 Known Distributions

5 (a, b, 0) Class

6 Constructing New Distributions

c Bin Zou MATH 5639 14/64

 Discrete Random Variables

Definition 7
Let X be a r.v. defined on some probability space (⌦, F, P).
X is called a discrete random variable if set R(X) := {X(!) : ! 2 ⌦}
has up to countably many elements.
A discrete r.v. X is uniquely determined by its probability function
(pf ) fX
fX (x) = P(X = x), 8 x 2 R(X)

Immediately, for a discrete r.v. X with pf fX , we obtain

X
FX (x) = fX (y), 8y 2 R
y2R(X), yx

The cdf of a discrete r.v. is a step function growing from 0 to 1, which

only jumps at points in R(X).

c Bin Zou MATH 5639 15/64

 Theorem 2
A function f is a pf if and only if
X
(1) 0f 1 and (2) f (x) = 1
x

Example 5
Suppose X can only take finitely many values, R(X) = {x1 , · · · , xn },
where n is a positive integer (or infinite) and x1 < · · · < xn .
Any sequence (pi )i=1,··· ,n is a pf of X if
n
X
0 < pi < 1, i = 1, · · · , n and pi = 1
i=1

c Bin Zou MATH 5639 16/64

 Moments

Definition 8
Let X be a discrete r.v., which takes values in set R(X) and has pf
fX (·). We define the n-th moment (raw moment) of X by
X
n
E[X ] = xn · fX (x)
x2R(X)

Remark: n = 1, expected value (or expectation) of X, µX := E[X]

We define the n-th central moment of X by

X
n
E [(X µX ) ] = (x µX )n · fX (x)
x2R(X)

Remark: The 2nd central moment of X is called the variance of X,

denoted by V(X) = X2
( X > 0 is the standard deviation of X).

V(X) = E[X 2 ] (E[X])2

c Bin Zou MATH 5639 17/64

 Moment Generating Function

Definition 9
The moment generating function (mgf ) of a r.v. X is defined by
⇥ t X⇤
MX (t) : = E e , t2R

which reads as
X
MX (t) = etx · fX (x)
x2R(X)

for a discrete r.v. with pf fX (·).

For some distributions, we need to restrict t to a subset of R, on

which MX (t) is well defined.

c Bin Zou MATH 5639 18/64

 Properties of mgf

It is straightforward to verify that

0
⇥ ⇤ 0
MX (t) = E X · et·X ) MX (0) = E[X]
00
⇥ ⇤ 00
MX (t) = E X 2 · et·X ) MX (0) = E[X 2 ]

Do you see a pattern?

(n) ⇥ ⇤ (n)
MX (t) = E X n · et·X ) MX (0) = E[X n ]

c Bin Zou MATH 5639 19/64

 Probability Generating Function
Definition 10
The probability generating function (pgf ) of a discrete r.v. X taking
only nonnegative integer values, denoted by PX (·), is defined by
1
X
⇥ X
⇤
PX (t) := E t = tx · fX (x)
x=0

if the expectation on the r.h.s exists.

I It is obvious that
MX (t) = PX (et ) and PX (t) = MX (ln(t))
I The n-th order derivative of pgf PX is given by
1
X
(n)
PX (t) = x(x 1) · · · (x n + 1) · tx n
· fX (x)
x=n

Hence, we obtain
(n)
PX (0) = n! · fX (n), 8 n 2 N+

c Bin Zou MATH 5639 20/64

 Continuous Random Variables
Definition 11
Let X be a r.v. defined on some probability space (⌦, F, P).
X is called a continuous random variable if there exists a function fX
such that
Z x
FX (x) = P(X  x) = fX (x) dx, 8x 2 R
1

The function fX is called the probability density function (pdf) of X,

and is the derivative of the cdf FX , i.e.,

dFX (x)
= fX (x)
dx
I FX is non-decreasing , fX 0.
R1
I FX (+1) = 1 , f (x) dx = 1.
1 X

Remark. We can define moments and mgf similarly for continuous

r.v.’s, with summation changing to integral.
c Bin Zou MATH 5639 21/64