Are you sure?
This action might not be possible to undo. Are you sure you want to continue?
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Gov2000: Quantitative Methodology for
Political Science I
Lecture 2: Basic Probability, Random Variables, and some
Elementary Asymptotics
September 24, 2007
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Outline
1
Deﬁnitions and Notation
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
2
Random Variables and Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
3
Expectation and Transformations
Expectation and Variance
Conditional Expectation and Variance
4
Elementary Asymptotics
Convergence of a Sequence
Convergence in Probability
Convergence in Distribution
5
Some Important Distributions
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Intuitive Deﬁnition
While there are several interpretations of what probability is,
most modern (post 1935 or so) researchers agree on an
axiomatic deﬁnition of probability.
3 Axioms (Intuitive Version):
1
The probability of any particular event must be
nonnegative.
2
The probability of anything occurring among all possible
events must be 1.
3
The probability of one of many mutually exclusive events
happening is the sum of the individual probabilities.
The rules of probability can be derived from these axioms.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Subjective Interpretation
Probability is a subjective belief about the likelihood of an event.
Example 1: The probability of drawing 5 red cards out of 10
drawn from a deck of cards is whatever you want it to be.
Example 2: The probability of state failure among partial
democracies is whatever you want it to be.
But...
1
If you don’t follow the three axioms, a smart bookie can set
up a Dutch book against you.
2
There is a correct way to update your beliefs once you
collect evidence (data).
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Frequency Interpretation
Suppose some process can produce different events (e.g. coin
ﬂip).
Probability of is the relative frequency with which an event
would occur if the process were repeated a large number of
times under similar conditions.
Example 1: The probability of drawing 5 red cards out of
10 drawn from a deck of cards is the frequency with which
this event occurs in repeated samples of 10 cards.
Example 2: The probability of state failure among partial
democracies is the ...
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
If you want to explore this debate further, check out this article
in the Stanford Encyclopedia of Philosophy.
http://plato.stanford.edu/entries/probabilityinterpret/
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Basic Set Theoretic Notation
Let A denote a set. If a is a member of A we write a ∈ A.
If a
1
, a
2
, and a
3
are the members of A, we write
A = {a
1
, a
2
, a
3
}.
The empty set ∅ is the set with no members.
If A is a subset of B we write A ⊂ B.
For example, if A = {red, blue} and B = {red, blue, green},
then A ⊂ B.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
The intersection of two sets A and B is the set containing all
elements that belong to both sets. We write the intersection of
A and B as A ∩ B.
For example, if A = {red, blue} and B = {blue, green}, then
A ∩ B = {blue}
The union of two sets A and B is the set that contains the
intersection of A and B, the elements in A that aren’t in B and
the elements of B that aren’t in A.
For example, if A = {red, blue} and B = {blue, green}, then
A ∪ B = {red, blue, green}
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Sample Spaces
The sample space is the set of all possible outcomes, and is
often written as Ω.
For example, if we ﬂip a coin twice, there are four possible
outcomes,
Ω =
¸
{heads, heads}, {heads, tails}, {tails, heads}, {tails, tails}
¸
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Events
Events are subsets of the sample space.
For Example, if
Ω =
¸
{heads, heads}, {heads, tails}, {tails, heads}, {tails, tails}
¸
,
then
∅
¸
{heads, heads}, {heads, tails}, {tails, tails}
¸
{heads, tails}
are all events.
If A is an event, then "everything else" in the sample space is
called the compliment of A, and is written as A
c
.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Probability Function
A probability function P(·) is a function deﬁned over all subsets
of a sample space Ω and that satisﬁes the three axioms:
1
P(A) ≥ 0 for all A in the set of all events.
2
P(Ω) = 1
3
if events A
1
, A
2
, . . . are mutually exclusive then
P(
¸
∞
i =1
A
i
) =
¸
∞
i =1
P(A
i
).
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Marginal and Joint Probability
So far we have only considered situations where we are
interested in the probability of a single event A occurring. We’ve
denoted this P(A). P(A) is sometimes called a marginal
probability.
Suppose we are now in a situation where we would like to
express the probability that an event A and an event B occur.
This quantity is written as P(A ∩ B), P(B ∩ A), P(A, B), or
P(B, A) and is the joint probability of A and B.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Conditional Probability
If P(B) > 0 then the probability of A conditional on B can be
written as
P(AB) =
P(A, B)
P(B)
This implies that
P(A, B) = P(B) ×P(AB)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
For example, if we randomly draw two cards from a standard 52
card deck and deﬁne the events A = {King on Draw 1} and
B = {King on Draw 2}, then
P(A) = 4/52
P(BA) = 3/51
P(A, B) = P(A) ×P(BA) = 4/52 ×3/51
Question: P(B) =?
a) 3/51
b) 4/52
c) 4/51
d) not enough information
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Law of Total Probability (LTP)
With 2 Events:
P(B) = P(B, A) + P(B, A
c
)
= P(BA) ×P(A) + P(BA
c
) ×P(A
c
)
In general, if {C
n
: n = 1, 2, 3, . . . } forms a partition of the
sample space, then
P(B) =
¸
n
P(B ∩ C
n
)
=
¸
n
P(BC
n
) ×P(C
n
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Conﬁrming Intuition with the LTP
P(B) = P(BA) + P(BA
c
)
= P(BA) ×P(A) + P(BA
c
) ×P(A
c
)
P(B) = 3/51 ×1/13 + 4/51 ×12/13
=
3 + 48
51 ×13
=
1
13
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Some other useful rules
P(A ∪ B) = P(A) + P(B) −P(A ∩ B)
Also, If P(A) > 0 and P(B) > 0, then we can write the following.
P(AB) = P(A)P(BA) = P(B)P(AB)
P(AB) =
P(A)P(BA)
P(B)
P(AB) =
P(A)P(BA)
P(BA) ×P(A) + P(BA
c
) ×P(A
c
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
False Positive Problem
Suppose we have a test for a rare disease (1/100,000) with the
following properties (shown through extensive trials):
P(+ test disease) = .999 (Sensitivity)
P(− test no disease) = .999 (Speciﬁcity)
Question: Suppose you receive a positive test, what is the
probability that you have the disease?
a) < 1/3
b) between 1/3 and 2/3
c) > 2/3
d) not enough information
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Answer to the False Positive Problem
P(dis. + test) =
P(+ test, dis.)
P(+ test)
=
P(+ test dis.) ×P(dis.)
P(+ test)
=
P(+ test dis.) ×P( dis.)
P(+testdis.) ×P(dis.) + P(+testno dis.) ×P(no dis.)
≈
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Independence
Intuitive Deﬁnition
Events A and B are independent if knowing whether A occurred
provides no information about whether B occurred.
Formal Deﬁnition
P(AB) = P(A)P(B) =⇒ A⊥⊥B
With all the usual > 0 restrictions, this implies
P(AB) = P(A)
P(BA) = P(B)
This type of independence is sometimes called “marginal”
independence.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Coins vs. Cards
A two coin ﬂip thought experiment provides a good example of
independence because the outcome from the ﬁrst ﬂip doesn’t
affect the outcome from the second ﬂip. If A = {Heads on ﬂip 1}
and B = {Heads on ﬂip 2}, then
P(A, B) = P(A) ×P(B)
Contrast this with our two card thought experiment. If
A = {King on Draw 1} and B = {King on Draw 2}, then
P(A, B) = P(A)P(BA) = 1/13 ×3/51 = P(A)P(B)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Conditional Independence
Intuitive Deﬁnition
Events A and B are conditionally independent given C, if
knowing whether C occurred and knowing whether A occurred
provides no information about whether B occurred.
Formal Deﬁnition
With P(C) > 0, we can write
P(A, BC) =
P(A, B, C)
P(C)
and we say that A is conditionally independent of B given C
(A⊥⊥BC) if
P(A, BC) = P(AC)P(BC)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Rain and Sprinklers
Suppose I ﬂip a coin every morning in the Summer. If it comes
up heads, I turn on my sprinkler. I never turn on my sprinkler in
Fall, Winter, and Spring.
Events:
A = {the sprinkler was on today}
B = {it rained today}
C = {it is Summer}
Question 1: Are A and B independent?
Question 2: Conditional on knowledge of C, are A and B
independent?
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is Probability?
Notation and Deﬁnitions
Marginal, Joint and Conditional Probability
Why is the grass wet?
Suppose I ﬂip a coin every morning. If it comes up heads, I turn
on my sprinkler. When I get home from work at night, I turn the
sprinkler off if it is on.
Events:
A = {the sprinkler was on today}
B = {it rained today}
C = {the grass is wet}
Question 1: Are A and B independent?
Question 2: Conditional on knowledge of C, are A and B
independent?
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
A random variable X is a function that maps the sample space
to the real numbers.
Returning to our previous example with
Ω =
¸
{heads, heads}, {heads, tails}, {tails, heads}, {tails, tails}
¸
we could deﬁne a random variable X(ω) to be the function that
returns the number of heads for each element of Ω.
X({heads, heads}) = 2
X({heads, tails}) = 1
X({tails, heads}) = 1
X({tails, tails}) = 0
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Discrete Distributions
For discrete distributions, the random variable X takes on a
ﬁnite, or a countably inﬁnite number of values.
Example 1: The number of Clinton supporters in a poll of
1,000 likely voters.
Example 2: The number of calls to the Clinton campaign
headquarters on a given day.
A common shorthand is to think of discrete RVs taking on
distinct values.
A probability mass function (pmf) and a cumulative
distribution function (cdf) are two common ways to deﬁne
the distribution for a discrete RV.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Discrete Probability Mass Functions
A probability mass function f (x) of a random variable X is a
nonnegative function that gives the probability that X = x and
¸
x
f (x) = 1.
For example, when X is the number of heads in two coin ﬂips,
f (x) =
1/4 x = 0
1/2 x = 1
1/4 x = 2
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
PMF Plot
q
q
q
−0.5 0.0 0.5 1.0 1.5 2.0 2.5
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
x
f
(
x
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Discrete Cumulative Distribution Function
A cumulative distribution function F(x) of a random variable X
is a nondecreasing function that gives the probability that
X ≤ x.
For example, when X is the number of heads in two coin ﬂips,
F(x) =
0 x < 0
1/4 0 ≤ x < 1
3/4 1 ≤ x < 2
1 2 ≤ x
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Discrete CDF Plot
q
q
q
−0.5 0.0 0.5 1.0 1.5 2.0 2.5
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
x
F
(
x
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Discrete CDF Question
Question: If X = “the number of heads in two coin ﬂips”, how
can you calculate the probability of X = 1 with the CDF?
a) F(1)
b) F(2)
c) F(1) −F(0)
d) F(2) −F(1)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Continuous Distributions
Continuous random variables take on an uncountably
inﬁnite number of values.
Example: SegalCover scores for US Supreme Court
justices
A probability density function (pdf) and a cumulative
distribution function (cdf) are two common ways to deﬁne
the distribution for a continuous RV.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Continuous Probability Density Function
The probability density function f (x) of a continuous random
variable X is the nonnegative function that satisﬁes
1
f (x) ≥ 0 for all x ∈ R
2
∞
−∞
f (x)dx = 1
For example
f (x) =
1/4 0 < x < 4
0 otherwise
f (x) =
1/4 0 ≤ x ≤ 4
0 otherwise
Think of densities as inﬁnite data histograms.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
0 1 2 3 4
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
x
f
(
x
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Continuous Cumulative Distribution Functions
A cumulative distribution function F(x) of a random variable X
is a nondecreasing function that gives the probability that
X ≤ x. However, for a continuous RV, the cdf is continuous.
F(x) =
x
−∞
f (z)dz
For example,
F(x) =
0 x < 0
x/4 0 ≤ x < 4
1 4 ≤ x
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Continuous CDF Plot
0 1 2 3 4
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
x
F
(
x
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Continuous Probability Questions
For the continuous distribution, described by the following pdf
f (x) =
1/4 0 < x < 4
0 otherwise
Question 1: What is the probability that X = 3?
a) 0
b) 1/4
c) 3/4
Question 2: What is the probability that 1 < X < 3?
a) 1/4
b) 2/4
c) 3/4
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Marginal, Joint, and Conditional Distributions
Just as marginal, joint, and conditional probabilities can be
deﬁned for two arbitrary events A and B; marginal, joint, and
conditional probability distributions can be deﬁned for two
random variables X and Y.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Discrete Joint Distributions
The joint mass function f
X,Y
(x, y) of two discrete random
variables X and Y is the function that gives the probability that
X = x and Y = y for all x and y.
Example:
Y
1 2 3
1 0.22 0.04 0.09 0.35
X 2 0.15 0.10 0.20 0.45
3 0.01 0.07 0.12 0.20
0.38 0.21 0.41 1.00
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Continuous Joint Distributions
The joint density function f
X,Y
(x, y) of two continuous random
variables X and Y is the function that gives the density height
where X = x and Y = y for all x and y.
0.0 0.2 0.4 0.6 0.8 1.0
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
x
y
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Continuous Joint Distributions
The joint density function f
X,Y
(x, y) of two discrete random
variables X and Y is the function that gives the density height
where X = x and Y = y for all x and y.
x
y
f
(
x
,
y
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Discrete Marginal Distributions
The marginal mass function f
X
(x) of a discrete random variable
X gives the probability that X = x for all x, and can be
calculated from the joint probability function f
X,Y
(x, y) of X and
Y according to
f
X
(x) =
¸
y
f
X,Y
(x, y).
Y
1 2 3
1 0.22 0.04 0.09 0.35
X 2 0.15 0.10 0.20 0.45
3 0.01 0.07 0.12 0.20
0.38 0.21 0.41 1.00
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Continuous Marginal Distributions
The marginal density function f
X
(x) of a continuous random
variable X gives the density height that X = x for all x, and can
be calculated from the joint density function f
X,Y
(x, y) of X and
Y according to
f
X
(x) =
∞
−∞
f
X,Y
(x, y)dy.
x
y
f
(
x
,
y
)
0.0 0.2 0.4 0.6 0.8 1.0
0
.
3
6
0
.
3
7
0
.
3
8
0
.
3
9
0
.
4
0
x
f
(
x
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Conditional Discrete Distributions
The conditional mass function f
XY
(xy) of two discrete random
variables gives the probability that X = x given the fact that
Y = y for all all values of x and y and is given by:
f
XY
(xy) =
f
X,Y
(x, y)
f
Y
(y)
where it is assumed that f
Y
(y) > 0. It follows that
f
X,Y
(x, y) = f
XY
(xy)f
Y
(y),
f
Y
(y) =
f
X,Y
(x, y)
f
XY
(xy)
.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Table: Joint and Marginal Probabilities
Y
1 2 3
1 0.22 0.04 0.09 0.35
X 2 0.15 0.10 0.20 0.45
3 0.01 0.07 0.12 0.20
0.38 0.21 0.41 1.00
Table: Conditional f (xy) Probabilities
Y
1 2 3
1 0.58 0.19 0.22
X 2 0.39 0.48 0.49
3 0.03 0.33 0.29
1.00 1.00 1.00
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Conditional Continuous Distributions
The conditional density function f
YX
(yx) when Y is a
continuous random variable gives the density height for Y = y
given the fact that X = x for all all values of x and y and is
given by:
f
YX
(yx) =
f
Y,X
(y, x)
f
X
(x)
where it is assumed that f
X
(x) > 0.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Conditional Continuous Distributions
0.0 0.2 0.4 0.6 0.8 1.0
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
Joint Density
x
y
0.0 0.2 0.4 0.6 0.8 1.0
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
Conditional Density
x
y
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Conditional Continuous Distributions
x
y
f
(
x
,
y
)
Joint Density
x
y
f
(
y

x
)
Conditional Density
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
What is a Random Variable?
Discrete and Continuous Distributions
Marginal, Joint, and Conditional Distributions
Conditional Densities Discrete X
−3 −2 −1 0 1 2 3 4
0
.
0
0
.
2
0
.
4
Marginal Density
y
f
(
y
)
−3 −2 −1 0 1 2 3 4
0
.
0
0
.
2
0
.
4
Conditional Density X=−1
y
f
(
y

x
)
−3 −2 −1 0 1 2 3 4
0
.
0
0
.
2
0
.
4
Conditional Density X=2
y
f
(
y

x
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Expectation
The expected value of a random variable X is denoted by E[X]
and is a measure of central tendency of X. Roughly speaking,
an expected value is like a weighted average.
The expected value of a discrete random variable X is deﬁned
as
E[X] =
¸
all x
xf
X
(x).
The expected value of a continuous random variable X is
deﬁned as
E[X] =
∞
−∞
xf
X
(x)dx.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
An example will make this more clear. Suppose X is a discrete
random variable that can take values of 0, 1, and 2. The
probability function of X is given by:
f
X
(x) =
0.20 if x = 0
0.45 if x = 1
0.35 if x = 2
The expected value of X is:
E[X] = 0 ×f
X
(0) + 1 ×f
X
(1) + 2 ×f
X
(2)
= 0 ×0.20 + 1 ×0.45 + 2 ×0.35
= 1.15
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Interpreting Discrete Expected Value
The expected value for a discrete random variable is the
balance point of the mass function.
q
q
q
−0.5 0.0 0.5 1.0 1.5 2.0 2.5
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
x
f
(
x
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Interpreting Continuous Expected Value
The expected value for a continuous random variable is the
balance point of the density function.
0 2 4 6 8 10 12
0
.
0
0
0
.
0
5
0
.
1
0
0
.
1
5
x
f
(
x
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Sample Mean as an Expected Value
Let x
1
, . . . , x
n
be our sample. Then the sample mean is deﬁned
as the following
¯
x =
1
n
n
¸
i =1
x
i
This can be rewritten in the following form:
¯
x =
n
¸
i =1
x
i
·
1
n
Note how this resembles the deﬁnition of discrete expected
value.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Example
2 3 4 5 6
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
2
.
5
3
.
0
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Example
2 3 4 5 6 7
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
2
.
5
3
.
0
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Example
2 3 4 5 6 7
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
2
.
5
3
.
0
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Why the balance point?
It is a reasonable measure for the “center” of the data.
We have some intuition about balance points.
The sample balance point has properties that are easy to
describe. (It is a linear combination of the data.)
It is most accurate in a certain sense (next week).
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Useful Properties of Expected Values
Suppose we have k random variables X
1
, . . . , X
k
. If E[X
i
] exists
for all i = 1, . . . , k, then
E
¸
k
¸
i =1
X
i
¸
= E[X
1
] +· · · + E[X
k
]
If two random variables X and Y are independent and have
ﬁnite expectations then
E[XY] = E[X]E[Y]
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Suppose a and b are constants and X is a random variable.
Then
E[aX] = aE[X]
E[b] = b
E[aX + b] = aE[X] + b
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Expectation Question
Question: If X
1
, . . . , X
n
are random variables with
E[X
1
] = µ, ..., E[X
n
] = µ, what is the expected value of
X
n
=
1
n
(X
1
+. . . + X
n
)?
a)
µ
n
b) nµ
c) µ
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Variance
The expected value of a function of the random variable X
(g(X))is denoted by E[g(X)] and is a measure of central
tendency of g(X).
The variance is a special case of this and the variance of a
random variable X (a measure of its dispersion) is given by
V[X] = E[(X −E[X])
2
]
= E[X
2
−2E[X]X + E[X]
2
]
= E[X
2
] −2E[X]
2
+ E[X]
2
= E[X
2
] −E[X]
2
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
For a discrete random variable X
V[X] =
¸
all x
(x −E[X])
2
f
X
(x)
For a continuous random variable X
V[X] =
∞
−∞
(x −E[X])
2
f
X
(x)dx
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Physical Interpretation of Variance
−6 −2 2 4 6
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
x
f
(
x
)
−6 −2 2 4 6
0
.
0
0
0
.
0
5
0
.
1
0
0
.
1
5
0
.
2
0
x
f
(
x
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Sample Variance
The sample variance is usually written in one of two ways:
1
1
n
¸
n
i =1
(x
i
−
¯
x)
2
2
1
n−1
¸
n
i =1
(x
i
−
¯
x)
2
The ﬁrst option can be rewritten in the following form.
n
¸
i =1
(x
i
−
¯
x)
2
(
1
n
)
Notice how this relates to the discrete deﬁnition of variance.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Physical Interpretation of Sample Variance
2 3 4 5 6
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
2
.
5
3
.
0
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Physical Interpretation of Sample Variance
2 3 4 5 6
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
2
.
5
3
.
0
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Physical Interpretation of Sample Variance
2 3 4 5 6
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
2
.
5
3
.
0
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Useful Properties of Variances
If X
1
, . . . , X
n
are independent random variables and c
1
, . . . , c
n+1
are arbitrary constants then
V[c
1
X
1
+· · · + c
n
X
n
+ c
n+1
] = c
2
1
V[X
1
] +· · · + c
2
n
V[X
n
]
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Variance Question
Question: If X
1
, . . . , X
n
are i.i.d. random variables with
V[X
1
] = σ
2
, ..., V[X
n
] = σ
2
, what is the variance of
X
n
=
1
n
(X
1
+. . . + X
n
)?
a)
σ
2
n
b) nσ
2
c) σ
2
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Conditional Expectation
The concept of conditional expectation is fundamental to
regression analysis.
Suppose we have two RVs X and Y that have some bivariate
distribution.
The conditional expectation of Y given X = x (denoted E[Yx])
is the expected value of Y under the conditional distribution of
Y given X = x.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
In the discrete case:
E[Yx] =
¸
y
yf
YX
(yx)
In the continuous case:
E[Yx] =
∞
−∞
yf
YX
(yx)dy
Similar deﬁnitions apply to the case of multiple conditioning
variables.
E[Yx] is a function of x (realized values of X) and can be
interpreted as the balance point for the conditional distribution.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Conditional Expectation  X discrete
−3 −2 −1 0 1 2 3 4
0
.
0
0
.
2
0
.
4
Marginal Density
y
f
(
y
)
−3 −2 −1 0 1 2 3 4
0
.
0
0
.
2
0
.
4
Conditional Density X=−1
y
f
(
y

x
)
−3 −2 −1 0 1 2 3 4
0
.
0
0
.
2
0
.
4
Conditional Density X=2
y
f
(
y

x
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Conditional Expectation  X continuous
0.0 0.2 0.4 0.6 0.8 1.0
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
E[X],E[Y]
x
y q
0.0 0.2 0.4 0.6 0.8 1.0
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
E[YX]
x
y
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Conditional Variance
Likewise, we can deﬁne the conditional variance of Y given
X = x (denoted V[Yx]) to be the variance of Y under the
conditional distribution of Y given X = x.
In the discrete case:
V[Yx] =
¸
y
(y −E[Yx])
2
f
YX
(yx)
In the continuous case:
V[Yx] =
∞
−∞
(y −E[Yx])
2
f
YX
(yx)dy
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Expectation and Variance
Conditional Expectation and Variance
Conditional Variance  X discrete
−3 −2 −1 0 1 2 3 4
0
.
0
0
.
2
0
.
4
Marginal Density
y
f
(
y
)
−3 −2 −1 0 1 2 3 4
0
.
0
0
.
2
0
.
4
Conditional Density X=−1
y
f
(
y

x
)
−3 −2 −1 0 1 2 3 4
0
.
0
0
.
2
0
.
4
Conditional Density X=2
y
f
(
y

x
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Convergence of a Sequence
Convergence in Probability
Convergence in Distribution
Deﬁnition: Convergent Sequences of Real Numbers
A sequence of real numbers c
n
is said to converge to c if for
every > 0 there exists an integer N such that for n ≥ N,
c
n
−c < .
We will write this as
c
n
→c
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Convergence of a Sequence
Convergence in Probability
Convergence in Distribution
Example
If c
n
is 1 + 1/n, then c
n
→1.
0 20 40 60 80 100
1
.
0
1
.
2
1
.
4
1
.
6
1
.
8
2
.
0
n
c
n
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Convergence of a Sequence
Convergence in Probability
Convergence in Distribution
Deﬁnition: Convergence in Probability
We say that a sequence of random variables X
n
converges in
probability to a real number θ if for every > 0
P(X
n
−θ > ) →0 as n →∞
We will write this as
X
n
→
p
θ
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Convergence of a Sequence
Convergence in Probability
Convergence in Distribution
Example: The Weak Law of Large Numbers
If X
1
, X
2
, . . . , X
n
, . . . are i.i.d. with −∞< E[X
1
] = µ < ∞, then
X
n
→
p
µ
0 1 2 3 4
0
.
0
5
0
.
1
0
0
.
1
5
0
.
2
0
0
.
2
5
0
.
3
0
0
.
3
5
0
.
4
0
n = 1
X
n
f
(
X
n
)
0 1 2 3 4
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
1
.
2
n = 10
X
n
f
(
X
n
)
0 1 2 3 4
0
1
2
3
4
n = 100
X
n
f
(
X
n
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Convergence of a Sequence
Convergence in Probability
Convergence in Distribution
Convergence Question
Question: Does X
n
appear to be converging in probability to 2?
0 1 2 3 4
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
n = 1
X
n
f
(
X
n
)
0 1 2 3 4
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
1
.
2
n = 10
X
n
f
(
X
n
)
0 1 2 3 4
0
1
2
3
4
n = 100
X
n
f
(
X
n
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Convergence of a Sequence
Convergence in Probability
Convergence in Distribution
Deﬁnition: Convergence in Distribution
We say that a sequence of random variables X
n
converges in
distribution to a random variable X if the cumulative distribution
functions F
n
and F of X
n
and X satisfy the following
F
n
(x) →F(x) as n →∞for each continuity point x of F
We will write this as
X
n
→
d
X
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Convergence of a Sequence
Convergence in Probability
Convergence in Distribution
The Classical Central Limit Theorem
If X
1
, X
2
, . . . , X
n
, . . . are i.i.d. with E[X
1
] = µ and V[X
1
] = σ
2
and EX
2
< ∞, then
√
n(X
n
−µ) →
d
N(0, σ
2
)
0 2 4 6 8
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
0
.
5
n = 1
X
n
f
(
X
n
)
0 2 4 6 8
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
0
.
5
0
.
6
n = 10
X
n
f
(
X
n
)
0 2 4 6 8
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
n = 100
X
n
f
(
X
n
)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
The Univariate Normal Distribution
The univariate normal (Gaussian) probability density function is
given by
f
N
(xµ, σ
2
) =
1
√
2πσ
exp
−
1
2σ
2
(x −µ)
2
−4 −2 0 2 4
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
x
D
e
n
s
i
t
y
N(0,1)
N(2, 1)
N(0, .25)
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Some facts about the univariate normal distribution:
The normal distribution with mean 0 and variance 1 is
called the standard normal distribution
If a large random sample is taken from any distribution with
ﬁnite variance the sampling distribution of the sample
mean will be approximately normal
If a sample (X
1
, . . . , X
n
) of any size n is taken from a
normal distribution with known variance then the sampling
distribution of the sample mean will be normal with mean
E[X] and variance V[X]/n
A linear function of a normal RV is itself a normal RV
The R functions rnorm(), dnorm(), and pnorm()
calculate pseudorandom normal deviates, the normal
density function, and the normal distribution function
respectively.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
The Multivariate Normal Distribution
The dvariate normal density function is given by
f
N
(xµ, Σ) = (2π)
−d/2
Σ
−1/2
exp
−
1
2
(x −µ)
Σ
−1
(x −µ)
Here x and µ are vectors of length d and Σ is a d ×d
positivedeﬁnite matrix. The mean of x is µ and the
variancecovariance matrix of x is Σ.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
The ChiSquare Distribution
The chisquare probability density function is given by
f
χ
2(xν) =
2
(−ν/2)
Γ(ν/2)
x
(ν/2−1)
exp(−x/2) for x > 0.
where Γ(z) =
∞
0
t
z−1
exp[−t ]dt (if z is an integer then
Γ(z) = (z −1)!).
The mean of a chisquare random variable is ν, its variance is
2ν, and (when ν ≥ 2) its modal value is ν −2.
The parameter ν is referred to as the degrees of freedom.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
0 10 20 30 40
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
0
.
5
x
D
e
n
s
i
t
y
chisquare 1
chisquare 4
chisquare 15
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Some facts about the chisquare distribution:
The chisquare distribution is important because the
asymptotic sampling distribution of many test statistics will
be chisquare.
If the random variables X
1
, . . . , X
k
are i.i.d. and if each of
these variables has a standard normal distribution, then
the sum of squares X
2
1
+· · · + X
2
k
has a chisquare
distribution with k degrees of freedom.
If the random variables X
1
, . . . , X
k
are independent and if
X
i
follows a chisquare distribution with ν
i
degrees of
freedom for i = 1, . . . , k then the sum X
1
+· · · + X
k
has a
chisquare distribution with ν
1
+· · · +ν
k
degrees of
freedom.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
If a sample (X
1
, . . . , X
n
) of any size n is taken from a
normal distribution then the random variable
1
V[X]
n
¸
i =1
(X
i
−
¯
X
n
)
2
follows a chisquare distribution with n −1 degrees of
freedom.
The R functions rchisq(), dchisq(), and pchisq()
calculate pseudorandom chisquare deviates, the
chisquare density function, and the chisquare distribution
function respectively.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
The t Distribution
The t probability density function is given by
f
t
(xν) =
Γ ((ν + 1)/2)
√
πνΓ(ν/2)
×
1
1 +
x
2
ν
(ν+1)/2
The mean of a t
ν
random variable is 0 and it’s variance is
ν/(ν −2) as long as ν > 2.
The mean of a t
1
RV does not exist.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
−4 −2 0 2 4
0
.
0
0
.
1
0
.
2
0
.
3
0
.
4
0
.
5
0
.
6
x
D
e
n
s
i
t
y
t 1
t 4
t 15
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Some facts about the t distribution:
The t distribution can be motivated as follows. If
Z ∼ N(0, 1), Y ∼ χ
2
ν
, and Z and Y are independent, then
X ≡
Z
Y
ν
follows a t
ν
distribution.
If a sample (X
1
, . . . , X
n
) of any size n is taken from a
normal distribution with zero mean and unknown variance
then the sampling distribution of the sample mean divided
by the sample standard error will have the t distribution
with ν = n −1.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
The sampling distribution of regression coefﬁcients (after
some standardization) can be shown to follow a
t distribution.
As ν →∞the t
ν
distribution approaches the N(0, 1)
distribution.
The R functions rt(), dt(), and pt() calculate
pseudorandom t deviates, the t density function, and the t
distribution function respectively.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
The F Distribution
The F density is given by:
f
F
=
Γ((ν
1
+ν
2
)/2
Γ(ν
1
/2)Γ(ν
2
/2)
(ν
1
/ν
2
)
ν
1
/2
x
(ν
1
−2)/2
1 +
ν
1
ν
2
x
−(ν
1
+ν
2
)/2
ν
1
is sometimes called the numerator degrees of freedom and
ν
2
is sometimes called the denominator degrees of freedom.
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
0.0 0.5 1.0 1.5 2.0
0
1
2
3
4
x
D
e
n
s
i
t
y
F 1,2
F 5,5
F 30, 20
F 500, 200
Gov2000: Quantitative Methodology for Political Science I
Deﬁnitions and Notation
Random Variables and Distributions
Expectation and Transformations
Elementary Asymptotics
Some Important Distributions
Some facts about the F distribution:
if X
1
and X
2
are independent chisquare RVs with ν
1
and
ν
2
degrees of freedom respectively then (X
1
/ν
1
)/(X
2
/ν
2
)
follows an F distribution with ν
1
numerator df and ν
2
denominator df.
If X follows a t distribution with ν df, then X
2
follows an F
distribution with 1 numerator df and ν denominator df.
The F distribution will be useful for testing hypotheses
about multiple regression coefﬁcients.
The R functions rf(), df(), and pf() calculate
pseudorandom F deviates, the F density function, and
the F distribution function respectively.
Gov2000: Quantitative Methodology for Political Science I
This action might not be possible to undo. Are you sure you want to continue?