Professional Documents
Culture Documents
Stat20f Note4 PDF
Stat20f Note4 PDF
Uploaded by
최윤Original Title
Copyright
Available Formats
Share this document
Did you find this document useful?
Is this content inappropriate?
Report this DocumentCopyright:
Available Formats
Stat20f Note4 PDF
Stat20f Note4 PDF
Uploaded by
최윤Copyright:
Available Formats
Chapter 4.
Two Random Variables
1. Joint and Marginal Distributions
(1) Let and be two random variables defined on the same probability space. A bivariate
random vector is a function from the sample space into the -dimensional
Euclidean space.
(2) Let be a discrete bivariate random vector and let be the range of .
A real valued function is called the joint probability mass function (pmf) of
if and only if for all ∈ .
(3) If is the joint pmf of a discrete bivariate random vector , then
≥ , ,
and
∈ .
∈
(4) Let be the joint pmf of a discrete bivariate random vector . The function
is called the marginal pmf of . Similarly, the function
is called the marginal pmf of .
(5) Calculus class example
Consider a population of students who have just finished a first course in calculus.
Of these , have earned ’s, ’s, and ’s, ’s, or ’s. A sample of size is
taken at random and without replacement from this population in a way that each possible
sample has probability /
of being selected. Within the sample of , let be the
number of students, the number of students, then is the number of other
students. The range of is defined by the collection of nonnegative integers such
that ≤ . The joint pmf of and is
1
,
⋯ , ⋯ , ≤ .
The probability that or student and more than students in the sample is
≤
.
The probability that there are at least A or B students in the sample is
≥
The marginal pmf of is
,
⋯ .
Thus the marginal distribution of is a hypergeometric distribution .
Similarly, the marginal distribution of is , that is
,
⋯ .
(6) The joint probability density function (pdf) of a continuous bivariate random vector
is an integrable function with the following properties:
(a) ≥ for all
∞ ∞
(b)
∞ ∞
(c) ∈
2
(7) Let be a continuous bivariate random vector with joint pdf . The function
∞
∞
is called the marginal pdf of . Similarly,
∞
∞
is called the marginal pdf of .
(8) Let be a continuous bivariate random vector with joint pdf
.
(a) Find the value of the constant .
Solution: Since the joint pdf integrates to ,
∞ ∞
∞ ∞
.
Hence and
.
(b) Find the probability .
Solution:
(c) Find the marginal pdf of .
Solution:
(d) Find the marginal pdf of .
Solution:
3
(9) For a bivariate random vector ,
, ,
are called the support of , the support of , and the support of , respectively.
2. Conditional Distributions and Independence
(1) Let be a bivariate random vector with joint pdf or pmf and marginal pdfs
or pmfs and . The conditional pdf or pmf of given is defined as
for any fixed with . The conditional pdf or pmf of given is defined as
for any fixed with .
(2) In accordance with the definition of the conditional pdf or pmf and
we may write joint pdf or pmf as
.
(3) If random variables and are of continuous type,
∈
is called the conditional probability that ∈ , given . Similarly, the conditional
probability that ∈ , given , is
∈ .
If and are discrete random variables,
∈
∈
and
∈ .
∈
(4) Let be a discrete bivariate random vector with joint pmf
.
4
Since is a joint pmf,
.
Thus
and
.
Then the marginal probability mass functions are
,
and
,
Thus, the conditional pmf , given , is equal to
when .
Similarly, the conditional pmf , given , is equal to
when .
(5) Calculus class example
and have the joint pmf
,
⋯ , ⋯ , ≤ .
We showed that and have marginal hypergeometric distributions and
, respectively. That is,
and
.
⋯ ⋯
Thus,
5
,
⋯ .
That is , the conditional distribution of given is ,
∼ . Similarly, we obtain
,
⋯ .
Hence ∼ and the conditional probability is equal to
.
(6) Let and have the joint pdf
.
The marginal pdfs are
and
.
Thus, the conditional pdf of given is
.
The conditional probability is
.
The conditional pdf of given is
.
(7) Let have the joint pdf
∞ .
6
The marginal pdfs are
∞
∞
and
∞ .
Thus ∼ and ∼ . The conditional pdf of given is
so ∼ . The conditional pdf of given is
∞ ∞
and
.
(8) Let and have joint pdf or pmf and marginals and . and are
(stochastically) independent if and only if
, ∈ , ∈ .
(9) Two random variables and are independent if and only if the conditional distribution of
given does not depend on . That is,
, ∈ , ∈ .
Similarly, and are independent if and only if
∈ , ∈ .
(10) Let two random variables and have supports and , respectively, and have the
joint pdf or pmf . Then and are independent if and only if can be written
as a product of a nonnegative function of and a nonnegative function of . That is,
.
(11) Let and be two independent random variables. Then
× and ∈ ∈ ∈ ∈ .
7
(12) Let have the joint pdf
.
Since
,
and are independent. The marginal pdfs are
and
.
Hence
.
(13) Let have the joint pdf
.
Since cannot be expressed as , and are not independent.
The marginal pdfs are
and
.
Thus
≠ .
(14) Let have the joint pdf
∞ .
Since ∞ , ∞ , and ∞ ,
≠ × .
So and are not independent. The marginal pdfs are
∞
and
∞ .
8
Thus
≠ .
3. Expectation
(1) Let be a bivariate random vector with joint pdf or pmf . The expected value
of a function of the two random variables and is defined as
if is discrete, and
∞ ∞
∞ ∞
if is continuous.
(2) We define expectations with special names for continuous . The definitions associated
with expectations in the discrete case are the same as those associated with the continuous
case after replacing the integrations with summations.
(a) If , then
∞ ∞ ∞
∞ ∞
∞
is called the mean of .
(b) If , then
∞ ∞ ∞
∞ ∞
∞
is called the variance of .
(c) If , then
∞ ∞ ∞
∞ ∞
∞
is called the mean of .
(d) If , then
∞ ∞ ∞
∞ ∞
∞
9
is called the variance of .
(e) If , then
∞ ∞
∞ ∞
is called the product moment of and .
(3) The covariance of random variables and , denoted by or , is defined as
.
(4) The correlation coefficient, denoted by or , of and is defined as
(5) For any random variables and ,
.
Proof:
(6)
(7)
(8) If and are any two random variables and , , , and are real constants, then
.
Proof:
10
(9) Let have the joint pdf
.
The marginal pdf of is given by
,
and the marginal pdf of is
.
Note that and have the same distribution. Hence,
,
,
and
.
The product moment of and is given by
.
Therefore, the covariance of and is given by
.
and the correlation coefficient is
.
(10) Let and be two independent random variables and let be a function only of
and be a function only of . Then
.
11
∞ ∞
Proof:
∞ ∞
∞ ∞
∞ ∞
∞ ∞
∞
∞
(11) If and are independent, then . Thus and .
(12) does not imply that and are independent.
Let be a random variable. Define sin and cos . and are
clearly not independent since with probability . Now,
sin sin ,
and
cos cos .
Thus,
sin cos sin
sin .
(13) Cauchy-Schwarz inequality
Let and have finite second moments, then
≤ ,
with equality if and only if for some constant .
Proof: Define
.
Now is a quadratic function in which is greater than or equal to .
If , then the roots of are not real, so
.
12
If for some , say , then , which implies
.
(14) For any random variables and , ≤ with equality if and only if one random
variable is a linear function of the other with probability .
Proof: Set and and apply the Cauchy-Schwarz inequality to get
≤ .
Hence,
≤
with equality if and only if for some constant . That is,
± if and only if for some constants ≠ and .
(15) If and are two random variables and , , , and are real constants, then
and
.
(16) Let and be two random variables with the variances , , and the correlation
coefficient . Find the correlation coefficient between and .
Solution: The covariance of and is
.
Thus the covariance of and is given by
.
Now, the variance of and the variance of are
13
and
.
Hence the correlation coefficient between and is given by
.
4. Conditional Expectation
(1) Let be a bivariate random vector with joint pdf or pmf . The conditional
expectation of given , denoted by or , is defined as
∞
∞
if is continuous, and
if is discrete. Similarly, the conditional expectation of given is defined as
∞
∞
if is continuous, and
if is discrete.
(2) The conditional mean of given is defined as
,
which is the mean of the conditional distribution of given .
Similarly, the conditional mean of given is defined as
.
(3) The conditional variance of given which is the variance of the conditional
distribution of given , is defined as
.
Similarly, the conditional variance of given is defined as
14
.
(4) Let have the joint pdf
∞ .
The conditional pdf of is
,
so ∼ . Thus,
and .
Since the conditional pdf of is
∞ ,
we can express as , where is a random variable. Hence
and
.
(5) The conditional expectation is a function of . That is, for each value of ,
is a real number obtained by computing the appropriate integral or sum. Thus,
is a random variable whose value depends on the value of . If , the value
of the random variable is . The random variable is called
the conditional expectation of given . Similarly, the random variable is
the conditional expectation of given . Since the conditional variance is
also a function of , the random variable is a function of . Similarly,
is a random variable which is a function of .
(6) Let have the joint pdf
∞ .
We showed
, , , .
15
Thus,
, , , .
(7) Let be a bivariate random vector with joint pdf or pmf . If or
is given, the random variables can be replaced by their values when we compute the
conditional expectation. Thus,
,
,
,
.
(8) If ∼ ∞ , then
,
,
,
.
(9) Let have joint pdf or pmf . If and exist, then
and
.
Proof: Let be of continuous type. Then
,
16
.
The proof in the discrete case is similar.
(10) Let be a bivariate random vector. Then
and .
The expectation can be obtained using the distribution of or . That is,
or
.
For example, can be obtained by
,
or
.
(11) For any two random variables and ,
,
provided that the expectations exist.
Proof:
.
17
(12) Let have the distribution. If the conditional distribution is , then
∞
.
It is not easy to find the marginal pdf of , but we can easily compute the mean and
the variance of using the distribution of and the distribution of . We know
, ,
,
.
Hence
and
.
5. Hierarchical Model
(1) A random variable is said to have a mixture distribution if the distribution of depends
on a quantity that also has a distribution. In this case, the probability model of
can be expressed as
∼ , ∼ .
This is called a hierarchical model.
(2) Let have the joint pdf
∞ .
Then the probability model of can be expressed as
∼ , ∼ ∞ ,
or
∼ ∞ , ∼ ∞ .
18
(3) Suppose that a hierarchical model is given by
∼ , ∼ .
If both and are continuous or discrete, then the joint pdf or pmf is given by
and we can find , , probabilities, and expectations as before. If one random
variable is continuous and the other one is discrete, the joint pdf or pmf cannot be
defined. However, we can use as a joint probability function and probabilities
and expectations can be calculated by summing discrete variables and integrating
continuous variables. If is discrete and is continuous, then the marginal pmf of
is given by
∞ ∞
∞
∞
∞
∞
and the conditional pdf of given can be obtained by
.
∞
∞
Similarly, if is continuous and is discrete, then
and .
(4) Suppose that has distribution and the conditional distribution is .
The means, variances, and correlation coefficient can be obtained using the distribution of
and the distribution of . We know that
, , , .
The mean and variance of are
,
and
19
.
The product moment is given by
.
Hence, the covariance and the correlation coefficient are
,
and
.
The joint pdf of is
.
The marginal pdf of is
∞
∞
ln ln ln
.
Hence the conditional pdf of given is
ln
when ,
ln ln
when ≤ .
Now, since follows distribution,
has distribution.
20
Thus, has distribution. Since the conditional distribution of given
does not depend on , and are independent. The joint model for can be
expressed as
∼ , ∼ , and are independent.
(5) Suppose that has distribution and the conditional distribution is .
We have
, , , .
Thus,
,
and
.
The joint pmf of and is
,
⋯ , ⋯
⋯ ⋯ .
Thus, the marginal pmf of is given by
∞
∞
,
⋯ ,
that is, has distribution.
21
The conditional pmf of is
, ⋯
⋯ .
Thus, has distribution and has
distribution. Since the conditional distribution of given does not depend on ,
and are independent. Thus the probability model for can be expressed as
∼ , ∼ , and are independent.
The independence of and can be used to find the correlation coefficient.
and so
.
Hence
.
(6) Let the conditional distribution be and let be a random variable.
Then
, , ,
and
, .
Therefore
and
.
22
The marginal pmf of is given by
∞
∞
⋯ .
This is called the beta-binomial distribution pmf.
6. Trinomial distribution
(1) A discrete bivariate random vector is said to have the trinomial distribution with
parameters , , , ∼ , if its joint pmf is of the form
, ⋯ , ⋯ , ≤ ≤ ,
where , , , and is a positive integer.
If we let and , then ∼ and ∼ .
(2) A certain game involves rolling a fair die and watching the numbers of rolls of and .
What is the probability that in rolls of the die one and three will be observed?
Solution: Let denote the number of and denote the number of . Then follows
trinomial distribution with , , , and . Hence
the probability that in rolls of the die one and three will be observed is
(3) Let ∼ , then ∼ and ∼ .
Proof: The marginal pmf of is given by
23
, ⋯
Similarly, the pmf of is given by
, ⋯
(4) Let ∼ , then
∼ ,
and
∼ , .
Proof: The conditional pmf of is given by
, ⋯ , .
Similarly, is given by
, ⋯ , .
(5) If has distribution, then
, , .
24
Thus,
.
Hence,
and
7. Bivariate normal distribution
(1) A continuous bivariate random vector is said to have a bivariate normal
distribution, ∼ , if the joint pdf is of the form
, ∞ ∞ ∞ ∞ ,
where
, , .
(2) If ∼ , then the marginal distributions are
∼ , ∼ ,
and the conditional distributions are
∼ , , ,
∼ , , ,
25
and
.
Proof: First, we change to sum of perfect squares.
.
Therefore,
,
where
and .
Thus,
.
Similarly, we can show that
,
where
,
and
, , .
26
Now,
.
Hence,
,
and
.
(4) If and have a bivariate normal distribution with correlation coefficient , then
and are independent if and only if .
Proof: We see that the product , which is the joint pdf of and , equals
because when , is a normal pdf with mean and variance
. That is, if , then the joint pdf factors into the product of the two marginal
pdfs, and hence and are independent random variables.
(5) The conditional pdf of is and the marginal pdf of is
. Find the distribution of and the conditional distribution of .
Solution: Since is the pdf of the distribution and is the pdf of the
distribution, ∼ and ∼ . Thus has a bivariate
normal distribution. Let ∼ . Then
27
, ,
and
, .
This gives
, , .
Thus,
, ,
and
∼ .
Hence, has the distribution and the conditional distribution of given
is a normal distribution with the mean
and the variance
.
That is,
∼
28
You might also like
- Devil in the Grove: Thurgood Marshall, the Groveland Boys, and the Dawn of a New AmericaFrom EverandDevil in the Grove: Thurgood Marshall, the Groveland Boys, and the Dawn of a New AmericaRating: 4.5 out of 5 stars4.5/5 (266)
- A Heartbreaking Work Of Staggering Genius: A Memoir Based on a True StoryFrom EverandA Heartbreaking Work Of Staggering Genius: A Memoir Based on a True StoryRating: 3.5 out of 5 stars3.5/5 (231)
- The Sympathizer: A Novel (Pulitzer Prize for Fiction)From EverandThe Sympathizer: A Novel (Pulitzer Prize for Fiction)Rating: 4.5 out of 5 stars4.5/5 (122)
- Grit: The Power of Passion and PerseveranceFrom EverandGrit: The Power of Passion and PerseveranceRating: 4 out of 5 stars4/5 (590)
- The World Is Flat 3.0: A Brief History of the Twenty-first CenturyFrom EverandThe World Is Flat 3.0: A Brief History of the Twenty-first CenturyRating: 3.5 out of 5 stars3.5/5 (2259)
- Shoe Dog: A Memoir by the Creator of NikeFrom EverandShoe Dog: A Memoir by the Creator of NikeRating: 4.5 out of 5 stars4.5/5 (540)
- The Little Book of Hygge: Danish Secrets to Happy LivingFrom EverandThe Little Book of Hygge: Danish Secrets to Happy LivingRating: 3.5 out of 5 stars3.5/5 (401)
- The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good LifeFrom EverandThe Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good LifeRating: 4 out of 5 stars4/5 (5813)
- Never Split the Difference: Negotiating As If Your Life Depended On ItFrom EverandNever Split the Difference: Negotiating As If Your Life Depended On ItRating: 4.5 out of 5 stars4.5/5 (844)
- Her Body and Other Parties: StoriesFrom EverandHer Body and Other Parties: StoriesRating: 4 out of 5 stars4/5 (822)
- Team of Rivals: The Political Genius of Abraham LincolnFrom EverandTeam of Rivals: The Political Genius of Abraham LincolnRating: 4.5 out of 5 stars4.5/5 (234)
- The Emperor of All Maladies: A Biography of CancerFrom EverandThe Emperor of All Maladies: A Biography of CancerRating: 4.5 out of 5 stars4.5/5 (271)
- Hidden Figures: The American Dream and the Untold Story of the Black Women Mathematicians Who Helped Win the Space RaceFrom EverandHidden Figures: The American Dream and the Untold Story of the Black Women Mathematicians Who Helped Win the Space RaceRating: 4 out of 5 stars4/5 (897)
- Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic FutureFrom EverandElon Musk: Tesla, SpaceX, and the Quest for a Fantastic FutureRating: 4.5 out of 5 stars4.5/5 (474)
- The Hard Thing About Hard Things: Building a Business When There Are No Easy AnswersFrom EverandThe Hard Thing About Hard Things: Building a Business When There Are No Easy AnswersRating: 4.5 out of 5 stars4.5/5 (348)
- The Gifts of Imperfection: Let Go of Who You Think You're Supposed to Be and Embrace Who You AreFrom EverandThe Gifts of Imperfection: Let Go of Who You Think You're Supposed to Be and Embrace Who You AreRating: 4 out of 5 stars4/5 (1092)
- On Fire: The (Burning) Case for a Green New DealFrom EverandOn Fire: The (Burning) Case for a Green New DealRating: 4 out of 5 stars4/5 (74)
- The Yellow House: A Memoir (2019 National Book Award Winner)From EverandThe Yellow House: A Memoir (2019 National Book Award Winner)Rating: 4 out of 5 stars4/5 (98)
- Vivid S60N R4.0 Intl v18Document19 pagesVivid S60N R4.0 Intl v18Lotfi HedhliNo ratings yet
- The Unwinding: An Inner History of the New AmericaFrom EverandThe Unwinding: An Inner History of the New AmericaRating: 4 out of 5 stars4/5 (45)
- MT-15 IN 1: User ManualDocument16 pagesMT-15 IN 1: User ManualIndriyani CigullNo ratings yet
- HedEx Lite-Obtaining and Using Huawei Documentation Easily V5.1Document13 pagesHedEx Lite-Obtaining and Using Huawei Documentation Easily V5.1Khalid MahmoodNo ratings yet
- Ricky Earf Oct 24 Oct 28Document5 pagesRicky Earf Oct 24 Oct 28JR DelantarNo ratings yet
- ASR Manager InstallDocument7 pagesASR Manager InstallHieu TranNo ratings yet
- Sample MCQs DVDocument11 pagesSample MCQs DVNIKET PATADIYANo ratings yet
- Unit-15 - Project ManagementDocument18 pagesUnit-15 - Project ManagementCLARA D SOUZA THOMASNo ratings yet
- EB7SMDocument18 pagesEB7SMAfif ZarradNo ratings yet
- EP Board PDFDocument2 pagesEP Board PDFReza KühnNo ratings yet
- DHI-NVR608R-64/128-4KS2: 64/128 Channel Ultra 4K H.265 Network Video RecorderDocument3 pagesDHI-NVR608R-64/128-4KS2: 64/128 Channel Ultra 4K H.265 Network Video RecorderMmd KaissNo ratings yet
- Css 10 Lesson 3 4th QuarterDocument31 pagesCss 10 Lesson 3 4th QuarterMarcusNo ratings yet
- Readme - en ParcadDocument6 pagesReadme - en ParcadHogen KoglerNo ratings yet
- Ms AccessDocument26 pagesMs AccessAkrati GuptaNo ratings yet
- TCCC Quick Ref Guide Final Nov2020Document12 pagesTCCC Quick Ref Guide Final Nov2020vinyrolanNo ratings yet
- Check Point Appliance Comparison ChartDocument4 pagesCheck Point Appliance Comparison Chartrocky ron100% (1)
- D127/D128 Service ManualDocument41 pagesD127/D128 Service ManualJohn ArtunduagaNo ratings yet
- Table of Specification GRADE 3 Mathematics End of Term Test DECEMBER 2019Document4 pagesTable of Specification GRADE 3 Mathematics End of Term Test DECEMBER 2019minnett pinnockNo ratings yet
- Resume PDFDocument3 pagesResume PDFChandra Sekhar DNo ratings yet
- 4th Sem SyllabusDocument12 pages4th Sem Syllabusayushib1220No ratings yet
- GF Python Data StructuresDocument17 pagesGF Python Data StructuresarupghoshbietNo ratings yet
- Asus p5vd2-Mx Rev 1.03gDocument32 pagesAsus p5vd2-Mx Rev 1.03gLourencosud SudNo ratings yet
- Cse Ds Btech IV Yr Vii Sem Syllabus July 2023Document24 pagesCse Ds Btech IV Yr Vii Sem Syllabus July 2023Gourav mulevaNo ratings yet
- Seritag Launch Tapstart Ready To Use QR Code and NFC Tag LabelsDocument3 pagesSeritag Launch Tapstart Ready To Use QR Code and NFC Tag LabelsPR.comNo ratings yet
- Personal Digital Assistant (PDA) : Quick Reference GuideDocument52 pagesPersonal Digital Assistant (PDA) : Quick Reference GuideKhadi MbarkiNo ratings yet
- Detection of Fake News Using Machine Learning AlgorithmsDocument7 pagesDetection of Fake News Using Machine Learning AlgorithmsIJRASETPublicationsNo ratings yet
- Perancangan AlgoritmaDocument29 pagesPerancangan AlgoritmaenricoNo ratings yet
- JNCIA Cloud QuesDocument5 pagesJNCIA Cloud Quespattabiram100% (1)
- XSSDBDocument37 pagesXSSDBtesterNo ratings yet
- CS416 - Parallel and Distributed Computing: Lecture # 01Document20 pagesCS416 - Parallel and Distributed Computing: Lecture # 01abdullah ashrafNo ratings yet
- Matlab 6Document15 pagesMatlab 6hurairabaig37No ratings yet
