Stat Notes

Stat Notes
Aritrabha Majumdar
March 2024
## Loading required package: viridisLite
1 Empirical Distribution Function
Random variable Y has the ”empirical distribution”
Range(Y ) = {X1 , X2 , ... < Xn }
Goal: Inferring maximum information about the distribution.
2 Sample Mean
X1 + ... + Xn
X=
n
Suppose E[X] = µ and V ar[X] = σ 2 Then E[X] = µ and V ar[X] = σ 2
3 Sample Variance
n
1 X
Sn2 = (Xi − X)2
n − 1 i=1
features
• X follows standard normal distribution.

(n−1) 2
• σ 2 Sn follows χ2n−1 distribution.
• X and Sn2 are independent.
1
4 Observation
Say X1 , ...Xn follows Normal(0, σ 2 ).

Take Zi = Xσi Then
X Z
UX = q Pn , UZ = q Pn
1 1
n−1 i=1 (Xi − X)2 n−1 i=1 (Zi − Z)2
Has same values, and they follow t-Distribution.
5 Student’s t-Distribution
Say,
n
X1 + ... + Xn 1 X
X= and Sn2 = (Xi − X)2
n n − 1 i=1
Then √
n(X − µ)
σ
follows tn−1 distribution.
x = seq(-3,3, length = 1000); density = dnorm(x)
6 Some Inequalities
6.1 Markov’s Inequality
Let X be a non-negative random variable with finite mean and variance. Then
E(X)
P(X ≥ c) ≤ , c>0
c
Note: It is meaningless if µ > c.
Proof: Let X be a continuous random variable.

Z ∞ Z c Z ∞ Z ∞
E(X) = xf (x) dx = xf (x) dx + xf (x) dx ≥ c f (x) dx = c · P(X ≥ c)
0 0 c c
E(X)
=⇒ P(X ≥ c) ≤
c
6.2 Chebychev’s Inequality
1
P (|X − µ| ≥ kσ) ≤
k2
2
(X−µ)2
Proof: We put Y = σ2 and applying Markov’s Inequality,
2
E(Y ) E |X − µ| 1
P(Y ≥ k 2 ) ≤ = = 2
k2 σ2 k2 k
And we are done. Note that the assumptions (or Conditions) applicable in Markov’s Inequality still
prevails!!
Let ε > 0 be given.
√
σ2 1

|X − µ| ε ε n σ
P (|X − µ| > ε) = P > = P |X − µ| > · √ ≤ 2 ·
σ σ σ n ε n
Now as n → ∞, P (|X − µ| > ε) = 0. This is indeed WEAK LAW OF LARGE NUMBERS
7 Law of Large Numbers
7.1 Weak Law of Large Numbers
Let X1 , .., Xn be i.i.d with finite mean and variance. then
lim P (|X − µ| > ε) = 0

n→∞
runningmean = function(x,N)
{
y = rpois(N,x)
y = runif(N,2,3)
c = cumsum(y)
n = 1:N
c/n
}
u = runningmean(1,1000)
v=1:1000; plot(u~v, type="l")
invisible(replicate(9, lines(runningmean(c(0,1), 1000)~v,
type="l", col = sample(viridis(10000),1))))
3
2.5
2.4
2.3
u
2.2
2.1
0 200 400 600 800 1000
par(mfrow=c(1,3))
u = runningmean(1, 100)
x=1:100; plot(u~x, type="l");
invisible(replicate(10,
lines(runningmean(1, 100)~x, type="l", col =
sample(viridis(15, option="A",1))
)
))
x=1:1000; plot(u~x, type="l");
invisible(replicate(10, lines(runningmean(1, 1000)~x, type="l",
col = sample(viridis(15, option="B",1))
)
))
x=1:10000; plot(u~x, type="l");
invisible(replicate(10, lines(runningmean(1, 10000)~x, type="l",
col = sample(viridis(15, option="C",1))
)
))
4
2.6
2.65
2.6
2.5
2.60
2.5
2.4
u
u
2.55
2.3
2.4
2.50
2.2
2.3
0 20 40 60 80 0 200 600 1000 0 4000 8000
x x x
7.2 Strong Law of Large Numbers
Let X1 , .., Xn be i.i.d with finite mean, and

X1 + X2 + ... + Xn
A = lim =µ
n→∞ n
then
P(A) = 1
This is similar as that of saying
∞ \
∞
!
\ [
P |Xn − X| < ε =1
ε>0 n=1 n=N
8 Another Question
Does √
n|X − µ|
→ N (0, 1)
σ
Always occur?
binomialsim1 = rbinom(100,10,0.1)
# generates 100 Binomial (10,0.1) samples
5
par(mfrow=c(1,3))
hist(binomialsim1, main = "Binomial(10, 0.1)")
Binomial(10, 0.1) Binomial(10, 0.25) Binomial(10, 0.5)

40
30
25
25
30
20
20
Frequency
Frequency
Frequency
15
20
15
10
10
10
5
5
0
0 1 2 3 4 0 1 2 3 4 5 6 2 4 6 8 10
binomialsim1 binomialsim2 binomialsim3
par(mfrow=c(1,3))
6
30
40
30
25
25
30
20
20
Frequency
Frequency
Frequency
15
20
15
10
10
10
5
5
0
0
5 10 15 20 10 20 30 40 35 45 55 65
binomialsim1 binomialsim2 binomialsim3
binomial0.1sim1 = rbinom(100,10,0.1)
par(mfrow=c(1,3))
hist(binomial0.1sim1, main = "Binomial(10, 0.1)")
7
30
20
25
30
20
15
Frequency
Frequency
Frequency
20
15
10
10
10
5
5
0
0
0 1 2 3 4 5 10 15 20 80 90 110
binomial0.1sim1 binomial0.1sim2 binomial0.1sim3
8.1 What are we doing?
We have plotted Sn :=Binomial(n, p) for n = 10, 100, 1000.

n
X
Sn = Xi , Xi ∼ Ber(p)
i=1
Now, √ Sn
√
n|X − µ| n| − p| Sn − np
= p n =p → N (0, 1)
σ p(1 − p) np(1 − p)
stdbinom1 = (binomial0.1sim1 - 10*0.1)/sqrt(10*0.1*0.9)
par(mfrow=c(1,3))
hist(stdbinom1)
hist(stdbinom2)
hist(stdbinom3)
8
Histogram of stdbinom1 Histogram of stdbinom2 Histogram of stdbinom3
40
20
15
30
15
Frequency
Frequency
Frequency
10
20
10
5
10
5
0
0
−1 0 1 2 −2 0 1 2 −2 0 1 2 3
stdbinom1 stdbinom2 stdbinom3
par(mfrow=c(1,3))
qqnorm(stdbinom1)
qqline(stdbinom1)
qqnorm(stdbinom2)
qqline(stdbinom2)
qqnorm(stdbinom3)
qqline(stdbinom3)
9
Normal Q−Q Plot Normal Q−Q Plot Normal Q−Q Plot
2.0
3
2
1.5
2
1
1.0
Sample Quantiles
Sample Quantiles
Sample Quantiles
1
0.5
0
0.0
−1
−1
−0.5
−2
−2
−1.0
−2 0 1 2 −2 0 1 2 −2 0 1 2
Theoretical Quantiles Theoretical Quantiles Theoretical Quantiles
par(mfrow=c(1,3))
x= rnorm(100)
boxplot(x,stdbinom1)
10
2
3
2
2
1
1
0
0
−1
−1
−1
−2
−2
−2
−3
1 2 1 2 1 2
8.2 A compact code
S1000std = (binomial0.1sim3-1000*0.1)/sqrt(1000*0.1*0.9)
par(mfrow=c(1,3))
qqnorm(S1000std)
qqline(S1000std)
boxplot(x,S1000std)
hist(S1000std,main="STD-1000")
11
Normal Q−Q Plot STD−1000
3
3
15
2
Sample Quantiles
1
1
Frequency
10
0
0
−1
5
−1
−2
−2
0
−3
−2 0 1 2 1 2 −2 0 1 2 3
Theoretical Quantiles S1000std
8.3 Coming to the Main Story: The Central Limit Theorem
Let {Xn }n≥1 be a sequence of random variable with finite mean and variance. For all x ∈ R
√ x
n(Xn − X)
Z
1 y2
P ≤x → √ e− 2 dy
V ar(X) −∞ 2π
100 −0.5
10·S√
Exercise: Let X ∼Uniform(0, 1). Generate 100 samples of 1
.
12
u1 <- replicate(100,mean(runif(100)))
u2 <- 10*(u1-0.5)/(sqrt(1/12))
par(mfrow=c(1,3))
hist(u2)
qqnorm(u2)
qqline(u2)
boxplot(rnorm(100),u2)
12
Histogram of u2 Normal Q−Q Plot
25
3
2
2
20
1
Sample Quantiles
1
15
Frequency
0
10
−1
−1
5
−2
−2
0
−3
−3
−3 −1 0 1 2 −2 0 1 2 1 2
u2 Theoretical Quantiles
library(moments)
c(skewness(x),skewness(u2))
## [1] -0.0831059 -0.1945186
c(kurtosis(x), kurtosis(u2))
## [1] 3.290796 3.182537
Let’s do for Exponential
u1 <- replicate(100,mean(rexp(100,10)))
u2 <- 10*(u1-0.1)/(sqrt(1/100))
par(mfrow=c(1,3))
hist(u2)
qqnorm(u2)
qqline(u2)
boxplot(rnorm(100),u2)
13
Histogram of u2 Normal Q−Q Plot
2
20
1
Sample Quantiles
15
Frequency
0
10
−1
−1
5
−2
−2
0
−3 −1 0 1 2 −2 0 1 2 1 2
u2 Theoretical Quantiles
library(moments)
c(skewness(x),skewness(u2))
## [1] -0.0831059 0.1399611
c(kurtosis(x), kurtosis(u2))
## [1] 3.290796 2.606101
14

Stat Notes

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Stat Notes

Uploaded by

Copyright:

Available Formats

Stat Notes

## Loading required package: viridisLite

1 Empirical Distribution Function

Random variable Y has the ”empirical distribution”

Range(Y ) = {X1 , X2 , ... < Xn }

Goal: Inferring maximum information about the distribution.

• X follows standard normal distribution.

• X and Sn2 are independent.

Say X1 , ...Xn follows Normal(0, σ 2 ).

Has same values, and they follow t-Distribution.

x = seq(-3,3, length = 1000); density = dnorm(x)

6.1 Markov’s Inequality

Note: It is meaningless if µ > c.

Proof: Let X be a continuous random variable.

6.2 Chebychev’s Inequality

Now as n → ∞, P (|X − µ| > ε) = 0. This is indeed WEAK LAW OF LARGE NUMBERS

7 Law of Large Numbers

7.1 Weak Law of Large Numbers

Let X1 , .., Xn be i.i.d with finite mean and variance. then

lim P (|X − µ| > ε) = 0

0 200 400 600 800 1000

0 20 40 60 80 0 200 600 1000 0 4000 8000

7.2 Strong Law of Large Numbers

Let X1 , .., Xn be i.i.d with finite mean, and

Binomial(10, 0.1) Binomial(10, 0.25) Binomial(10, 0.5)

binomialsim1 binomialsim2 binomialsim3

binomialsim1 binomialsim2 binomialsim3

binomial0.1sim1 binomial0.1sim2 binomial0.1sim3

8.1 What are we doing?

We have plotted Sn :=Binomial(n, p) for n = 10, 100, 1000.

stdbinom1 stdbinom2 stdbinom3

Theoretical Quantiles Theoretical Quantiles Theoretical Quantiles

8.2 A compact code

Theoretical Quantiles S1000std

8.3 Coming to the Main Story: The Central Limit Theorem

## [1] -0.0831059 -0.1945186

## [1] 3.290796 3.182537

Let’s do for Exponential

## [1] -0.0831059 0.1399611

## [1] 3.290796 2.606101

You might also like