You are on page 1of 3

The Rules of Summation

n

1 x i ¼ x 1 þ x 2 þ þ x n

i¼

n

1 a ¼ na

i¼

n

n

1 ax i ¼ a

i¼

n

i ¼ 1 x i

n

n

1 ðx i þ y i Þ ¼

i¼

n

i

¼1 x i þ

i ¼ 1 y i

n

n

1 ðax i þ by i Þ ¼ a

i¼

n

i

¼1 x i þ b

i ¼1 y i

n

1 ða þ bx i Þ ¼ na þ b

i¼

i¼ 1 x i

x

¼

n

i¼1 x i

¼ x 1 þ x 2 þ þ x n

n n

n

1 ðx i x Þ ¼ 0

i¼

2

3

2

1 f ðx i ; y j Þ ¼ ½

i

¼ 1

i ¼1 j¼

f ðx ; y

i

1

Þ þ f ðx ; y

i

2

Þ þ f ðx ; y

i

3

Þ

¼ f ðx 1 ; y 1 Þ þ f ðx 1 ; y 2 Þ þ f ðx 1 ; y 3 Þ

þ f ðx 2 ; y 1 Þ þ f ðx 2 ; y 2 Þ þ f ðx 2 ; y 3 Þ

Expected Values & Variances

E

ðX Þ ¼ x 1 f ðx 1 Þ þ x 2 f ðx 2 Þþ þ x n f ðx n Þ

n

¼ 1 x i f ð x i Þ ¼ x f ðx Þ

i

¼

x

E ½ gðX Þ ¼ gðx Þ f ðx Þ

x

E ½ g

1

ðX Þ þ g

2

ðX Þ ¼ ½

x

g ðx Þ þ g

1

2

ðx Þ f ðx Þ

¼ g 1 ðx Þ f ðx Þ þ g 2 ðx Þ f ðx Þ

¼

x x

E ½ g ðX Þ þ E g

1

½

2

ðX Þ

E( c) ¼ c E( cX ) ¼ cE( X ) E( a þ cX ) ¼ a þ cE( X ) var(X ) ¼ s 2 ¼ E[ X E( X )] 2 ¼ E( X 2 ) [E (X )] 2 var(a þ cX ) ¼ E [(a þ cX ) E(a þ cX )] 2 ¼ c 2 var(X )

Marginal and Conditional Distributions

f

ðx Þ ¼ f ðx ; y Þ for each value X can take

y

f ðy Þ ¼ f ðx ; y Þ for each value Y can take

f

x

ðx j yÞ ¼ P½ X ¼ x j Y ¼ y ¼ f ðx; y Þ

f ðy Þ

If X and Y are independent random variables, then f (x,y ) ¼ f ( x )f ( y ) for each and every pair of values

x and y. The converse is also true.

If X and Y are independent random variables, then the conditional probability density function of X given that

Y ¼ y is f ðx j yÞ ¼ f ðx ; yÞ

f ðy Þ

¼

f ðx Þ f ðyÞ

f ð yÞ

¼

f ðx Þ

for each and every pair of values x and y. The converse is also true.

Expectations, Variances & Covariances

covðX ; Y Þ ¼ E ½ðX E ½ X ÞðY E ½ Y Þ

¼ ½ x E ðX Þ ½ y E ðY Þ f ðx ; y Þ

x y

covðX ; Y Þ

r ¼ p

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

var ðX ÞvarðY Þ

E( c 1 X þ c 2 Y ) ¼ c 1 E( X ) þ c 2 E( Y )

E( X þ Y ) ¼ E( X ) þ E( Y )

var( aX þ bY þ cZ ) ¼ a 2 var(X ) þ b 2 var(Y ) þ c 2 var( Z )

þ 2abcov(X,Y ) þ 2accov(X,Z ) þ 2bccov(Y,Z )

If X, Y, and Z are independent, or uncorrelated, random variables, then the covariance terms are zero and:

var ðaX þ bY þ cZ Þ ¼ a 2 var ðX Þ

Normal Probabilities

þ b 2 var ðY Þ þ c 2 var ðZ Þ

If X N ( m, s 2 ), then Z ¼ X m N ð0; 1Þ

s

If X N ( m, s 2 ) and a is a constant, then

PðX aÞ ¼ P Z

a m

s

If X N ðm; s 2 Þ and a and b are constants; then

Pða X bÞ ¼ P

a

m

Z

b m

s s

Assumptions of the Simple Linear Regression Model

SR1

The value of y, for each value of x, is y ¼ b 1 þ

b 2 x þ e

SR2

The average value of the random error e is

SR3

E(e) ¼ 0 since we assume that E( y) ¼ b 1 þ b 2 x The variance of the random error e is var(e) ¼

SR4

s 2 ¼ var( y) The covariance between any pair of random

SR5

errors, e i and e j is cov(e i , e j ) ¼ cov( y i , y j ) ¼ 0 The variable x is not random and must take at least two different values.

SR6 ( optional ) The values of e are normally dis- tributed about their mean e N(0, s 2 )

Least Squares Estimation

If b 1 and b 2 are the least squares estimates, then ^y i ¼ b 1 þ b 2 x i

^e i ¼ y i ^y i ¼ y i b 1 b 2 x i

The Normal Equations Nb 1 þ S x i b 2 ¼ Sy i

Sx i b 1 þ Sx 2 b 2 ¼ Sx i y i

i

Least Squares Estimators

b 2 ¼ Sðx i x Þðy i yÞ S ðx i x Þ 2
b 2 ¼ Sðx i x Þðy i yÞ
S ðx i x Þ 2
b 1 ¼ y b 2 x

Elasticity

h

¼ percentage change in y

percentage change in x ¼ Dx =x ¼ D y y

D x x

Dy =y

h ¼ DE ðy Þ=E ðy Þ ¼ D E ðy Þ

Dx

=x

Dx

E ðy Þ ¼ b 2

x

x

E

ðy Þ

Least Squares Expressions Useful for Theory

b 2 ¼ b 2 þ S w i e i x i x w
b 2 ¼ b 2 þ S w i e i
x i x
w i ¼
Sðx i x Þ 2
Sw i ¼ 0;
S w i x i ¼ 1;
Sw 2 ¼ 1=Sðx i x Þ 2
i
Properties of the Least Squares Estimators
"
Sx 2
s 2
i
varðb 1 Þ ¼ s 2
#
varðb 2 Þ ¼
N Sðx i x Þ 2
S ðx i xÞ 2
"
#
x
covðb 1 ; b 2 Þ ¼ s 2
2
Sðx x Þ
i

Gauss-Markov Theorem: Under the assumptions SR1–SR5 of the linear regression model the estimators b 1 and b 2 have the smallest variance of all linear and unbiased estimators of b 1 and b 2 . They are the Best Linear Unbiased Estimators (BLUE) of b 1 and b 2 .

If we make the normality assumption, assumption SR6, about the error term, then the least squares esti- mators are normally distributed.

s 2 x 2

xÞ 2 ! ; b 2 N b 2 ;

b 1 N b 1 ;

N S ðx i

i

Estimated Error Variance

2 s S ðx i x Þ 2 !
2
s
S ðx i x Þ 2 !

s^ 2 ¼

S^e 2

i

N 2

Estimator Standard Errors

se ðb 1 Þ ¼

t -distribution

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

q

b

varðb 1 Þ ;

se ðb 2 Þ ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

q

b

varðb 2 Þ

If assumptions SR1–SR6 of the simple linear regression model hold, then

t ¼ b k b k

se ðb k Þ

t ð N 2Þ ;

Interval Estimates

k ¼ 1; 2

P[ b 2 t c se( b 2 ) b 2 b 2 þ t c se(b 2 )] ¼ 1 a

Hypothesis Testing

Components of Hypothesis Tests

1. A null hypothesis, H 0

2. An alternative hypothesis, H 1

3. A test statistic

4. A rejection region

5. A conclusion

If the null hypothesis H 0 : b 2 ¼ c is true, then

c t ¼ b se ðb 2 Þ t ð N 2Þ

2

Rejection rule for a two-tail test: If the value of the test statistic falls in the rejection region, either tail of the t-distribution, then we reject the null hypothesis and accept the alternative.

Type I error: The null hypothesis is true and we decide

to reject it.

Type II error: The null hypothesis is false and we decide not to reject it.

p-value rejection rule: When the p-value of a hypoth- esis test is smaller than the chosen value of a, then the test procedure leads to rejection of the null hypothesis.

Prediction

y 0 ¼ b 1 þ b 2 x 0 þ e 0 ; ^y 0 ¼ b 1 þ b 2 x 0 ; f ¼ ^y 0 y 0

b

var ð f Þ ¼ s^ 2

" 1 þ

1 ðx 0 x Þ 2 N þ Sðx i x Þ 2
1 ðx 0 x Þ 2
N þ
Sðx i x Þ 2

ffiffiffiffiffiffiffiffiffiffiffiffiffi

# ; se ð f Þ ¼ varð f Þ

q

b

A (1 a) 100% confidence interval, or prediction

interval, for y 0

^y 0 t c se ð f Þ

Goodness of Fit

Sðy i y Þ 2 ¼ Sð^y i y Þ 2 þ S^e 2

SST ¼ SSR þ SSE

i

R 2 ¼

SSR

SST ¼ 1 SSE

SST

¼ ðcorrðy ; ^y ÞÞ 2

Log-Linear Model

lnðy Þ ¼ b 1 þ b 2 x þ e ;

100 b 2 % change in y given a one-unit change in x :

^y n ¼ expðb 1 þ b 2 x Þ

^y c ¼ expðb 1 þ b 2 x Þ expðs^ 2 =2 Þ

Prediction interval:

b lnð yÞ ¼ b 1 þ b 2 x

b

lnðy Þ t c se ð f Þ i ;

Generalized goodness-of-fit measure

Assumptions of the Multiple Regression Model

exp

h

h

b

lnð yÞ þ t se ð f Þ

c

i

exp

R ¼ ðcorrðy; ^y n ÞÞ 2

2

g

MR1

y i ¼ b 1 þ b 2 x i2 þ þ b K x iK þ e i

MR2

E ( y i ) ¼ b 1 þ b 2 x i2 þ þ b K x iK , E (e i ) ¼ 0.

MR3

var( y i ) ¼ var( e i ) ¼ s 2

MR4

cov( y i , y j ) ¼ cov(e i , e j ) ¼ 0

MR5

The values of x ik are not random and are not exact linear functions of the other explanatory variables.

MR6

y i N ½ðb 1 þ b 2 x i 2 þ þ b K x iK Þ; s 2

,

e i N ð0; s 2 Þ

Least Squares Estimates in MR Model

Least squares estimates b 1 , b 2 ,

Sðb 1 , b 2 ,

, b K minimize

, b K Þ ¼ ðy i b 1 b 2 x i2 b K x iK Þ 2

Estimated Error Variance and Estimator Standard Errors

s^ 2 ¼ N ^e 2 K

i

se ðb k Þ ¼

q

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

b

var ðb k Þ

Hypothesis Tests and Interval Estimates for Single Parameters

Use t -distribution t

¼ b k b k

seð b k Þ

t ð N K Þ

t-test for More than One Parameter

When H 0 is

se ðb 2 þ

true

cb 3 Þ ¼

H 0 : b 2 þ c b 3 ¼ a

t ¼ b 2 þ cb 3 seðb 2 þ

a

cb 3 Þ t ð N K Þ

q

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

b b

var ðb 2 Þ þ c 2 varðb 3 Þ þ 2c b

covðb 2 ; b 3 Þ

Regression with Stationary Time Series Variables

Finite distributed lag model

y t ¼ a þ b 0 x t þ b 1 x t 1 þ b 2 x t 2 þ þ b q x t q þ v t Correlogram

r k ¼ ðy t yÞð y t k y Þ= ðy t yÞ 2

For H 0 : r k ¼ 0;

z ¼

p

ffiffiffiffi T r k N ð0; 1Þ

LM test

b 1 þ b 2 x t þ r^e t 1 þ ^v t

^e t ¼ g 1 þ g 2 x t þ r^e t 1 þ ^v t

y t ¼

Test H 0 : r ¼ 0 with t -test

Test using LM ¼ T R 2

Joint F -tests

AR(1) error

y t ¼ b 1 þ b 2 x t þ e t

e t ¼ re t 1 þ v t

To test J joint hypotheses,

F

¼ ðSSE R SSE U Þ=J

SSE U =ðN K Þ To test the overall significance of the model the null and alternative hypotheses and F statistic are

H 0 : b 2 ¼ 0;

H

b 3 ¼ 0 ;:::; b K ¼ 0

1 : at least one of the b k is nonzero

F

¼ ðSST SSE Þ=ðK 1Þ

SSE =ðN K Þ

RESET: A Specification Test

y i ¼ y i ¼

y i ¼ b 1 þ b 2 x i 2 þ b 3 x i 3 þ g 1 ^y 2 þ g 2 ^y 3 þ e i ;

Model Selection

b 1 þ b 2 x i 2 þ b 3 x i 3 þ e i

b 1 þ b 2 x i 2 þ b 3 x i 3 þ g 1 ^y 2 þ e i ;

i

i

i

^y i ¼ b 1 þ b 2 x i2 þ b 3 x i3

H 0 : g 1 ¼ 0

H 0 : g 1 ¼ g 2 ¼ 0

AIC ¼ ln(SSE =N ) þ 2K=N SC ¼ ln(SSE =N ) þ K ln( N ) =N

Collinearity and Omitted Variables

Nonlinear least squares estimation

y t ¼ b 1 ð1 rÞ þ b 2 x t þ ry t 1 b 2 rx t 1 þ v t

ARDL( p, q) model y t ¼ d þ d 0 x t þ d l x t 1 þ þ d q x t q þ u l y t 1

þ þ u p y t p þ v t AR( p) forecasting model y t ¼ d þ u l y t 1 þ u 2 y t 2 þ þ u p y t p þ v t

Exponential smoothing

Multiplier analysis d 0 þ d 1 L þ d 2 L 2 þ þ d q L q ¼ ð1 u 1 L u 2 L 2 u p L p Þ

^y t ¼ ay t 1 þ ð1 aÞ^y t 1

Unit Roots and Cointegration

ðb 0 þ b 1 L þ b 2 L 2 þ Þ

Unit Root Test for Stationarity: Null hypothesis:

H 0 : g ¼ 0 Dickey-Fuller Test 1 (no constant and no trend):

Dy t ¼ gy t 1 þ v t

Dickey-Fuller Test 2 (with constant but no trend):

Dy t ¼ a þ g y t 1 þ v t

y i ¼ b 1 þ b 2 x i 2 þ b 3 x i 3 þ e i

var ðb 2 Þ ¼

s 2

2

ð1 r 23 Þ ðx i2 x 2 Þ 2

Dickey-Fuller Test 3 (with constant and with trend):

Dy t ¼ a þ g y t 1 þ l t þ v t

Augmented Dickey-Fuller Tests:
b

When x 3 is omitted; biasðb 2 Þ ¼ E ðb 2 Þ b 2 ¼ b 3

Heteroskedasticity

covðx 2 ; x 3 Þ

b

var ðx 2 Þ

2

var( y i ) ¼ var( e i ) ¼ s i General variance function

s 2 ¼ expða 1 þ a 2 z i2 þ þ a S z iS Þ

i

m

Dy t ¼ a þ g y t 1 þ

s

¼ 1

a s Dy t s þ v t

Test for cointegration D^e t ¼ g^e t 1 þ v t

Random walk:

Random walk with drift:

Random walk model with drift and time trend:

y t ¼ y t 1 þ v t

y t ¼ a þ y t 1 þ v t

y t ¼ a þ dt þ y t 1 þ v t

Breusch-Pagan and White Tests for H 0 : a 2 ¼ a 3 ¼ ¼ a S ¼ 0

When H 0 is true x 2 ¼ N R 2 x 2

ðS 1Þ

Goldfeld-Quandt test for H 0 : s M ¼ s R versus H 1 : s M s

2

2

2

2

When H 0 is true F ¼ s^ M =s^

2

R

F ðN M K M ; N R K R Þ

Transformed model for varðe i Þ¼ s 2 ¼ s 2 x i ffiffiffiffi

y

i =

p

ffiffiffiffi

x

i

¼

b 1 ð 1=

p

ffiffiffiffi

x

i

i

Þ þ b 2 ð x i =

p

x

i

Þ þ e i =

p

ffiffiffiffi

x

i

Estimating the variance function

ln ð^e 2 Þ ¼ lnðs 2 Þ þ v i ¼ a 1 þ a 2 z i2 þ þ a S z iS þ v i

i

i

Grouped data

var ðe i Þ ¼ s 2 i ¼

(

2

s M i

2

¼

1; 2;

s R i ¼ 1; 2;

;

;

N M

N R

2

R

Transformed model for feasible generalized least squares

Panel Data

Pooled least squares regression

y it ¼ b 1 þ b 2 x 2 it þ b 3 x 3it þ e it

Cluster robust standard errors cov(e it , e is ) ¼ c ts

Fixed effects model

y it ¼ b 1i þ b 2 x 2it þ b 3 x 3it þ e it

b 1i not random

y

it y i ¼ b 2 ðx 2it x 2i Þ þ b 3 ðx 3it x 3i Þþðe it e i Þ

Random effects model

y it ¼ b 1i þ b 2 x 2it þ b 3 x 3it þ e it

b it ¼ b 1 þ u i random

y it ay i ¼ b 1 ð1 aÞþ b 2 ðx 2it ax 2i Þþ b 3 ðx 3it ax 3i Þþ v ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

a ¼ 1 s e

q

T s u 2 þ s 2

e

Hausman test

it

y i .

p

ffiffiffiffiffi s^ i ¼ b 1

1

.

ffiffiffiffiffi s^ i þ b 2

p

x i .

ffiffiffiffiffi s^ i þ e i .

p

p

ffiffiffiffiffi

s^ i

t ¼ ðb FE ; k b RE ;k Þ

h var b ðb FE ;k Þ varðb b RE ; k Þ

i 1=2