46 views

Uploaded by Jorge Rojas-Vallejos

These notes are a summary of the basic math that you need to know to understand Ordinary Least Squares in a proper way. I hope this can be useful for you.

save

- Design and Analysis of Single Factor Experiments
- Explaining Wage Differentials in Belgium - Mincerian Wage Model. An empirical Study.
- How Citizens Integrate Information without Ideological Cues Local Weather and Americans’ Beliefs about Global Warming
- Problem With LPM
- Determinants of Corporate Profitability in Developing Economies
- Calibration by Linear Regression - Tutorial
- Spatial Prediction of Soil Properties Using EBLUP-Minasny & McBratney, 2007
- Multi Col Linearity
- 2.6_Linear+Regression+with+One+Regressor+一元线性回归
- Riv Regress Post Estimation
- Racial Supermarket Access
- Mini-mock Exam - Questions Only - 2.28.2018
- 1981 Trends in Identification
- 3447
- Exercises - Run Test
- SP
- Lampiran Output Spss
- Multiple Regression Review
- Advertisement effectiveness
- 6 Chi-square Analysis
- Impact of Human Capital Investment on Bank
- Trade, Unemployment and Inequality
- Soluciones Ejercicios de Macro 1
- Happiness and Clean Air
- Macroeconomic Theory Notes
- OLS estimators Econometrics
- Soluciones Set 1 Macro 1
- General Equilibrium and Welfare Economics
- Ejercicios de Macroeconomía
- Auctions, best responses, reactions functions, Nash Equilibria
- Economic Growth
- Turbulence characteristics of a Fluid
- The Inflation Tax in RBC: A revision
- Demanda, Oferta y Mercado
- Impact of Income Distribution on Economic Growth, Rojas and Khor
- Lógica y Teoría de Conjuntos.
- Mary Trini Ejercicios Resueltos
- Statistics
- MANOVA
- MR Course Detail
- 03 Violacion Supuestos Multicolinealidad
- exx
- Principal Components Analysis
- MSc Statistics
- 4 Análisis de dos variables
- MATH203 Midterm 2012F
- CMJ 20 Stat Errors
- Vaccines and Death: a quantitative analysis of infant mortality
- Apuntes Estadística descriptiva.docx
- 1. Descripcion de Datos
- Semana 12 y 13 Prueba de Hipótesis[1]
- UNIDAD 1
- Informe Regresion Multiple Laboratorio
- Multiple Linear Regression
- Properties of Estimators New Update Spin
- Soda Comparison Lab1
- Estadistica III
- Math 540 Strayer Midterm Quiz (3 Different Quizzes)
- MB0050 B1700 Answer Key
- MS 8 final
- Lee (1992)
- Solutions Chapter 5
- Teoria Basica Del Muestreo - Monografias_com
- Stata Commands
- Applied Econometrics Using Stata
- RCU INTESHIP GUIDELINES 2016

You are on page 1of 10

**Jorge Rojas Freedom Fighter Seattle, WA, USA January 25, 2012
**

Abstract This is a summary containing the main ideas in the subject. This is not a summary of the lecture notes, this is a summary of ideas and basic concepts. The mathematical machinery is necessary, but the principles are much more important.

1

Linear Algebra

1. (AT )T = A 2. (A + B)T = AT + B T 3. (AB)T = B T AT 4. (cA)T = cAT ∀c ∈ R

Properties of Transpose

5. det(AT ) = det(A) 6. a · b = aT b =< a, b > (inner product) 7. This is important: If A has only real entries, then (AT A) is a positivesemideﬁnite matrix. 8. (AT )−1 = (A−1 )T 1

9. If A is a square matrix, then its eigenvalues are equal to the eigenvalues of its transpose. Notice that if A ∈ M(n×m) , then AAT is always symmetric. Properties of the Inverse 1. (A−1 )−1 = A

1 2. (kA)−1 = k A−1

∀k ∈ R \ {0}

**3. (AT )−1 = (A−1 )T 4. (AB)−1 = B −1 A−1 5. det(A−1 ) = [det(A)]−1 Properties of the trace 1. Deﬁnition. tr(A) =
**

n i=1

aii

2. tr(A + B) = tr(A) + tr(B) 3. tr(cA) = c · tr(A) ∀c ∈ R 4. tr(AB) = tr(BA) 5. Similarity invariant: tr(P −1 AP ) = tr(A) 6. Invariant under cyclic permutations: tr(ABCD) = tr(BCDA) = tr(CDAB) = tr(DABC) 7. tr(X ⊗ Y ) = tr(X) · tr(Y ) where ⊗ is the tensor product, also known as Kronecker product. 8. tr(XY ) =

i,j

**Xij · Yji matrices A ∈ M(m×n) and B ∈ M(p×q) · · · a1n B . .. . . . ··· amn B
**

mp×nq

The Kronecker product is deﬁned for as follows: a11 B . A⊗B = . . am1 B

2

**Properties of the Kronecker Product 1. (A ⊗ B)−1 = (A−1 ⊗ B −1 ) 2. If A ∈ M(m×m) and B ∈ M(n×n) , then: |A ⊗ B| = |A|n |B|m (A ⊗ B)T = AT ⊗ B T tr(A ⊗ B) = tr(A)tr(B) 3. (A ⊗ B)(C ⊗ D) = AC ⊗ BD
**

Careful! it doesn’t distribute with respect to the usual multiplication

Properties of Determinants

only deﬁned for A ∈ Mn×n

**1. det(aA) = an · det(A) ∀a ∈ R 2. det(−A) = (−1)n · det(A) 3. det(AB) = det(A) · det(B) 4. det(In ) = 1 5. det(A) =
**

1

det(A−1 )

6. det(BAB −1 ) = det(A) similarity transformation. 7. det(A) = det(AT ) ¯ 8. det(A) = det(A) the bar represents complex conjugate. Diﬀerentiation of Linear Transformations 1. 2. 3. 4. 5.

∂aT x ∂x ∂Ax ∂x

(matrices)

= =

∂xT a ∂x ∂xT AT ∂x

=a = AT

∂xT A ∂x

=A = (A + AT )x = abT x + baT x 3

∂xT Ax ∂x

∂aT xxT b ∂x

Diﬀerentiation of traces 1. 2. 3. 4.

∂tr(AX) ∂X

=

∂tr(XA) ∂X

= AT = (BA)T = (BX T CA)T + (CAXB)

∂tr(AXB) ∂X

=

∂tr(XBA) ∂X

∂tr(AXBX T C) ∂X ∂|X| ∂X

=

∂tr(XBX T CA) ∂X

= cof actor(X) = det(X) · (X −1 )T

2

Probability Distributions

Here, we could say that, starts the summary for Econometrics ECON581. Deﬁnition 2.1 Normal distribution: where µ is the mean and σ 2 is the variance. (x−µ)2 1 ∀x ∈ R f (x) = √ e− 2σ2 σ 2π If the mean is zero and the variance is one, then we have the standard normal distribution N (0, 1). The normal distribution has no closed form solution for its cumulative density function CDF. Deﬁnition 2.2 Chi-square Distribution: We say that χ2 has r degrees (r) of freedom.

r

Zi ∼ iidN (0, 1)∀i = 1, . . . , r =⇒ A =

i=1

Zi2 ∼ χ2 (r)

E(A) = r and V (A) = 2r Thus, the χ2 is just a square sum of standard normal distributions. We (r) use this distribution to test the value of the variance of a population. For instance, H0: σ 2 = 5 against H1: σ > 5

4

Deﬁnition 2.3 t-student Distribution: We say that t(r) has r degrees of freedom. The t-distribution has “fatter” tails than the standard normal distribution. Z ∼ t(r) Z ∼ N (0, 1) ∧ A ∼ χ2 ∧ (Z and A are independent) =⇒ T = (r) A/r E(T ) = 0 and V (T ) =

r r−2

The t distribution is an “appropriate” ratio of a standard normal and a χ2 (r) random variables. Deﬁnition 2.4 F Distribution: We say that F (r1 , r2 ) has r1 degrees of freedom in the numerator and r2 degrees of freedom in the denominator. A1 /r1 ∼ F (r1 , r2 ) A1 ∼ χ2 1 ) ∧A2 ∼ χ2 2 ) ∧(A1 and A2 are independent) =⇒ F = (r (r A2 /r2 We use the F distribution to test whether two variances are the same or not 2 2 2 2 after a structural break. For instance, H0: σ0 = σ1 against H1: σ0 > σ1 .

3

Probability Deﬁnitions

**Deﬁnition 3.1 The expected value of a continuos random variable is given by: E[X] =
**

Ω

xf (x)dx

(1)

The notation variable.

Ω

just means that Ω is the domain of the relevant random

**Deﬁnition 3.2 The variance of a continuos random variable is given by: V [X] = V ar[X] = E[(x − µ)2 ] =
**

Ω

(x − µ)2 f (x)dx

(2)

Deﬁnition 3.3 The covariance of two continuos random variables is given by: C[X, Y ] = Cov[X, Y ] = E[XY ] − E[X]E[Y ] (3) Notice that the covariance of a r.v. X with itself is its variance. In addition, if two random variables are independent, then its covariance is zero. The reverse is not necessarily true.

5

Some useful properties: 1. E(a + bX + cY ) = a + bE(X) + cE(Y ) 2. V (a + bX = cY ) = b2 V (X) + c2 V (Y ) + 2bcCov(X, Y ) 3. Cov(a1 + b1 X + c1 Y, a2 + b2 X + c2 Y ) = b1 b2 V (X) + c1 c2 V (Y ) + (b1 c2 + c1 b2 )Cov(X, Y ) 4. If Z = h(X, Y ), then E(Z) = EX [EY |X (Z|X)] Law of iterated expectations

4

Econometrics

A random variable is a real-valued function deﬁned over a Sample Space. The Sample Space (Ω) is the set of all possible outcomes. Before collecting the data (ex-ante) all our estimators are random variables. Once we have realized the data(ex-post), we get a speciﬁc number for our estimators. These numbers are what we called estimates. Remark 4.1 A simple Econometric Model: yi = µ+ei is not a regression model, but is an econometric one. ∀i = 1, . . . , n. This

In order to estimate µ we make the following assumptions: 1. E(ei ) = 0 ∀i 2. V ar(ei ) = E(e2 ) = σ 2 i ∀i

3. Cov(ei , ej ) = E(ei ej ) = 0 ∀i = j In a near future, we will further assume that the residual term follows a normal distribution with µ = 0 and variance σ 2 . This is not necessary for the estimation process, but we need to run some hypothesis tests. What we are looking for is for a line that ﬁts the data, minimising the distance between the ﬁtted line and the data. In other words, Ordinary Least Squares (OLS).

n

Min

i=1

(yi − µ)2 ⇔ Min SSR ⇔ ˆ

i 1 n n i=1

ei 2 ˆ

The estimator is then given by µ = ˆ

yi = y ¯

6

Deﬁnition 4.2 We say that an estimator is Unbiased if: E(ˆ) = µ µ In other words, if after inﬁnitely sampling we are able to achieve the true population value. For this particular estimator (ˆ) is easy to see that is indeed unbiased µ 1 2 and its variance is V ar(ˆ) = n σ , given the assumption that the draws are µ iid. Note 4.3 Linear combination of normal distribution is a normal distribution. Proposition 4.4 If µ ∼ N (µ, σ ), then Z = ˆ n

2

µ−µ ˆ√ σ/ n

∼ N (0, 1)

**Standard normal values: Φ(z ≥ 1.96) = 0.025 and Φ(z ≥ 1.64) = 0.05 Note: 1. 2.
**

e2 i σ2 ei 2 ˆ σ2

∼ χ2 n

2

σ = (n−1)ˆ ∼ χ2 We lose one degree of freedom here because we n−1 σ2 need to use one datum to estimate µ. ˆ µ−µ ˆ √ σ2 / n ˆ

3. When we do not know σ 2 our standardise variable is Z =

∼ t(n−1)

Hypothesis Testing

Reject H0 Reject H1

H0 is true H1 is true Type I error OK OK Type II error

Thus, we deﬁne the following probabilities: • P(Type I error) = P(Reject H0| H0 is true) = α • P(Type II error) = P(Fail to reject H0| H0 is false) = 1 − β and β is the so-called “power of the test”. Remark 4.5 Multiple Linear Regression (Population) yi = xi β + ei ∀i = 1, . . . , n vector notation

7

ASSUMPTIONS E(ei ) E(e2 ) i E(ei ej) ei = = = ∼ 0 ∀i σ 2 ∀i 0 ∀i = j N (0, σ 2 ) (4)

X variables are non-stochastic. There is NO exact linear relationship among X variables. If ei is not normal, we may apply the Central Limit Theorem (CLT). However, for this we need to have a large sample size. How large is large enough? 30 (n − K) is one number, but it will depend on the problem. OLS estimator results from minimising the SSE(error sum of squares)

n n

ˆ β=(

i=1

xi xi )−1

i=1

xi y i

(5)

The above estimator is useful if we are in “Asymptopia”. In matrix notation we have: y = Xβ + e e ∼ iid N (0, σ 2 In ) X is non-stochastic The OLS from the sample is: ˆ β = (X X)−1 X Y = β + (X X)−1 X e (6)

(7)

This mathematical form is useful to run analysis in the “ﬁnite sample” world. The OLS estimator is unbiased and its variance-covariance matrix is given by: ˆ ˆ ˆ ˆ ˆ Cov(β) = E[(β − E(β))(β − E(β)) ] (8) = E[(X X)−1 X ee X(X X)−1 ] = σ 2 (X X)−1 ˆ Thus, β ∼ N (β, σ 2 (X X)−1 ) 8

Deﬁnition 4.6 The matrix MX = In − X(X X)−1 X is symmetric and T idempotent, i.e., MX = MX and MX × MX = MX In general, we can have Mi = In − Xi (Xi Xi )−1 Xi . Thus, Mi Xj is interpreted as the residuals from regressing Xj on Xi . Note 4.7 The following properties are important for demonstrations: 1. If A is a square matrix, then A = CΛC −1 where Λ is a diagonal matrix with the eigenvalues of A, and C is the matrix of the eigenvectors in column form. 2. If A is symmetric, then C C = CC = In and hence A = CΛC 3. If A is symmetric and idempotent, then Λ is a diagonal matrix with either eigenvalues 1 or 0. 4. If A = CΛC , then rank(A)=r where r = n λi i=1 Using this deﬁnition we get that ˆˆ e e = e MX e and hence E(ˆ e) = σ 2 (n − K) eˆ Theorem 4.8 Gauss-Markov Theorem: In a linear regression model in which the errors have expectation zero and are uncorrelated and have equal variances, the best linear unbiased estimator (BLUE) of the coeﬃcients is given by the OLS estimator. Best means giving the lowest possible mean squared error of the estimate. Notice that the errors need not be normal, nor independent and identically distributed (only uncorrelated and homoscedastic). The proof for this theorem is based on supposing an estimator β ∗ = CY ˆ that is better than β and ﬁnding the related contradiction.

**¡¡¡Viva la Revoluci´n Libertaria!!! o
**

9

5

**One page Summary
**

Y ˆ β ˆ E(β) ˆ Cov(β) ˆ β

2

= = = = ∼

**Xβ + e (X X)−1 X Y β σ 2 (X X)−1 N (β, σ 2 (X X)−1 ) 1 1 ˆˆ ee= n−K n−K 2σ 4 (n − K)
**

n

σ ˆ

=

e2 i

i=1

E(ˆ 2 ) = σ 2 σ V ar(ˆ ) = σ

2

e e ∼ χ2 (n) ˆˆ ee (n − K)ˆ 2 σ = ∼ χ2 (n−K) 2 σ σ2 e ∼ N (0, In ) e ∼ N (0, σ 2 In ) ⇔ σ MX = In − X(X X)−1 X ˆ e = MX Y Theorem 5.1 Gauss-Markov Theorem: In a linear regression model in which the errors have expectation zero and are uncorrelated and have equal variances, the best linear unbiased estimator (BLUE) of the coeﬃcients is given by the OLS estimator. Best means giving the lowest possible mean squared error of the estimate. Notice that the errors need not be normal, nor independent and identically distributed (only uncorrelated and homoscedastic).

10

- Design and Analysis of Single Factor ExperimentsUploaded byCotadom
- Explaining Wage Differentials in Belgium - Mincerian Wage Model. An empirical Study.Uploaded byFranz Eigner
- How Citizens Integrate Information without Ideological Cues Local Weather and Americans’ Beliefs about Global WarmingUploaded byBogdan Ivascu
- Problem With LPMUploaded bytoilanhan1977
- Determinants of Corporate Profitability in Developing EconomiesUploaded byAlexander Decker
- Calibration by Linear Regression - TutorialUploaded bypatrickjanssen
- Spatial Prediction of Soil Properties Using EBLUP-Minasny & McBratney, 2007Uploaded byParag Jyoti Dutta
- Multi Col LinearityUploaded bynai09
- 2.6_Linear+Regression+with+One+Regressor+一元线性回归Uploaded byJames Jiang
- Riv Regress Post EstimationUploaded byJilani Osmane
- Racial Supermarket AccessUploaded byjustine lindemann
- Mini-mock Exam - Questions Only - 2.28.2018Uploaded byKeith Lo
- 1981 Trends in IdentificationUploaded byMohammed Abdul Hai
- 3447Uploaded bySohail Ahmad
- Exercises - Run TestUploaded byRohaila Rohani
- SPUploaded byjvb_buena2734
- Lampiran Output SpssUploaded byCoreHard
- Multiple Regression ReviewUploaded byakirank1
- Advertisement effectivenessUploaded byNikhil Hande
- 6 Chi-square AnalysisUploaded byJessica Vallely
- Impact of Human Capital Investment on BankUploaded byoptimistic07

- Trade, Unemployment and InequalityUploaded byJorge Rojas-Vallejos
- Soluciones Ejercicios de Macro 1Uploaded byJorge Rojas-Vallejos
- Happiness and Clean AirUploaded byJorge Rojas-Vallejos
- Macroeconomic Theory NotesUploaded byJorge Rojas-Vallejos
- OLS estimators EconometricsUploaded byJorge Rojas-Vallejos
- Soluciones Set 1 Macro 1Uploaded byJorge Rojas-Vallejos
- General Equilibrium and Welfare EconomicsUploaded byJorge Rojas-Vallejos
- Ejercicios de MacroeconomíaUploaded byJorge Rojas-Vallejos
- Auctions, best responses, reactions functions, Nash EquilibriaUploaded byJorge Rojas-Vallejos
- Economic GrowthUploaded byJorge Rojas-Vallejos
- Turbulence characteristics of a FluidUploaded byJorge Rojas-Vallejos
- The Inflation Tax in RBC: A revisionUploaded byJorge Rojas-Vallejos
- Demanda, Oferta y MercadoUploaded byJorge Rojas-Vallejos
- Impact of Income Distribution on Economic Growth, Rojas and KhorUploaded byJorge Rojas-Vallejos
- Lógica y Teoría de Conjuntos.Uploaded byJorge Rojas-Vallejos

- Mary Trini Ejercicios ResueltosUploaded byPiero Heredia
- StatisticsUploaded byVirencarpediem
- MANOVAUploaded byVishnu Prakash Singh
- MR Course DetailUploaded byapi-3841013
- 03 Violacion Supuestos MulticolinealidadUploaded byJose Luis LabraCh
- exxUploaded byaronl10
- Principal Components AnalysisUploaded bysupermanvix
- MSc StatisticsUploaded bySilambu Silambarasan
- 4 Análisis de dos variablesUploaded byPedro J López Cabello
- MATH203 Midterm 2012FUploaded byexamkiller
- CMJ 20 Stat ErrorsUploaded byJasna Pavlovic
- Vaccines and Death: a quantitative analysis of infant mortalityUploaded bybiko97jcj
- Apuntes Estadística descriptiva.docxUploaded byioki3
- 1. Descripcion de DatosUploaded byespinozaisac
- Semana 12 y 13 Prueba de Hipótesis[1]Uploaded byTom Dc
- UNIDAD 1Uploaded byCinthia Chiñas de Martinez
- Informe Regresion Multiple LaboratorioUploaded byOzkar Martinez
- Multiple Linear RegressionUploaded byngrckr
- Properties of Estimators New Update SpinUploaded byNi Sien
- Soda Comparison Lab1Uploaded byAlejandra Zepeda
- Estadistica IIIUploaded byElliot Medina Luján
- Math 540 Strayer Midterm Quiz (3 Different Quizzes)Uploaded byGaryoFrobon
- MB0050 B1700 Answer KeyUploaded byjmshetty
- MS 8 finalUploaded byDeepak Pandey
- Lee (1992)Uploaded byFranco Patricio Perez Ojeda
- Solutions Chapter 5Uploaded byYoussef Afkir
- Teoria Basica Del Muestreo - Monografias_comUploaded byFabiola Montoya Gonzales
- Stata CommandsUploaded byNdivhuho Neosta
- Applied Econometrics Using StataUploaded bynamedforever
- RCU INTESHIP GUIDELINES 2016Uploaded byLaxman Mittal