0 Up votes0 Down votes

18 views160 pagestis

Mar 15, 2014

© Attribution Non-Commercial (BY-NC)

PPT, PDF, TXT or read online from Scribd

tis

Attribution Non-Commercial (BY-NC)

18 views

tis

Attribution Non-Commercial (BY-NC)

- Chapter 009
- Multiple Regression
- Standard Deviation Practice
- GRM: Generalized Regression Model for Clustering Linear Sequences
- PROMIS Social Isolation Scoring Manual
- Pooled Time Series
- 1171924
- Globalization And
- cherry_30 (1)
- Buffet Food Purchase Background
- Public and Private Universities in Bangladesh
- Ch 14 Regression Inference
- SBE10_07a [Read-Only] [Compatibility Mode]
- Reversal of Fortune Geography and Institutions in the Making of the Modern World Income Distribution
- abstract
- In d 44306116
- Advanced Regression 1
- Lecture6 Module2 Anova 1
- Multiple Regression
- output spss.doc

You are on page 1of 160

Managerial Economics

Dr. Manoj Kumar Dash

M.A; M.Phil; MBA; Ph.D

What is Econometrics?

Simply, Econometrics, means economic

measurement.

Some formal definitions:

Econometrics is concerned with the empirical

determination/quantification of theoretical

postulations economics, management, political

science etc.

Econometrics is defined as the social science in

which the tools of economic theory, mathematics,

and statistical inference are applied to the

analysis of economic phenomena (Goldberger,

1964).

Econometrics .. consists of the application of

mathematical statistics to economic data to lend

empirical support to the models constructed by

mathematical economics and to obtain numerical

results (Gerhard Tintner, 1968).

Econometrics renders a positive help in trying

to dispel the poor public image of economics

as a subject in which empty boxes are opened by

assuming the existence of can-openers to reveal

contents which any ten economists will interpret

in 11 ways.

Examples

Consumption expenditure = F (Income, Wealth

etc)

Hourly earnings = F (Education, Labour

productivity etc)

Quantity Demanded = F (Price, Income of

consumer, Prices of relative commodities etc)

Election Outcome = F (Economic performance,

Populism etc)

Debt/Equity = F (Corporate tax rate, Capital

gains tax rate, Inflation rate etc)

Sales = F (Price, Advertising expenses etc)

Crop yield = F (Temperature, Rainfall, Fertilizer

use etc.)

Why Study Econometrics?

(1)To provide empirical support to theories:

Theories make statements or hypotheses that are

mostly qualitative in nature.

They do not provide numerical measure of the

proposed ideas.

Example: Law of Demand in Economics

In a given situation how to quantify this law?

Econometrics comes to the rescue.

(2)To provide empirical support to mathematical

models:

Many theories are expressed in mathematical

form/models (Common in Economics, Finance)

These theoretical models are developed without

regard to their measurability

Econometrics helps to put these models into

empirical testing (e.g. Tax competition models).

(3) For academicians:

Research in economics, finance, management,

marketing etc is becoming increasingly

quantitative [Show examples].

Hence, knowledge about econometrics is very

important to conduct empirical research.

(4) For students:

For students a command over econometrics will

be of great use in their employment.

Example: forecasting of sales, consumer

behaviour (KPOs) corporate research.

Steps in Econometric Analysis

1. Identify research issue

2. Select Variables

3. Check Data Availability

4. Specify Econometric Model

5. Data Collection

6. Model Estimation

7. Hypothesis Testing

8.Using Estimated Model for

Forecasting/Prediction

Step 1: Identify research issue

You should have a research problem to probe

(How to identify a problem?)

Example from Labour Economics: Effect of

economic conditions on peoples willingness to

work (or LFPR).

Two hypothesises/arguments are postulated.

Argument 1 (Discouraged worker hypothesis): As

economic conditions worsen, unemployed may

drop out of labour force (falling LFPR) as they

loose hope of finding a job.

Argument 2 (Added worker hypothesis): As

economic conditions worsen, unemployed may

join labour force (rising LFPR) if the main bread

winner in the family loses job.

In this context, our research objective could be

to test relative strength/validity of these two

contrasting claims.

Step 2: Select Variables

For running a regression, we have to select a

DEPENDENT variable and an INDEPENDENT

variable(s).

A model with one dependent variable and one

independent variable is called simple or two

variable regression model (TWRM).

A model with one dependent variable and more

than one independent variable is called multiple

regression model (MRM).

Example:

LFPR = f (Unemployment Rate) - TWRM

LFPR = f (Unemployment Rate; Hourly

Earnings; Family Wealth) MRM

Generally, MRM is preferred because it

enhances credibility of research findings.

How many independent variables do we include?

Strictly, guiding principle should be underlying

theory or prior literature or nature of problem.

Even then, it may not always possible to include

every possible variable

Why?

Non-availability of data

Inherent randomness in human behaviour, which cannot

be captured easily (Ex: Being unemployed due to

attitude problem).

Inability to properly quantify certain variables (Ex.

Interest groups)

Due to ignorance we may miss relevant variables

Anyway, ultimate purpose of econometrics is not to

capture complete reality

How to rectify this problem?

Add a term called error term which could take

care of influence of all omitted variables.

In other words, error term captures all those

forces that affect the dependent variable but are

not explicitly included in the model.

Error term will also be useful when we use proxy

independent variables (e.g. interest groups)

If the error term is small it implies that the

combined influence of omitted variables is

small/negligible.

Step 3: Check Data Availability

Before proceeding further, check data availability

for relevant variables.

Three types of data are generally available for

empirical analysis Cross-sectional, Time-

series, Pooled Data/Panel Data.

In cross-section data, values of one or more

variables are collected for several sample units,

or entities at the same point in time (e.g. GDP

figure of countries for a given year).

Cross-Sectional data

States Tax Revenue in 2000-01 (in crores)

Andhra Pradesh 10551.92

Bihar 2934.75

Haryana 4311.48

Karnataka 9042.67

Kerala 5870.26

Maharashtra 19724.28

Orissa 2184.03

Punjab 4895.22

Rajasthan 5299.97

In time series data we observe the values of one

or more variables over a period of time daily

stock prices or annual GDP figures

In panel data, same cross-sectional unit (say a

firm or state or country) is surveyed over time

Thus, in panel data we have elements of both

time series and cross-sectional data.

Time Series Data

Year Tax Revenue of Andhra Pradesh

1990-91 2647.25

1991-92 3054.96

1992-93 3388.72

1993-94 3832.93

1994-95 4227.43

1995-96 4120.44

1996-97 4881.83

1997-98 7113.55

1998-99 7961.4

1999-00 9008.6

2000-01 10551.92

Panel Data: Tax Revenues of States

Year AP KAR KER TN

1990-91 2647.25 2332.12 1340.35 3124.05

1991-92 3054.96 2900.2 1673.93 3734.11

1992-93 3388.72 3097.81 1886.97 4162.06

1993-94 3832.93 3812.34 2344.87 4801.37

1994-95 4227.43 4289.31 2799.1 5833.76

1995-96 4120.44 5273.93 3382.68 7151.2

1996-97 4881.83 5767.84 3898.5 7983.45

1997-98 7113.55 6411.87 4501.05 8682.64

1998-99 7961.4 6943.1 4649.5 9625.3

1999-00 9008.6 7744.37 5193.51 10918.93

2000-01 10551.92 9042.67 5870.26 12282.25

Step 4: Specify Econometric Model

LFPR = B

1

+ B

2

UR +u

LFPR Labour force participation rate

UR Unemployment rate (a proxy for economic

condition. Other option could be GDP)

B

1

- Intercept term. Gives value of LFPR (dependent

variable) when UR (value of independent variable) is

zero

B

2

- Slope term. Measures the rate of change in LFPR

for a unit change in UR. Together B

1

and B

2

are

known as the parameters of the regression model

u Error term [LFPR (B

1

+ B

2

UR)]

This is a linear regression model - LFPR is linearly

related to UR

Our objective is to explain the behaviour of dependent

variable in relation to the explanatory variable.

Step 5: Data Collection

For empirical estimation, we need to collect data

on the variables used in the econometric model.

Data can be obtained either from primary sources

[HR] or from secondary sources [Finance,

Economics]

Step 6: Model Estimation

By estimation we mean estimating the parameters

(B

1

& B

2

) of the chosen model.

Estimation is carried out using the technique of

regression analysis.

What is regression analysis?

Regression analysis is concerned with the

study of the dependence of one variable, the

dependent variable, on one or more other

variables, the explanatory variables, with a view

to estimating and/or predicting the (population)

mean or average value of the former in terms of

the known or fixed (in repeated sampling)

values of the latter

Assume that after estimation we get the following

result:

LFPR = 50.63 - 0.45UR

B

2

= 0.45. Implies that if UR goes up by 1%

point, ceteris paribus, LFPR is expected to

decrease on the average by about 0.45% points

Discouraged worker hypothesis finds support.

B

1

= 50.63. Implies that average value of LFRP

will be about 50.63% if the UR were zero (i.e.

full employment).

Sometimes/often, intercept term has no

particular economic meaning.

But, in the present example it has meaning

(How?)

We say on the average because the presence

of error term is likely to make the relationship

somewhat imprecise.

Step 7: Hypothesis Testing

Objective here is to test certain hypotheses

suggested by theory and/or prior empirical

experience on parameters of the model.

Specifically, we are interested to verify how

close the estimated parameter is to a pre-

supposed value of that parameter

(e.g.

) 45 . 0

; 1 :

1 1 0

= ( B B H

Hypothesis testing helps to verify whether the

results obtained through regression analysis

conform to the underlying theory

Step 8: Using Estimated Model for

Forecasting/Prediction

LFPR = 50.6333 - 0.4486UR

If we want to predict LFPR in some future

periods for a given value of UR (say 5) we can

obtain it (How?)

Substitute UR value (5) in the above equation

When data on LFPR (for a given UR) for

future period is out, we can compare the

predicted value with the actual value.

The discrepancy between the two is called the

prediction error.

Regression Vs. Correlation

Regression Correlation

Estimate/predict average value of

one variable on the basis of the

fixed values of other variables

Measure the strength or degree of

linear association between two

variables

The dependent variable is treated

as stochastic or random

The explanatory variables are

assumed to have fixed values

Both the variables are treated as

random

Our Immediate Task

Identification of research issue, selection of

variables, checking of data availability,

specification of econometric model and data

collection are not big tasks

Hence, our focus would be on

Model Estimation

Hypothesis Testing

Criticisms

By and large, events can be explained without

econometric analysis

Data mining Results are created!

Problem with intercept term

Two-Variable Regression Model

Meaning of the term Linear

Term linear can be interpreted in 2 ways:

Linearity in the variables and linearity in the

parameters.

Linear regression always means a regression

that is linear in the parameters. It may or may not

be linear in the explanatory variables

Example:

Y = B

1

+ B

2

X +u (or)

Y = B

1

+ B

2

X

2

+u

Linear Regression Models

Model linear in

parameters?

Model linear in variables?

Yes No

Yes LRM LRM

No NLRM NLRM

Population Regression Function (PRF)

Let us consider an example of the law of demand.

The demand schedule for commodity x

Price

(X)

Quantity Demanded (Y) No. of

Consumers

Average

demand

1 45,46,47,48,49,50,51 7 48

2 44,45,46,47,48 5 46

3 40,42,44,46,48 5 44

4 35,38,42,44,46,47 6 42

5 36,39,40,42,43 5 40

6 32,35,37,38,39,42,43 7 38

7 32,34,36,38,40 5 36

8 31,32,33,34,35,36,37 7 34

9 28,30,32,34,36 5 32

10 29,30,31 3 30

Total 55

Figure 1: Population Regression Line

0

10

20

30

40

50

60

0 1 2 3 4 5 6 7 8 9 10 11 12

Price

Q

u

a

n

t

i

t

y

D

e

m

a

n

d

e

d

The impression we get from scattergram is that

demand (Y) decreases as price (X) increases, and

vice versa.

The downward slopping line is called Population

Regression Line (PRL).

It is nothing but the locus of conditional means of

the dependent variable for the fixed values of the

explanatory variable(s)

Thus, PRL gives the average value of the

dependent variable corresponding to each value

of the independent variable.

The point on the PRL represents expected or

population mean value of Y corresponding to

the various Xs.

The adjective population comes as our example

deals with entire population of 55 consumers.

PRL can be expressed in following functional

form:

E(Y/X

i

) = B

1

+B

2

X

i

(1)

where i is ith subpopulation

Eq (1) gives average value of Y corresponding to

each value of X and is called Population

Regression Function (PRF) or non-stochastic

PRF.

In regression analysis our interest is in

estimating the PRFs (i.e. B

1

and B

2

) on the basis

of observations on Y and X.

Stochastic Specification of the PRF

How to explain the demands of the individual

consumer in relation to price?

The best we can do is to say that any

individuals demand is equal to the average for

that group plus or minus some quantity.

D

e

m

a

n

d

O

2

7

X

36

u

32

Price

u

46

Y

48

We can express the deviation of an individual Y

i

around its expected value as follows:

Y

i

= E(Y/X

i

) + u

i

(2)

In Eq. (2), u

i

is an unobservable random variable

called stochastic error term - taking positive or

negative values.

Now, by substituting Eq. (1) in (2), we get

Y

i

= B

1

+B

2

X

i

+u

i

(3)

Eq. (3) is called stochastic PRF (SPRF), whereas

Eq. (1) is called non-stochastic PRF (NPRF).

NPRF represents means of various Y values

corresponding to specified prices.

SPRF tells us how individual demands vary around

their mean values due to presence of stochastic error

term, u.

How to interpret SPRF (Eq.3)?

We can say that demand of an individual consumer

(say i) corresponding to a specific price can be

expressed as sum of following 2 components:

(i) Systematic/deterministic component:

B

1

+B

2

X

i

(Nothing but average quantity demanded

by all the consumers at a given price level X

i

)

(ii) Nonsystematic/random component: u

i

(Determined by factors other than price) [See

Figure]

D

e

m

a

n

d

O

2

7

X

36

u

32

Price

u

46

Y

48

Sample Regression Function (SRF)

If we have data on whole population (like in

following Table) arriving at PRF is an

easy/straightforward exercise

That is, find conditional means of Y

corresponding to each X and then join these

means

Unfortunately, in practice, we rarely have entire

population at our disposal.

The demand schedule for commodity x

Price

(X)

Quantity Demanded (Y) No. of

Consumers

Average

demand

1 45,46,47,48,49,50,51 7 48

2 44,45,46,47,48 5 46

3 40,42,44,46,48 5 44

4 35,38,42,44,46,47 6 42

5 36,39,40,42,43 5 40

6 32,35,37,38,39,42,43 7 38

7 32,34,36,38,40 5 36

8 31,32,33,34,35,36,37 7 34

9 28,30,32,34,36 5 32

10 29,30,31 3 30

Total 55

We only have a sample from the population.

The following is an example from our case

Sample 1

Y (Demand) X (Price)

49 1

45 2

44 3

39 4

38 5

37 6

34 7

33 8

30 9

29 10

Hence, our task is to estimate the PRF on the basis of

sample information

[OR]

Task is to estimate average quantity demanded in the

population as a whole corresponding to each X (price)

from sample data such as above

But, we may not be able to estimate the PRF accurately

because of sampling error

To see this clearly,

suppose another random

sample (Sample 2) is

drawn from the population

of above Table.

If we plot the data of these

two samples, and entire

population we may obtain

corresponding SRLs and

PRL as follows

Y (Demand) X (Price)

51 1

47 2

46 3

42 4

40 5

37 6

36 7

35 8

32 9

30 10

D

e

m

a

n

d

Y

X

SRL

1

SRL

2

PRL

Price

Now the question is: which of the two SRLs

represents the true PRL?

If we avoid temptation of looking at above

figure, which represents the PRL, there is no way

we can be sure that either of the SRLs represents

the true PRL

In general, we get K different SRLs for K

different samples and all these SRLs are not

likely to be the same

Now, analogous to PRF that underlies the PRL,

we can develop SRF to represent SRL as

follows:

= estimator of E(Y/X

i

)

b

1

= estimator of B

1

b

2

= estimator of B

2

Where is read as Y-hat or Y-cap

Y

) 4 (

2 1 i i

X b b Y + =

i

Y

how to estimate the population parameter at

hand.

A particular numerical value obtained by the

estimator is an estimate.

Now, we can express SRF (Eq.4) in its

stochastic form as follows:

= b

1

+ b

2

X

i

+ e

i

(5)

Where e

i

is the estimator of u

i

It is analogous to u

i

and is introduced for same

reasons as u

i

was introduced in the PRF

i

Y

To sum up, our primary objective in regression

analysis is to estimate the stochastic PRF (Eq.3)

on the basis of stochastic SRF (Eq.5) because

more often than not our analysis is based on a

single sample from some population.

But, because of sampling variation, our estimate

of PRF based on SRF is only approximate.

D

e

m

a

n

d

Y

X

PRF

SRF

Price

Error

Error

Granted that SRF is only an approximation of the PRF

the question is:

Can we find a method or a procedure that will make

this approximation as close as possible?

[OR]

How should we construct the SRF so that b

1

is as close

as possible to B

1

and b

2

is as close as possible to B

2

?

This can be done by adopting the method of Ordinary

Least Squares (OLS).

What is OLS method?

It chooses SRF or b

1

and b

2

in such a way that that

the sum of the squares of the residuals is as small

as possible.

Symbolically,

Minimize

Where Y

i

= actual Y value

= estimated/predicted Y value

In this way, SRF is made as close as possible to

PRF

( )

=

2

2

i i i

Y Y e

i

Y

Due to two reasons.

Reason 1: To give different weightage to

residuals according to the extent of their

closeness to SRF

Example: e

1

= 10, e

2

= -2, e

3

= +2 and e

4

= -10

By squaring e

i

we can give more weightage to

large errors (10) comparing others (2)

2

i

e

i

e

Reason 2: This procedure avoids the problem of

sign of the residuals which can be positive as

well as negative, and therefore can add to zero.

How to select b

1

and b

2

values to minimise ?

2

i

e

Which is nothing but

Hence, = f (b

1

, b

2

)

Hence, for any given set of data, choosing

different values of b

1

and b

2

will give different

e

i

s and hence different values of

( )

2

2

i i i

e Y Y =

( )

2

2 1

i i

X b b Y

2

i

e

2

i

e

4 1 2.929 1.071 1.147 4 0 0

5 4 7.000 -2.000 4.000 7 -2 4

7 5 8.357 -1.357 1.841 8 -1 1

12 6 9.714 2.286 5.226 9 3 9

Sum:28 16 0 12.214 0 14

An Example:

i

Y i

X

i

Y

1

i

e

1

i e 1

2

i

Y

2

i

e

2

i e 2

2

Notes:

Now which sets of estimated b (parameter)

values should we choose?

Since b values of 1

st

experiment gives us lower

than that obtained from b values of 2

nd

experiment,

we say bs of first experiment are best values.

( ) 357 . 1 ; 572 . 1 357 . 1 572 . 1

2 1 1

= = + = b b X Y

i i

) 1 ; 3 ( 1 3

2 1 2

= = + = b b X Y

i i

)

1 1 i i i

Y Y e =

( )

i i i

Y Y e

2 2

=

2

i

e

But how do we ascertain this?

We still can choose many more values for bs

that gives us the least possible value of

However, in doing so we must be sure that we

have considered all the conceivable values of b

1

and b

2

If we have infinite time and patience we can do

this exercise

2

i

e

But, fortunately, OLS method chooses b

1

and b

2

in such a manner that, for a given set of data,

is as small as possible.

[OR]

For a given sample, OLS method provides us

with unique estimates of b

1

and b

2

that give the

smallest possible values of

2

i

e

2

i

e

How do we accomplish this?

This is a straight-forward exercise in differential calculus.

Values of b

1

and b

2

that minimize are obtained by

solving the following two simultaneous equations:

These simultaneous equations are known as least squares

normal equations.

2

i

e

+ =

i i

X b nb Y

2 1

+ =

2

2 1 i i i i

X b X b X Y

In above equations n is sample size

Unknowns are bs. Knowns are quantities

involving sums, sum of squares, and sums of

cross products of the variables Y and X

The knowns can be obtained from sample at

hand

Solving these equations simultaneously, we

obtain following solutions for b

1

and b

2

:

Where

- Mean of Y variable

- Mean of X variable

- Deviation from sample mean values

- OLS estimators

X b Y b

2 1

=

( )( )

( )

=

2

2

X X

Y Y X X

b

i

i i

Y

X

Y Y X X

i i

,

2 1

, b b

Example:

YX

Y X

49 1 -4.5 11.2 20.25 -50.4

45 2 -3.5 7.2 12.25 -25.2

44 3 -2.5 6.2 6.25 -15.5

39 4 -1.5 1.2 2.25 -1.8

38 5 -0.5 0.2 0.25 -0.1

37 6 0.5 -0.8 0.25 -0.4

34 7 1.5 -3.8 2.25 -5.7

33 8 2.5 -4.8 6.25 -12

30 9 3.5 -7.8 12.25 -27.3

29 10 4.5 -8.8 20.25 -39.6

Mean = 37.8 Mean = 5.5

82.5 -178

X X

i

Y Y

i

2

) ( X X

i

( )( ) Y Y X X

i i

Using the above formulas we obtain estimates

of b

1

and b

2

as follows

b

2

= -2.1576

b

1

= 49.667

The results can be obtained using regression

packages without much effort (Demonstration)

Properties of OLS estimators

OLS estimators b

1

and b

2

satisfy the BLUE

Best Linear Unbiased Estimator property

Linearity:

Estimators are a linear function of the dependent

and independent variables

A linear estimator is much easier to deal with

than a nonlinear estimator

Unbiasedness:

Average or expected value of the estimators is

equal to true/population value

Minimum variance/Efficiency:

It has minimum variance in the class of all such

linear unbiased estimators

Smaller the variance of b

1

or b

2

, the closer they

will be to true B

1

or B

2

Implication: Regression coefficient estimated by

OLS on average coincides with population/true

value

Assumptions Underlying OLS

Assumption 1:

The regression model is linear in parameters (Bs)

Assumption 2:

Explanatory variables (Xs) are fixed in repeated

sampling

Implication: Changes in Y is conditional on the

given values of X

Example:

The demand schedule for commodity x

Price (X) Quantity Demanded (Y)

1 45,46,47,48,49,50,51

2 44,45,46,47,48

3 40,42,44,46,48

4 35,38,42,44,46,47

5 36,39,40,42,43

6 32,35,37,38,39,42,43

7 32,34,36,38,40

8 31,32,33,34,35,36,37

9 28,30,32,34,36

10 29,30,31

Assumption 3:

Mean value of disturbance u

i

is zero

Because positive u

i

values cancel out negative u

i

values (see figure)

Implication: factors not explicitly included in the

model (i.e. u

i

) dont systematically affect mean

of Y (or) u

i

s average effect on Y is zero

X

X

1

X

2

X

3

PRF

+ u

i

- u

i

Mean

.

.

.

.

.

.

Assumption 4:

The variance of u

i

is same for all observations

(Xs) [Homoscedasticity or equal (homo) spread

(scedasticity)]

Implication: Variation around regression line of

individual Y values remains same regardless of

values taken by Xs; it neither increases or

decreases as X varies

Hence all Y values corresponding to the various

Xs are equally important.

Violation of this assumption (i.e. increase in variation

around regression line of Y values as X increases) is

called heteroscedasticity

Example for homoscedasticity: Richer families on

average consume more than poorer families, but there

is no/not much variability in consumption pattern

between richer and poorer families

Example for heteroscedasticity: There is greater

variability in consumption pattern of richer families

compared to poorer ones becoz. as income grows

people have more consumption choice.

Assumption 5:

No autocorrelation between error terms

(Implications for time series data)

Example for autocorrelation:

Y

t

= B

1

+B

2

X

t

+u

t

, where u

t

and u

t-1

are positively

correlated. Here, Y

t

depends not only on X

t

, but

also on u

t-1

for u

t-1

to some extent determined u

t

By invoking no autocorrelation assumption, we

consider only the effect on X

t

on Y

t

and not

worry about other influences that might act on Y

as a result of possible inter-correlations among

us

Assumption 6:

No correlation between u and explanatory

variables (Xs)

Implication: If X and u are correlated, it is not

possible to assess their individual effects on Y

If X and u are positively correlated, X increases

when u increases and it decreases when u

decreases.

Assumption 7:

Number of observations n must be greater than the

number of parameters to be estimated or explanatory

variables.

Assumption 8:

X values in given sample must not all be the same

(Applies to Y as well)

If so, X

i

= and hence denominator of estimator b

2

will be zero.

This makes it impossible to estimate b

2

and b

1

X

Assumption 9:

The regression model is correctly specified

Omission of important variables or inclusion of wrong

variables undermines the validity of regression exercise

Theory should be the guiding principle in building

econometric model

If theory is not clear, we have to use some judgment in

choosing the model and interpreting the results (e.g. tax

competition)

But, data mining should be avoided.

Assumption 10:

There are no perfect linear relationships among Xs

[No multicollinearity]

Important with respect to multiple regression models

Implication: In the presence of multicollinearity, we

cannot assess the separate influence of Xs on Y.

All these assumptions pertain to PRF only and not

SRF. This means that SRF may not always duplicate

all these assumptions (Example: presence of

autocorrelation and multicollinearity problems).

Coefficient of Determination (r

2

)

This is a measure of goodness of fit of the (sample)

regression line to a given set of data

[OR]

It is a summary measure that tells how well SRF fits

given data

r

2

measures % of total variation in Y explained by

regression model or X(s).

A perfect fit of regression line is rarely the case

Generally, there will be some positive and

negative errors

Our goal is to minimize the errors as far as

possible

See Figure: If all the observations were to lie on

the regression line, we would obtain a perfect

fit.

Y

X

X

1

X

2

X

3

X

4

SRF

Y

i

1

u

2

u

3

u

4

u

i

1 2 i

Y X =| +|

The concept of coefficient of determination can

be explained using the following diagram

Y

O

X

i

X

Y

i

Y

Total= (Yi - )

Y

SRF

e

i

= Due to residual

( - ) = Due to

regression/Explained by X

(Why?)

i

Y

Y

i

Y

- Mean of the sample data

- Predicted value of Y for a given X (Point on

SRF)

- An individual sample observation

Numerical proof (Consider following

example)

( ) ( ) ( )

i i i i

Y Y Y Y Y Y

+ =

i

Y

Y

i

Y

Y X

49 1

45 2

44 3

39 4

38 5

37 6

34 7

33 8

30 9

29 10

= 37.8

Y

For the above data b

2

= -2.1576; b

1

= 49.667

Hence = 49.667 2.1576X

i

For X

i

= 1, = 47.509

Now applying for Y

i

= 49 we

get

49-37.8 = (47.509-37.8) + (49-47.509)

11.2 = 11.2, i.e. LHS = RHS

i

Y

i

Y

( ) ( ) ( )

i i i i

Y Y Y Y Y Y

+ =

By squaring the above identity and summing

them we get,

- Called TSS (Total variation in Y)

- Called ESS (Variation due to X)

- Called RSS (Variation due to error)

( ) ( ) ( )

2 2

2

i i i i

Y Y Y Y Y Y + =

2

) ( Y Y

i

( )

2

Y Y

i

( )

2

2

i i i

e or Y Y

Thus, TSS = ESS + RSS

If all actual Ys lie on fitted SRF, RSS=0 and hence

ESS=TSS (Polar cases)

If X explains no variation in Y, ESS=0 and hence

RSS=TSS (Polar cases)

If ESS is relatively larger than RSS, then the chosen

SRF fits the data well

If RSS is relatively larger than ESS, then the chosen

SRF fits the data poorly

Now, r

2

is defined as

r

2

=

r

2

=

This is nothing but portion of variation in Y (TSS)

explained by X (ESS)

TSS

ESS

( )

( )

2

2

Y Y

Y Y

i

i

Properties of r

2

It is a nonnegative quantity (Why?)

Its limits are 0s r

2

s1

An r

2

of 1 means a perfect fit, that is, for

each i.

An r

2

of zero means that there is no relationship

between Y and X

i i

Y Y =

demanded as dependent variable and price

independent variable an r

2

value of 0.975

implies that price variable explains about 98%

of variation in quantity demanded. In this case,

we can say that sample regression gives an

excellent fit.

Coefficient of correlation (r)

It measures degree of linear association

between Y and X

It is nothing but

In practice r is of little importance

The more meaningful quantity is r

2

(Why?)

r is also called simple correlation coefficient or

correlation coefficient of zero order

2

r

Interpretation of r: r

12

means correlation

between variable 1 (say Y) and variable 2 (say

X

2

)

Standard Error (SE) of Regression Coefficients

We know that least-squares estimates (b

1

and b

2

)

are estimated using sample data

But since data are likely to change from sample

to sample, the estimates will change as well

Therefore, what is needed is some measure of

reliability of OLS estimators

The precision of an estimate or regression

coefficient is measured by SE

SE is the standard deviation (positive square root

of variance) of sampling distribution (SD) of the

estimator (say b

2

)

SD of an estimator is a distribution of set of

values of estimator obtained from all possible

samples of same size from a given population.

Thus, SE of an estimator is the amount it varies

across samples.

SEs of OLS estimates can be obtained as

follows:

Where var = variance; se = standard error and

is homoscedastic variance of u

i

(Assumption 4)

=

2

2

2

) var(

i

x

b

o

=

2

2

) (

i

x

b se

o

2

2

2

1

) var( o

=

i

i

x n

X

b

o

=

2

2

1

) (

i

i

x n

X

b se

2

o

2

2

2

=

n

u

i

o

Here the variance of b

2

is inversely proportional to

That is, given , larger the variation in X values, the

smaller the variance of b

2

and hence greater the

precision with which b

2

can be estimated.

In short, if there is substantial variation in Xs (recall

Assumption 8), b

2

can be measured more accurately

than when Xs do not vary substantially.

2

i

x

2

o

Hence, what is a big SE of regression

coefficients and what is a small SE depends on

the context (i.e. variation in Xs)

A more standardized statistic, which also gives a

measure of the goodness of fit of estimated

equation is R

2

SEs of regression coefficients can be used for

hypothesis testing and constructing confidence

intervals (discussed later)

Standard Error of Regression/Residuals

SE of regression is the standard deviation

(Positive square root of variance) of individual Y

values about the estimated regression line or

error term

If SE of residuals is high, then deviation will also

be high and hence fitness will be poor

SE of residuals can be obtained using the

following formula

2

2 2

2

2

=

n

x b y

i i

o

Multiple Regression Analysis

Meaning

In single/two variable model there is only one

explanatory variable

In practice, most problems cant be explained by

this model

Example: Apart from prices, demand is a function

of many other factors

Hence, we use multiple regression models which

contain more than one Xs

How the model looks like?

PRF for cross sectional data

PRF for time-series data

Any individual Y value can be expressed as

sum of 2 components

Deterministic component [E (Y

i

)]

Random component [ ]

i i i i

U X B X B B Y + + + =

3 3 2 2 1

t t t t

U X B X B B Y + + + =

3 3 2 2 1

i i

X B X B B

3 3 2 2 1

+ +

i

U

Assumptions

All the assumptions of two-variable

model are applicable in the case of

multiple regression as well

Interpreting Multiple Regression

Eq.

It gives conditional mean value of Y

conditional upon the given/fixed values of Xs

Symbolically

Thus, what we obtain is the mean value of Y for

the given values of Xs

( )

i i i i i

X B X B B X X Y E

3 3 2 2 1 3 2

, + + =

Partial Regression Coefficients

(PRCs)

In multiple regression B

2

&B

3

are called PRCs

B

2

measures change in the mean value of Y per

unit change in X

2

, holding value of X

3

constant

[OR]

B

2

gives direct or net effect of a unit change

in X

2

on E(Y), net of any effect that X

3

may have

on mean Y

Similar explanation is applicable for B

2

as well

To estimate parameters of model we use OLS

method

Let SRF corresponding to PRF described above

as:

Where b

1

, b

2

& b

3

are estimators of unknown

population coefficients B

1

, B

2

& B

3

respectively

e

i

is sample counterpart of residual term U

i i i i

e X b X b b Y + + + =

3 3 2 2 1

Estimating PRCs

OLS principle chooses values of unknown

parameters in such a way that the RSS is as

small as possible

Symbolically,

Minimization of this involves differentiation

with respect to unknowns, setting resulting

expressions to zero, and solving them

simultaneously

( )

2

i

e

( )

=

2

3 3 2 2 1

2

min

i i i i

X b X b b Y e

This procedure generates following formulas for

arriving at numerical values of OLS estimators

b

1

, b

2

& b

3

3 3 2 2 1

X b X b Y b =

( )( ) ( )( )

( )( ) ( )

2

3 2

2

3

2

2

3 2 3

3

2

2

2

=

i i i i

i i i i

i

i i

x x x x

x x x y x x y

b

( )( ) ( )( )

( )( ) ( )

2

3 2

2

3

2

2

3 2 2

2

2

3

3

=

i i i i

i i i i

i

i i

x x x x

x x x y x x y

b

In these formulas, lowercase letters denote

deviations from sample mean values

Properties of OLS estimators

The BLUE property continues to hold here as

well

Multiple coefficient of determination

(R

2

)

Explains proportion of variation in Y explained

by Xs jointly

Conceptually, R

2

is akin to r

2

As in two-variable case, R

2

is defined as

R

2

=

=

R

2

lies between 0 and 1

TSS

ESS

+

2

3 3 2

2

i

i i i i

y

x y b x y b

If R

2

=1, fitted regression line explains

100% of variation in Y.

If R

2

=0, model does not explain any of the

variation in Y

The fit of regression model is said to be

better, closer is R

2

to 1

By and large, as the number of Xs increases

R

2

value increases (Why? See below)

R

2

and Adjusted R

2

One aspect of R

2

is: As the no. of Xs increases,

R

2

almost invariably increases

Why? From elsewhere, we know

) R (

2

TSS

RSS

TSS

ESS

+ = 1

TSS

RSS

R + =

2

1

TSS

RSS

R =1

2

=

2

2

1

i

i

y

e

Here depends on no. of Xs, but not

denominator

Hence, as Xs increase is likely to decrease

(or at least it will not increase). Hence R

2

increases

2

i

e

2

i

e

Increasing R

2

Is it desirable to increase R

2

by adding more Xs?

We should adopt a cautious approach here

Why?

(i) With larger Xs, R

2

gives an overly optimistic

picture of regression fit

(ii) R

2

does not take into account d.f.

(iii) We need to have a measure of goodness of fit that

is adjusted for no. of Xs added in the model

Such a measure is known as [adjusted R

2

]

which is defined as

Here k is no. of parameters in the model

including intercept term

Term adjusted means adjusted for d.f.

associated with sums of squares entering into

above identity

2

R

) 1 (

) (

1

2

2

2

n y

k n e

R

i

i

Features of Adjusted R

2

If k>1, s R

2

; i.e. as Xs increase becomes

increasingly less than R

2

or increases less

than unadjusted R

2

This means that, a penalty is involved in adding

more Xs in to a regression model

can be negative, but not R

2

(Why?)

2

R

2

R

2

R

2

R

What we do in practice?

In practice, mainly R

2

is used to measure

goodness of fit

is used in deciding inclusion of a new

variable

If inclusion of a new variable increases , it is

retained in the model

When does increases? If value of the

coefficient of the added variable is larger than 1

2

R

2

R

2

R

t

Why maximizing opposed?

Our objective is not to obtain a high

The researcher should be more concerned about

logical or theoretical relevance of the Xs to Y

and their statistical significance

If this process produces a high it is well and

good

2

R

2

R

2

R

Hypothesis Testing

The procedure is same as in two-variable case

We can adopt both Confidence interval

approach and Test of significance approach

Testing under CIA

We construct a confidence interval and see

whether hypothesized value of population

parameters (B

1

/ B

2

/ B

3

) lies inside this interval.

If it lies inside, we do not reject H

0

If it lies outside, we can reject H

0

The remaining procedure is same as in the case

of two-variable model

The test (t) statistic we use for this purpose is

(For testing B

1

)

(For testing B

2

)

(For testing B

3

)

) (

1

1 1

b se

B b

t

=

) (

2

2 2

b se

B b

t

=

) (

3

3 3

b se

B b

t

=

Testing under ToSA

Step 1: Set H

0

and H

1

separately for each partial

regression coefficient

Examples: H

0

:B

2

=0 and H

1

:B

2

=0

H

0

:B

3

=0 and H

1

:B

3

=0

Step 2: Compute a test (t) statistic from sample

data (See above)

Step 3: Choose level of significance (o) (or)

probability of committing Type 1 error

(0.01 or 0.05 or 0.10)

Step 4: Find probability of obtaining computed test

(t) statistic for certain d.f.

Note: d.f. is (n-k), where n - no. of

observations, k- no. of Xs including

intercept term

Step 5: If this probability is less than the

prechosen o reject H

0

. Otherwise, accept H

0

(OR)

After Step 3 Use following rules to accept or

reject H

0

Null Hypothesis

(H

0

)

Alternative

Hypothesis (H

1

)

Critical region: Reject H

0

if

B

x

=

0

B

x

>

0

[One tailed] Calculated test statistic (t)> t

o, d.f.

(i.e. Table t value at o level of

significance and certain degrees of

freedom)

B

x

= 0 B

x

< 0 [One tailed] Calculated test statistic (t)< -t

o, d.f.

(i.e. Table t value at o level of

significance and certain degrees of

freedom)

B

x

= 0 B

x

= 0 [Two tailed] Calculated absolute value of test

statistic ( ) > t

o/2, d.f.

(i.e. Table t

value at o/2 level of significance

and certain degrees of freedom)

t

A summary of the t test

ANOVA or F test Relevance

This is a complementary way of hypothesis

testing

Commonly used to test joint H

0

in multiple

regression models

But, can be used in two variable regression model

as well

What is a joint H

0

?

H

0

: B

2

= B

3

= 0

Means that B

2

and B

3

are jointly equal to zero

(or) Xs have no influence on Y

A test of joint H

0

is called a test of the overall

significance of estimated regression line

How to construct F test statistic?

From R

2

discussion, we know that

TSS = ESS + RSS (Or)

The d.f. associated with components of this

identity is

TSS = n-1 because we lose 1 d.f. in computing

sample mean

ESS = 2 (k-1) because ESS is a function of B

2

and B

3

(where k is no Xs)

2

3 3 2 2

2

+ + =

i i i i i i

e x y b x y b y

Y

RSS = n-3 (n-k) because in computing RSS we

need to estimate B

1

B

2

and B

3

In case of two-variable model the corresponding

d.f. are:

TSS = n-1

ESS = 1

RSS = n-2

In general, in a regression model with k

explanatory variables (incl. intercept), the d.f. are

as follows

TSS = n-1 (always)

ESS = k-1

RSS = n-k

Source of variation Sum of squares (SS) d.f. Mean sum of Squares

(MSS) = SS/d.f.

Due to regression

(ESS)

2

Due to residuals

(RSS)

n-3

n-3

Total (TSS) n-1

+

i i i i

x y b x y b

3 3 2 2

2

i

e

2

i

y

2 i 2i 3 i 3i

b y x b y x / 2 +

2

i

e /

get ANOVA table

Now, define F statistic as

F = =

F =

. .

. .

f d RSS

f d ESS

) (

) 1 (

k n RSS

k ESS

) 3 (

2

2

3 3 2 2

n e

x y b x y b

i

i i i i

Using F-ratio for hypothesis testing

Set H

0

: B

2

= B

3

= 0 & H

1

: Not all Bs are

simultaneously zero

Calculate F ratio using formula

We reject H

0

, if F value computed from formula

exceeds critical/table F value at o level of

significance and given d.f. in numerator and

denominator

Otherwise, we do not reject H0:

Alternatively, if the p value of computed F ratio

is sufficiently low, we reject H

0

Intuitive Reasoning

In F =

Numerator explains variance of Y explained by

Xs

Denominator explains variance of Y not

explained by Xs

If numerator > denominator, F>1

Increasingly large F is an evidence against H

0

. .

. .

f d RSS

f d ESS

Relationship between F and R

2

The null B

2

= B

3

= 0 is same as saying that H

0

:

R

2

=0 (why?)

Thus, F test is also a test of significance of R

2

(i.e. whether R

2

is different from zero)

The relationship between F ratio and R

2

is as

follows

Here, when R

2

=0, F=0

The larger R

2

is, the greater the F value will be

One advantage of this formula is the ease of

computation of F value. All we need to know is

R

2

value

) /( ) 1 (

) 1 (

2

2

k n R

k R

F

=

Testing significance of R

2

using F test

Substitute R

2

value in and compute F

ratio

We reject H

0

: R

2

=0, if F value computed from formula

exceeds the critical/table F value at o level of

significance and given d.f. in numerator and

denominator

Otherwise, we do not reject the null

) /( ) 1 (

) 1 (

2

2

k n R

k R

F

=

Usefulness of this statistic

In cross-sectional data involving several

observations, one generally obtains low R

2

This is due to diversity of the cross-sectional

units

Here, the statistical significance of R

2

value can

be verified using

) /( ) 1 (

) 1 (

2

2

k n R

k R

F

=

- Chapter 009Uploaded bytrannnen
- Multiple RegressionUploaded byKhawaja Naveed Haider
- Standard Deviation PracticeUploaded byTrisha Bechard
- GRM: Generalized Regression Model for Clustering Linear SequencesUploaded byFanny Sylvia C.
- PROMIS Social Isolation Scoring ManualUploaded byMonica Arlegui
- Pooled Time SeriesUploaded byBoriša Mraović
- 1171924Uploaded byfabriciolafebre
- Globalization AndUploaded byOmar Hasan
- cherry_30 (1)Uploaded byanu riaz
- Buffet Food Purchase BackgroundUploaded bySiti Nor Hajar Hamid
- Public and Private Universities in BangladeshUploaded byMir Abdullah Shahneaz
- Ch 14 Regression InferenceUploaded byMichelle Yu
- SBE10_07a [Read-Only] [Compatibility Mode]Uploaded byVu Duc Hoang Vo
- Reversal of Fortune Geography and Institutions in the Making of the Modern World Income DistributionUploaded byTrang Doan
- abstractUploaded byGoher Ayub
- In d 44306116Uploaded byFrancisco Renteria
- Advanced Regression 1Uploaded bymustofa hilmi
- Lecture6 Module2 Anova 1Uploaded byApam Benjamin
- Multiple RegressionUploaded byNarayanan Subramanian
- output spss.docUploaded byDevi Asyaidah
- 310-11Uploaded byDedy Suryadi
- Dsur i Chapter 07 Linear RegressionUploaded byDanny
- Metode Analisis DataUploaded byhambyong
- Lesson 4 TheoryUploaded byuniversoexacto
- 2015.Test.ansUploaded byAnonymous gUySMcpSq
- Evaluating Semantic Analysis Methods for Short Answer Grading Using Linear RegressionUploaded byGlobal Research and Development Services
- RegressionUploaded byAnonymous m8oCtJB
- 12 Shibly RahmanUploaded bymailmaverick8167
- Using EViews for Principles of Econometrics, Carter HillUploaded byNaila Mehboob
- Ch 02 Wooldridge 6e PPT UpdatedUploaded byMy Tran Ha

- Proc summaryUploaded byMayank Chaturvedi
- Consumer Markets A Global View of Mergers and AcquisitionsUploaded byParitosh Jain
- Recommendations ShoppabilityUploaded bySheeti Das
- Testing of Weak-Form Efficiency in Indian Stock Market_2Uploaded byMayank Chaturvedi
- FInancial PLan of POPUploaded byMayank Chaturvedi
- PorterUploaded byMayank Chaturvedi

- 001-FSI-MS Nitrogen Filling , Molecular Sieve Adsorber.pdfUploaded byKöksal Patan
- Kant - Critique of Practical ReasonUploaded byMallory Margueron
- LAUNCHING OF THE NEW SAN CRISTOBAL BRIDGE.pdfUploaded byJavier Anaya
- CARBON AND ITS COMPOUNDS By Jagdish DevnathUploaded byjd_jagdish1
- The AeM June 2017Uploaded byAjayGupta
- proton csi finalUploaded byapi-285533414
- Apparatus for Separating Solvent in a Feed of Solvent and Deasphalted OilUploaded byCarlos Alberto Augusto Sánchez Quiñones
- Manual de Mantenimiento 108sd and 114sdUploaded byAngel Adolfo Meneses Leon
- Math 3Uploaded byAnonymous EvbW4o1U7
- Manual HGUploaded byF Sisniegas GC
- CVA CASE STUDY.docUploaded byEricson Somera
- 7720 Hw Guide StandaloneUploaded byZalina Abdool
- TIBCO CIM (collaborative information manager) Online TrainingUploaded byMindMajix Technologies
- Fournet & Bomhard - The Indo-European Elements in Hurrian (2010)Uploaded byAllan Bomhard
- CriterionUploaded byapi-3759646
- How dangerous is marijuana?Uploaded byLothar Graudins
- Canon - Trinitarian Bible Society - The Greek New TestamentUploaded byperepinyol
- marco estrada.docxUploaded bymidore60
- FS-0103Uploaded byMiguel H. Mazzeo Meneses
- BIO101-DNA-vs-RNA.pdfUploaded byhimself
- PATH National Tomorrows PlannersUploaded bybenjamin321
- Cover LetterUploaded byrrameshsmit
- Billion Ton Update 0Uploaded byzagazig123
- Proverbs and IdiomsUploaded byMukilan Murugasan
- 2019 senior project bluebook -varvara bulanovaUploaded byapi-451787685
- Medieval Conceptions of MagicUploaded byJimny34
- ReadmeUploaded byBoford Burton
- In the Flat World Maritime Students Reading HabitsUploaded bymaceleste_fusilero
- Hair Loss WomenUploaded bySusanne Seynaeve
- Plagiarism and How to AvoidUploaded byDyan Joseph

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.