Lecture No.
Simple Linear Regression Model
(The Ordinary Least Squares Method)
Demand Estimation
Instructor Dr. Jangraiz Khan
Suppose a manager wants to find the impact of Advertisement
expenditure on sales
Collects 10 years observations as given in the table
Takes sales as dependent variable while Advertisement expenditure
as independent variable
Sales = Y
Advertisement expenditure= X
The Data is shown in the table
Plots on graph
Takes Sales on Y-Axis and Adv. Exp on X-Axis
By plotting the data we Get various points as shown in the fig
The points are scattered
The figure is called scattered diagram
Demand Estimation
Now the points show visually a positive relationship between
X&Y
We can extend it to further observations too but it is difficult to
estimate from this figure that how much of the sales are increased
by how much of advertisement expenditure
We need a line which best fits the data
The visual line may not be very useful as different researchers may
have different views regarding the relationship
What is Regression
The Concept of regression is said to be developed from idea of Francis Galton
1822- 1911
Parents Heights and children heights
Found children of tall parents are tall and of short parents are short
“Regression analysis is a statistical technique for obtaining the line that best fits the data
points according to an objective statistical criterion so that all researchers looking at the
same data would get exactly the same result”
Regression analysis is the study of how a response variable depends on one or more
predictors.
Regression analysis is a set of statistical processes for estimating the relationships between
a dependent variable (often called the 'outcome variable') and one or more independent
variables
Regression analysis is the study of relationships between two or
more variables and is usually conducted for the following reasons:
When we want to know whether any relationship between two or
more variables actually exists
when we are interested in understanding the nature of the
relationship between two or more variables and
when we want to predict a variable given the value of others
(Mclntosh et al, 2010)
Regression analysis investigates the dependence of one variable
over other.
Example
i. Price and supply relationship
Supply depends on price so supply is dependent variable
Price is independent variable
Y= Supply
X= price
ii. Price and demand relation ship, Demand depends on price
so demand is dependent variables and price is independent
variable.
Y= demand X= price
iii. Sales is function of advertisement expenditure
Taking the example (i). Supply function
Supply = f (Price)
(1)
0 (2)
0 are parameters/coefficients
0 (3)
Variation in Y= Systematic variation+ Random variation
0 = Systematic variation
Ui = Random term
There can be other determinants of supply as well so all missing
variables are covered in the random term/error term.
ASSUMPTIONS OF Ui
1) Ui is a random variable (the value of U i is by chance, 0, +ve, -ve)
2) The mean value of Ui in any particular period is zero
E (Ui)= 0
3). The variance of Ui ( )remains constant in each period (Homoscedasticity)
If assumption no 3 is violated----Heteroscedasticity
4) Ui has normal distribution
5) The random terms of different observations (U i , Uj) are independent
If this assumption is violated, we have the problem of Autocorrelation
6) Ui is independent of explanatory variables 0
Koutsoyiannis, A. (1987). Theory of Econometrics: An Introductory Exposition of Economic Methods.
7). Explanatory (Independent) variables are not perfectly linearly correlated
If assumption No.7 is violated then we have the problem of Multicollinearity
8) The explanatory variables are measured without error
Human Capital…..Education, health, experience, training, specific skills
9) Macro variables should be correctly aggregated
GDP (Product method, expenditure method, income method, value add)
Product method---Agricultural production, industrial production, minerals, …….
10) The relationship being estimated is identified
11) The relationship is correctly specified
Y= 0i
LEAST SQUARES CRITERION AND NORMAL EQUATIONS
The regression line can be obtained by minimizing the sum of
square of residuals
The method is therefore, called Method of Ordinary Least Squares
We suppose
Y= b0 + b1 Xi
Koutsoyiannis, A. (1987). Theory of Econometrics: An Introductory Exposition of Economic Methods.
Koutsoyiannis, A. (1987). Theory of Econometrics: An Introductory Exposition of Economic Methods.
Example 1
Estimate the regression line Y= a +b X + Ui
a = b0 b= b1
Time Xt Yt Xt X Yt Y ( X t X )(Yt Y ) ( X t X )2
1 10 44 -2 -6 12 4
2 9 40 -3 -10 30 9
3 11 42 -1 -8 8 1
4 12 46 0 -4 0 0
5 11 48 -1 -2 2 1
6 12 52 0 2 0 0
7 13 54 1 4 4 1
8 13 58 1 8 8 1
9 14 56 2 6 12 4
10 15 60 3 10 30 9
120 500 106 30
n
X t 120
n
n 10 X 12
X t 120
t 1
t 1 n 10
n n
Yt 500
Yt 500
t 1
Y
t 1 n
10
50
(X
n
X )2 30
( X X )(Y Y ) 106
t 1
t t t 1
t
(X t X )(Yt Y ) ˆ
â Y bX
bˆ t 1
n
t
( X
t 1
X ) 2
aˆ 50 (3.533)(12) 7.60
106
bˆ 3.533
30
Y= Xt
Example 2
Given the following income and food expenditure data, estimate the
regression line of food expenditure on income and interpret your
Income 20 30 33 40 15 13 26 38 35 43
results.
Expenditur 7 9 8 11 5 4 8 10 9 10
e on food
Example.
Estimate the following linear regression line, given the following data
Y= β1 + β 2X + Ui
Solution
Example:
Estimate simple linear regression line , given the data in next slide
References
Salvatore, D. (2004). Managerial Economics in a Global Economy,
5th Edition. South-Western Publishing Co
https://en.wikipedia.org/wiki/Regression_analysis
Mclntosh, AM., Sharpe,M., and Lawrie,S.M . (2010). Statistics and
evidence Practice. Research Methods, Companion to Psychiatric
Studies.pp.157-198.
Koutsoyiannis, A. (1987). Theory of Econometrics: An Introductory
Exposition of Economic Methods. Palgrave MacMillan, Basingstoke,
United Kingdom
