Statistical

Methods
IPM – Term I, September 2022

Dr. Landis Conrad Felix Michel

Simple Linear Regression

• Regression analysis techniques help uncover relationships between variables.

• When you use regression analysis, you examine relationships by developing models that express how
accurately one or more independent variables predict the value of a dependent variable.
• By convention, the symbol Y represents the dependent variable and the symbol X, subscripted, if
necessary, represents the one or more independent variables.
• The mathematical relationship that a regression model identifies enables you to quantify the effect on the
dependent Y variable that a change in the one or more X independent variables produces.

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel
IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel
• The slope of the line, b1, represents the expected change in Y per unit change in X.
It represents the mean amount that Y changes (either positively or negatively) for a one-unit change
in X.
• The Y intercept, b0, represents the mean value of Y when X equals 0.

• ei, represents the random error in Y for each observation, i.

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

The most common approach to finding b0 and b1 is using the least-squares method.
This method minimizes the sum of the squared differences between the actual values and the predicted
values , using the simple linear regression equation.

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

• The least-squares method determines the values of b0 and b1 that minimize the sum of squared
differences around the prediction line.
• Any values for b0 and b1 other than those determined by the least-squares method result in a greater sum
of squared differences between the actual values and the predicted values .
• For small data sets, you can use a hand calculator to compute the least-squares regression
coefficients. Otherwise, you will use a software (Excel, R, Stata etc.)

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel
 2.

1.

3.
 1.

2.
4.

3.
A. Measures of variation
IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel
IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel
B. Coefficient of Determination
• The coefficient of determination measures the proportion of variation in Y that is explained
by the variation in the independent variable X in the regression model. The range of r2 is from
0 to 1 and the greater the value, the more the variation in Y in the regression model can be
explained by the variation in X.

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

C. Standard Error of the Estimate
IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel
 D. Inferences for
Parameters (Hypothesis
Testing)

Slope
IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel
 Confidence Intervals

pValue: Alternative to
calculating t(a/2) critical

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel
 • F-stat (F distribution) , unlike
t-stat (t distribution) or Z-
stat (Standard Normal
Distribution) is not
symmetrical, neither 2 tail.
• When we have only 1
independent variable
F = t squared and can be
used as an alternative to t for
the significance of the slope.

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

Multiple Regression Model

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel
IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel
• As k increases (e.g. as I add variables to my model, r squared will
always increase. So when using R squared to compare models with
different number of independent variables, it is more appropriate to
use adjusted r squared which penalizes for adding more variables.

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

 • In conjecture to Simple Regression
– when k>1, then we use F stat to
test if all independent variables are
jointly significant.

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

Dummy Variables
• A dummy variable Xd recodes the categories of a categorical variable using the numeric
values 0 and 1.
• In general, the number of dummy variables you need to define equals the number of
categories – 1.
• If a categorical independent variable has only two categories, you define one dummy
variable, Xd, and use the values 0 and 1 to represent the two categories. When the two
categories represent the presence or absence of a characteristic, use 0 to represent the
absence and 1 to represent the presence of the characteristic.

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

• Suppose we have 20 observations for Sales and want to classify based on Location of Sale

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

• Suppose we have time series data and we want to denote observation which corresponds to
January :

• Set dummy Xjan = 1 if observation refers to January and zero otherwise.

• If I want to include dummies for all months, I should use 11 (the number of categories – 1)

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

Interactions
• In the regression models discussed so far, the effect an independent variable has on the
dependent variable has been assumed to be independent of the other independent variables in the
model.

• An interaction occurs if the effect of an independent variable on the dependent variable

changes according to the value of a second independent variable.

• Interactions are often combined with dummy variables to measure the impact of specific
characteristic on another variable.

Where :

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel

References – Additional Readings
• Chapter 13,14 “Statistics for Managers, Using Microsoft Excel”, 8th Edition, David M. Levine,
David Stephan, Kathryn A. Szabat.

•.

IPM – Term I, September 2022 - Dr. Landis Conrad Felix Michel