2 .

6 REGRESSION
MODELING FOR
INFERENTIAL
STATISTICS
Deon Francis George
 REGRESSION

Regression is a statistical analysis technique used

to model the relationship between a dependent
variable and one or more independent variables. It
is commonly employed for the purpose of making
predictions or understanding how changes in the
independent variables affect the dependent
variable.

2023 Regression Modeling for Inferential Statistics Page : 2

ORIGIN OF
REGRESSION

Regression has its origins in Sir Francis

Galton's and Karl Pearson's later studies of
the hereditary traits of sweet peas, which
were conducted in the 1920s and 1930s.
Regression has since taken over as the
statistical method of choice for
characterizing the connections between
explanatory (input) and response (output)
variables.
PURPOSE OF REGRESSION IN INFERENTIAL
STATISTICS………..

Regression is a relatively simple statistical technique to model the dependence of a variable (response or output
variable) on one (or more) explanatory (input) variables. Once identified, this relationship between the variables
can be formally represented as a linear/additive function/equation.

2023 Regression Modeling for Inferential Statistics 5

 Regression can be used for one of two purposes:

1. Hypothesis Testing - Investigating potential relationships between different variables.

2. Prediction/forecasting - Estimating values of a response variables based on one or more explanatory

variables.

2023 Regression Modeling for Inferential Statistics 6

Hypothesis Testing

Hypothesis Testing - In hypothesis testing (theory building), regression analysis can reveal the
existence/strength and the directions of relationships between a number of explanatory variables (often
represented with x) and the response variable (often represented with y).

Prediction / Forecasting

In prediction, regression identifies additive mathematical relationships (in the form of an equation)
between one or more explanatory variables and a response variable. Once determined, this equation
can be used to forecast the values of the response variable for a given set of values of the explanatory
variables.

2023 Regression Modeling for Inferential Statistics 7

CORRELATION VERSUS REGRESSION

Professionals and even scientists sometimes mix these two phrases because correlation studies gave rise to
regression analysis and because both approaches seek to explain the relationship between two (or more) variables.

Correlation makes no a priori assumption of whether one variable is dependent on the others) and is not
concerned with the relationship between variables; instead it gives an estimate on the degree of association
between the variables. Correlation is interested in the low-level relationships between two variables,

Regression attempts to describe the dependence of a response variable on one (or more) explanatory variables
where it implicitly assumes that there is a one-way causal effect from the explanatory variables) to the response
variable, regardless of whether the path of effect is direct or indirect.
Regression is concerned with the relationships between all explanatory variables and the response variable.

2023 Regression Modeling for Inferential Statistics 8

 SIMPLE VERSUS MULTIPLE REGRESSION

If the regression equation is built between one response variable and one explanatory variable, then it is
called simple regression.

Example : The regression equation built to predict/explain the relationship between a

height of a person (explanatory variable) and the weight of a person (response variable) si a good
example of simple regression.

Multiple regression is the extension of simple regression where the explanatory variables are more than
one.

Example : If we were to include not only the height of the person but also other personal characteristics
(e.g., BMI, gender, ethnicity) to predict the weight of a person, then we would be performing multiple
regression analysis.

Regression Modeling for Inferential Statistics

2023 9
 SIMPLE VERSUS MULTIPLE REGRESSION

•There is just one x and one y variable in simple linear regression.

•There is one y variable and two or more x variables in multiple linear regression.

Regression Modeling for Inferential Statistics

2023 10
 SIMPLE VERSUS MULTIPLE REGRESSION
A simple linear regression model usually takes the form of:

With multiple regression models, on the other hand, the equation looks more
like this:

Regression Modeling for Inferential Statistics

2023 11
 ORDINARY LEAST SQUARES (OLS) METHOD

Even though there are several methods/algorithms proposed to identify the regression line, the one that is most
commonly used si called the ordinary least squares (OLS) method. The OLS method aims to minimize the sum of
squared residuals (squared vertical distances between the observation and the regression point) and leads to a
mathematical expression for the estimated value of the regression line (which are known as ß parameters).

Regression Modeling for Inferential Statistics

2023 12
THANK YOU

2023 Regression Modeling for Inferential Statistics

13