Linear Regression

Chapter
Linear Regression
Shirin Aslani, Fall 2021
1
Business Analytics– Dr. Shirin Aslani – GSME, Sharif University of Technology
The statistical Sommelier
• Large differences in price and quality between years,

although wine is produced in a similar way
• Meant to be aged, so hard to tell if wine will be good when

it is on the market
• Expert tasters predict which ones will be good
• Can analytics be used to come up with a different system

for judging wine?
2
• Large differences in price and quality between years,

although wine is produced in a similar way
• Meant to be aged, so hard to tell if wine will be good when

it is on the market
• Expert tasters predict which ones will be good
• Can analytics be used to come up with a different system

for judging wine?
• March 1990 – Orley Ashenfelter, a Princeton economics

professor, claims he can predict wine quality without tasting
the wine
3
• Ashenfelter used a method called linear regression

• Predicts an outcome variable, or dependent variable
• Predicts using a set of independent variables
• Dependent variable: typical price in 1990-1991 wine

auctions (approximates quality)
• Independent variables:
• Age – older wines are more expensive
• Weather
• Average Growing Season Temperature

• Harvest Rain
• Winter Rain
4
• Robert Parker, the world's most influential wine expert:
“Ashenfelter is an absolute total sham”
• “rather like a movie critic who never goes to see the movie
but tells you how good it is based on the actors and the
director”
5
One-Variable Linear Regression
8.5
8
Logarithm of Price
7.5
6.5
6
14.5 15 15.5 16 16.5 17 17.5 18
Average Growing Season Temp
6
8.5
8
Logarithm of Price
7.5
6.5
6
14.5 15 15.5 16 16.5 17 17.5 18
7
8.5
8
Logarithm of Price
7.5
6.5
6
14.5 15 15.5 16 16.5 17 17.5 18
8
The Regression Model
• The best model (choice of coefficients) has the

smallest error terms
9
Selecting the Best Model
10
Selecting the Best Model
11
Other Error Measures
• SSE can be hard to interpret

• Depends on N
• Units are hard to understand
• Root-Mean-Square Error (RMSE)
• Normalized by N, units of dependent variable

12
R
• Compares the best

model to a “baseline”
model
• The baseline model
does not use any
variables
• Predicts same outcome
(price) regardless of
the independent
variable (temperature)
13
R
14
15
Available Independent Variables
• Many different independent variables could be

used
• Average Growing Season Temperature
• Harvest Rain
• Winter Rain
• Age of Wine (in 1990)
• Population of France
16
Available Independent Variables
17
The Regression Model
• Multiple linear regression model with k variables
• Best model coefficients selected to minimize SSE
18
Adding Variables
• Adding more variables can improve the model

• Diminishing returns as more variables are added
19
Adding Variables
• Not all available variables should be used

• Each new variable requires more data
• Causes overfitting: high R2 on data used to create
model, but bad performance on unseen data
20
Correlation
• Correlation measures linear association, not

causation.
• The correlation (usually denoted by r) between
two variables (call them x and y) is a unit-free
measure of the strength of the linear relationship
between x and y.
• The correlation between any two variables is
always between –1 and +1.
• Although the exact formula used to compute the
correlation between two variables isn’t very
important, interpreting the correlation between
the variables is.
• A correlation near +1 means that x and y have a
strong positive linear relationship. 21
Correlation
22
Predictive Ability
23
24
The Result
• Parker:
• 1986 is “very good to sometimes exceptional”
• Ashenfelter:
• 1986 is mediocre
• 1989 will be “the wine of the century” and 1990 will be
even better!
• In wine auctions,
• 1989 sold for more than twice the price of 1986
• 1990 sold for even higher prices!
• Later, Ashenfelter predicted 2000 and 2003
would be great
• Parker has stated that “2000 is the greatest
vintage Bordeaux has ever
• produced” 25
Effects of light on Meadowfoam Flowering
• Meadowfoam is a small plant found growing in moist

meadows of the U.S. Pacific Northwest.
• It has been domesticated at Oregon State University for its
seed oil, which is unique among vegetable oils for its long
carbon strings which is nongreasy and highly stable.
• One study in a series designed to find out how to elevate
meadowfoam production to a profitable crop.
• In a controlled growth chamber, they focused on the effects
of two light-related factors:
• light intensity, 150, 300, 450, 600, 750, and 900 µmol/m2/sec;
• the timing of the onset of the light treatment, either at
photoperiodic floral induction (PFI)— the time at which the
photo period was increased from 8 to 16 hours per day to
induce flowering—or 24 days before PFI.
26
27
• 12 treatment groups—6 light intensities at 2 timing levels.

• Ten seedlings randomly assigned to each treatment group.
• The number of flowers per plant is the primary measure of
production, measured by averaging the numbers of flowers
produced by the 10 seedlings in each group.
• The entire experiment was repeated.
• No difference was found between the first and second runs
of the experiment.
• The 2 observations in each cell are thought of as

independent replicates under the specified conditions.
• What are the effects of differing light intensity levels?
• What is the effect of the timing? 28
Statistical Conclusion
• Increasing light intensity decreased the mean number of

flowers per plant by an estimated 4.0 flowers per plant per
100 µmol/m2/sec (95% confidence interval from 3.0 to 5.1).
• Beginning the light treatments 24 days prior to PFI increased
the mean numbers of flowers by an estimated 12.2 flowers
per plant (95% confidence interval from 6.7 to 17.6).
29
Statistical Conclusion
• The data provide no evidence that the effect of light intensity

depends on the timing of its initiation (two-sided p-
value=0.91, from a t -test for interaction, 20 degrees of
freedom).
30
Scope of Inference
• The researchers can infer that the effects above were caused
by the light intensity and timing manipulations, because this
was a randomized experiment.
31
The Multiple Linear Regression
• In multiple regression there is a single response variable

and multiple explanatory variables.
• Although they should not be thought of as the truth, one or

two models may adequately approximate the mean of the
response as a function of the explanatory variables, and
conveniently allow for the questions of interest to be
investigated.
32
Interpretation of Regression Coefficients
33
An Indicator Variable to Distinguish Between Two
Groups
• An indicator variable (or dummy variable) takes on one of two
values: “1” (one) indicates that an attribute is present, and “0”
(zero) indicates that the attribute is absent.
• Consider the regression model
• If time=0, then early = 0, and the regression line is
• If time = 24, then early = 1, and the regression line is
• Because the slopes are the same the model is called the parallel
lines regression model 34
MONEYBALL
• Moneyball tells the story of the

Oakland A’s in 2002
• One of the poorest teams in baseball
• New ownership and budget cuts in 1995
• But they were improving
• How were they doing it?

• Was it just luck?
• In 2002, the A’s lost three key
players
• Could they continue winning?
35
The Problem
• Rich teams have four times

the salary of poor teams
• The Oakland A’s can’t
afford the all-stars, but they
are still making it to the
playoffs. How?
• They take a quantitative
approach and find
undervalued players
36
A Different Approach
• The A’s started using a different method to select

players
• The traditional way was through scouting
• Scouts would go watch high school and college players
• Report back about their skills
• A lot of talk about speed and athletic build
• The A’s selected players based on their statistics,
not on their looks
“The statistics enabled you to find your way
past all sorts of sight-based scouting prejudices.”
37
Billy Beane
• The general manager since 1997

• Played major league baseball, but never made it
big
• Billy Beane succeeded in using analytics
• Had a management position
• Understood the importance of statistics – hired Paul
DePodesta (a Harvard graduate) as his assistant
• Didn’t care about being ostracized
38
Taking a Quantitative View
• Paul DePodesta spent a lot of time looking at the

data
• His analysis suggested that some skills were
undervalued and some skills were overvalued
• If they could detect the undervalued skills, they
could find players at a bargain
39
Taking a Quantitative View
40
Making it to the Playoffs
• How many games does a team need to win in the

regular season to make it to the playoffs?
• “Paul DePodesta reduced the regular season to a
math problem. He judged how many wins it
would take to make it to the playoffs: 95.”
41
Scoring Runs
• How does a team score more runs?

• The A’s discovered that two baseball statistics
were significantly more important than anything
else
• On-Base Percentage (OBP)
• Percentage of time a player gets on base
(including walks)
• Slugging Percentage (SLG)
• How far a player gets around the bases on
his turn (measures power)
42
Predicting Runs and Wins
• Can we predict how many games the 2002

Oakland A’s will win using our models?
• The models for runs use team statistics
• Each year, a baseball team is different
• We need to estimate the new team statistics
using past player performance
• Assumes past performance correlates with future
performance
• Assumes few injuries
• We can estimate the team statistics for 2002 by
using the 2001 player statistics
43
Predicting Runs Scored
• At the beginning of the 2002 season, the Oakland

A’s had 24 batters on their roster
• Using the 2001 regular season statistics for these
players
• Team OBP is 0.339
• Team SLG is 0.430
• At the beginning of the 2002 season, the Oakland

A’s had 17 pitchers on their roster
• Using the 2001 regular season statistics for these
players
• Team OOBP is 0.307
• Team OSLG is 0.373
44
The Goal of a Baseball Team
• Why isn’t the goal to win the World Series?

• Billy and Paul see their job as making sure the
team makes it to the playoffs – after that all bets
are off
• The A’s made it to the playoffs in 2000, 2001, 2002,
2003
• But they didn’t win the World Series
• Why?
45

Linear Regression

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Linear Regression

Uploaded by

Copyright:

Available Formats

Chapter

Shirin Aslani, Fall 2021

• Large differences in price and quality between years,

• Meant to be aged, so hard to tell if wine will be good when

• Expert tasters predict which ones will be good

• Can analytics be used to come up with a different system

• Large differences in price and quality between years,

• Meant to be aged, so hard to tell if wine will be good when

• Expert tasters predict which ones will be good

• Can analytics be used to come up with a different system

• March 1990 – Orley Ashenfelter, a Princeton economics

• Ashenfelter used a method called linear regression

• Dependent variable: typical price in 1990-1991 wine

• Average Growing Season Temperature

• Robert Parker, the world's most influential wine expert:

“Ashenfelter is an absolute total sham”

Average Growing Season Temp

Average Growing Season Temp

Average Growing Season Temp

• The best model (choice of coefficients) has the

• SSE can be hard to interpret

• Root-Mean-Square Error (RMSE)

• Normalized by N, units of dependent variable

• Compares the best

• Many different independent variables could be

• Multiple linear regression model with k variables

• Best model coefficients selected to minimize SSE

• Adding more variables can improve the model

• Not all available variables should be used

• Correlation measures linear association, not

• Meadowfoam is a small plant found growing in moist

• 12 treatment groups—6 light intensities at 2 timing levels.

• The 2 observations in each cell are thought of as

• Increasing light intensity decreased the mean number of

• The data provide no evidence that the effect of light intensity

• In multiple regression there is a single response variable

• Although they should not be thought of as the truth, one or

• Consider the regression model

• If time=0, then early = 0, and the regression line is

• If time = 24, then early = 1, and the regression line is

• Moneyball tells the story of the

• How were they doing it?

• Rich teams have four times

• The A’s started using a different method to select

• The general manager since 1997

• Paul DePodesta spent a lot of time looking at the

• How many games does a team need to win in the

• How does a team score more runs?

• Can we predict how many games the 2002

• At the beginning of the 2002 season, the Oakland

• At the beginning of the 2002 season, the Oakland

• Why isn’t the goal to win the World Series?

You might also like