Analisis de Regresion - Espol

ESCUELA SUPERIOR POLITCNICA DEL LITORAL
Anlisis de Regresin
Andrs G. Abad, PhD

agabad@espol.edu.ec
Facultad de Ingeniera en Mecnica y

Ciencias de la Produccin
Guayaquil, Ecuador
ESCUELA SUPERIOR POLITECNICA DEL LITORAL ESPOL, Guayaquil, Ecuador
Regresin Lineal
Data Mining, agabad@espol.edu.ec 2

Regresin lineal simple

Yi o 1 X i i
intercepto pendiente error
X
Data Mining, agabad@espol.edu.ec
Ejemplo: Toyota Corolla

Los datos corresponden a 1436
autos Toyota Corolla usados
El objetivo es predecir el precio de
venta en funcin de las
caractersticas del auto
Precio en USD
Edad en meses

Estimadores de Mnimos Cuadrados
1*=-170.93
0*=20294.06
1*=-170.93 0
0*=20294.06
1
0
1
Ecuaciones Normales

Variabilidad de los Coeficientes

Variabilidad de los Coeficientes

Constratando hiptesis sobre

coeficientes


coeficientes

Coeficiente de Determinacin
TOTAL SUM OF SQUARES
EXPLAINED SUM OF SQUARES SST = SSE + SSR

RESIDUAL SUM OF SQUARES
Variance to be
explained by
X1 predictors
(SST)
Variance
explained by
X1 Variance
NOT
(SSE) explained by
Y X1
(SSR)
SSE
R 2
SST
para evaluar la adecuacin del modelo de regression lineal

Supuestos del Modelo de Regresin Lineal
1. The linear model correctly describes the functional

relationship between X and Y.
2. The X variable is measured without error
3. For any given value of X, the sampled Y values are
independent
4. Residuals (errors) are normally distributed.
5. Variances are constant along the regression line.

Regresin Lineal Mltiple

The linear model with a single predictor variable X can easily
be extended to two or more predictor variables.
Y o 1 X1 2 X 2 ... p X p

Variance Explained by Model

A good model
X1 X2
Common variance
explained by X1 and X2
X2 Y
X1
Unique variance
explained by X2
Unique variance
Y
Variance NOT
explained by X1 explained by
X1 OR X2
Regresin Lineal Mltiple
Y o 1 X1 2 X 2 ... p X p
intercept Partial Regression errors

Coefficients
Partial Regression Coefficients (slopes):

Regression coefficient of X after controlling for
(holding all other predictors constant) influence of
other variables from both X and Y.

Computo de Parmetros

Cmputo de Coeficientes
( X ' X ) X 'Y 1

( X ' X ) X 'Y 1

( X ' X ) X 'Y 1

Estadsticos de Regresin
TOTAL SUM OF SQUARES
EXPLAINED SUM OF SQUARES
RESIDUAL SUM OF SQUARES
SSE
R 2
SST
Coefficient of Determination
to judge the adequacy of the regression model

Estadsticos de Regresin
Adjusted Coefficient of Determination

to judge the adequacy of the regression model
penalizing for complex models
n 1
R 2
1 (1 R )
2
n k 1
adj
n = sample size
k = number of independent variables


coeficientes
Es al menos uno de los predictors X1,
X2,,Xp til para predecir la variable de
respuesta Y ?

Seleccin del Modelo

Seleccin del Modelo

Seleccin hacia adelante

Seleccin del Modelo

Seleccin hacia atrs

Logistic Regression
Extends idea of linear regression to situation
where outcome variable is categorical
Widely used, particularly where a structured

model is useful to explain (=profiling) or to predict
We focus on binary classification

i.e. Y=0 or Y=1

Ejemplo: Aceptacin de Prstamos

Los datos contienen informacin de 5000
ofertas de prstamos
La variable respuesta es si una oferta de
prestamos ha sido aceptado en el pasado
Prstamo=0+ 1*Ingresos
UniversalBank Data
Age of customer
Prstamo {0,1} Experience: professional experience in years
Ingresos $K/ao Income of customer
Prstamo
Family size of customer

CCAvg: average monthly credit card spending
Mortgage: size of mortgage
SecuritiesAccount: No/Yes
CDAccount: No/Yes
Online: No/Yes
CreditCard: No/Yes
Educational level: three categories
(undergraduate, graduate, professional)
Ingresos
Acept prstamo
CCAvg: average monthly
No acept prstamo
credit card spending
Income


Funcin de Respuesta Logstica

p = probabilidad de pertenecer a la clase 1
Se debe garantizar que 0 p 1

Odds y logit en funcin de p

Data preprocessing
Partition 60% training, 40% validation
Create 0/1 dummy variables for categorical predictors

Single Predictor Model
Modeling loan acceptance on income (x)
Fitted coefficients (more later): b0 = -6.3525, b1 = -0.0392

Seeing the Relationship

Last step - classify
Model produces an estimated probability of

being a 1
Convert to a classification by establishing

cutoff level
If estimated prob. > cutoff, classify as 1

0.50 is popular initial choice


Estimates of s are derived through an iterative
process called maximum likelihood estimation


Predictive Accuracy
Number of correct classifications
Accuracy = Number of instances in our database
Accuracy is a single number, we may be better off looking at a

confusion matrix. This gives us additional useful information
Classified as a
100 0 0
True label is...
9 90 1
45 45 10
Curva ROC
True Positive Rate

TP/(TP+FN)
False Positive Rate

FP/(FP+TN)
Area Under the Curve (AUC)

0 AUC 1
The bigger AUC the better


La curva lift: compara el nmero acumulado de casos
positivas entre el modelo y los datos
Presenta la ventaja de utilizar el modelo frente a un
manejo aleatorio

Complete Example:
Predicting Delayed Flights DC to NY
Outcome: delayed or not-delayed
Predictors:
Day of week
Departure time
Origin (DCA, IAD, BWI)
Destination (LGA, JFK, EWR)
Carrier
Weather (1 = bad weather)

Data Preprocessing
Create binary dummies for the categorical

variables
Partition 60%-40% into training/validation

The Fitted Model (not all 28 variables shown)

Model Output (Validation Data)

After Variable Selection

(Model with 7 Predictors)

Predictor Model
Note that Weather is unknown at time of prediction

(requires weather forecast or dropping that predictor)

Analisis de Regresion - Espol

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Analisis de Regresion - Espol

Uploaded by

Copyright:

Available Formats

ESCUELA SUPERIOR POLITCNICA DEL LITORAL

Andrs G. Abad, PhD

Facultad de Ingeniera en Mecnica y

Data Mining, agabad@espol.edu.ec 2

Regresin lineal simple

Ejemplo: Toyota Corolla

Data Mining, agabad@espol.edu.ec 4

Estimadores de Mnimos Cuadrados

Data Mining, agabad@espol.edu.ec 6

Variabilidad de los Coeficientes

Data Mining, agabad@espol.edu.ec 7

Variabilidad de los Coeficientes

Data Mining, agabad@espol.edu.ec 8

Constratando hiptesis sobre

Data Mining, agabad@espol.edu.ec 9

Constratando hiptesis sobre

Data Mining, agabad@espol.edu.ec 10

EXPLAINED SUM OF SQUARES SST = SSE + SSR

Data Mining, agabad@espol.edu.ec

Supuestos del Modelo de Regresin Lineal

1. The linear model correctly describes the functional

Data Mining, agabad@espol.edu.ec

Regresin Lineal Mltiple

Data Mining, agabad@espol.edu.ec

Variance Explained by Model

Regresin Lineal Mltiple

intercept Partial Regression errors

Partial Regression Coefficients (slopes):

Data Mining, agabad@espol.edu.ec

Data Mining, agabad@espol.edu.ec 17

Data Mining, agabad@espol.edu.ec

Data Mining, agabad@espol.edu.ec

Data Mining, agabad@espol.edu.ec

EXPLAINED SUM OF SQUARES

RESIDUAL SUM OF SQUARES

Data Mining, agabad@espol.edu.ec

Adjusted Coefficient of Determination

Data Mining, agabad@espol.edu.ec

Constratando hiptesis sobre

Data Mining, agabad@espol.edu.ec 23

Seleccin del Modelo

Data Mining, agabad@espol.edu.ec 24

Seleccin del Modelo

Data Mining, agabad@espol.edu.ec 25

Seleccin del Modelo

Data Mining, agabad@espol.edu.ec 26

Widely used, particularly where a structured

We focus on binary classification

Data Mining, agabad@espol.edu.ec

Ejemplo: Aceptacin de Prstamos

Family size of customer

Ejemplo: Aceptacin de Prstamos

Data Mining, agabad@espol.edu.ec 29

Ejemplo: Aceptacin de Prstamos

Data Mining, agabad@espol.edu.ec 30

Funcin de Respuesta Logstica

Data Mining, agabad@espol.edu.ec

Odds y logit en funcin de p

Data Mining, agabad@espol.edu.ec

Data Mining, agabad@espol.edu.ec

Single Predictor Model

Modeling loan acceptance on income (x)

Fitted coefficients (more later): b0 = -6.3525, b1 = -0.0392

Data Mining, agabad@espol.edu.ec