Robust Regression

© All Rights Reserved

28 views

Robust Regression

© All Rights Reserved

- Wal-mart Sales Forecasting
- An Introduction to Robust Regression
- MMP using ACE
- Methods in Applied Econometrics
- Stats Notes l2ss3los10
- Margins 01
- Question Paper Unsolved - Eco No Metrics
- A Two-stage Robust Power System State Estimation Method With Unkmown Measurement Noise
- Hetero Test
- (P3) Inequality Convergence
- Regression With Life Data
- Course_ Decision Science
- 1-s2.0-S0261560606000891-main
- 3.Principals__Leadership_Style_and_School_Perfonnance.pdf
- untitled
- Ch3
- Dm 26785792
- PARALLEL WISARD OBJECT TRACKER: A RAMBASED TRACKING SYSTEM
- Huth Programa
- Courseoutline SE(ECO00037M) Term1 201415

You are on page 1of 20

ROBUST REGRESSION

(OLS)

Recall: [Classical Linear Regression Model]

where

Yi is the value of the response variable in the ith trial

Xij are the known constants; the value of the jth independent

variable on the ith trial

0, 1, 2, ..., k are parameters

i is a random error term,

(i = 1, 2, ..., n and j = 0, 1, 2, ..., k)

Robust Regression

(OLS)

optimizes the fit by making the residuals very small

easy to understand

DISADVANTAGE:

Highly influenced by outliers. OLS is not robust to outliers.

Note: Regression outliers (either in x or in y) pose a serious threat

to standard least squares analysis.

Robust Regression

(OLS)

2 ways to solve the problem of regression outliers:

Regression Diagnostics

- only work when there is only a single

outlier

Robust Regression

Robust Regression

ROBUST REGRESSION

An alternative method to the Ordinary least Squares

method

A regression method that is not as sensitive to

outliers when errors are not normally distributed as

opposed to usual assumption that errors in regression

models are normally distributed

Robust Regression

ROBUST REGRESSION

Robustness is the insensitivity to small deviations from the

assumptions the model imposes on the data (Huber, 1981)

Reasonably efficient and unbiased

Small deviations from the model assumptions will not

substantially impair the performance of the model

Somewhat larger deviations will not invalidate the model

completely

Robust Regression

ROBUST REGRESSION

The Breakdown Point of an estimate is the smallest

fraction of the data that can be changed by an arbitrarily

large amount and still cause an arbitrarily large change in

the estimate.

Robust Regression

Least Squares and Least Median Squares (LMS):

Robust Regression

Regression

1. Least Median Squares

2. Least Trimmed Squares

LMS and LTS both have a breakdown point of 50%

compared to the OLS which has a breakdown point of 0%.

Robust Regression

Consider the model,

where ri are the squared residuals

Robust Regression

10

Objective Function of the LMS estimator:

Minimize

Procedure: (one-dimensional case):

1.

2.

3.

4.

Compute for h = n/2 + 1

Compute for yh y1, yh+1 y2, ..., yn yn-h+1

Get the midpoint of the two observations that

yield the smallest difference.

5. The resulting midpoint is the LMS estimate.

Robust Regression

11

Exercises:

Cuteness Rating was taken from random people of

engineering. Here are the observations:

The Naked Truth was. Here are their responses:

Robust Regression

12

LMS estimator.

Procedure:

1. Compute for the residuals using the LMS estimator.

ri = yi LMS estimator

2. Square each of the residuals

3. Get the mean and standard deviation of the

squared residuals

4. Standardize each of the squared residuals using its

mean and standard deviation

5. Standardized squared residuals that are below -2.5

or above 2.5 are considered outliers. (Rousseeuw)

Robust Regression

13

Remarks:

1. Unlike LS, LMS does not have a closed form formula.

2. Since the median is an order or rank statistic, it is not

amenable to calculation via derivatives or other

calculations that rely on continuous functions.

3. LMS estimator may not be the estimator with the

smallest variance, but it generalizes to multiple

regressions.

4. The position of the LMS estimate lies where the points are

concentrated, not in the center of good observations.

5. The LMS is similar to a mode estimator.

6. For n=3, the LMS estimator is not satisfactory since the

two points have the tendency to be close to each other by

chance making the 3rd one an outlier.

7. The LMS has a 50% breakdown point

Robust Regression

14

The LTS method attempts to minimize the sum of

squared residuals over a subset, k.

trimmed the sum of squares.

Robust Regression

15

How to solve for LTS estimator (one-dimensional case):

Robust Regression

16

Exercises

Nursing Board Exam 2015, Philippine Dairy Inquirer

college and counted the number of handsome boys

entered the building for 30 minutes. The results are given

below:

Robust Regression

17

LMS estimator.

Procedure:

1. Compute for the residuals using the LMS estimator.

ri = yi LMS estimator

2. Square each of the residuals

3. Get the mean and standard deviation of the

squared residuals

4. Standardize each of the squared residuals using its

mean and standard deviation

5. Standardized squared residuals that are below -2.5

or above 2.5 are considered outliers. (Rousseeuw)

Robust Regression

18

Remarks:

1. The number of observations can be drastically

reduced by using the mean of the preceding half.

2. The residuals are squared first and then ordered.

3. According to Rousseeuw (1998), the LTS

procedure is more efficient than the LMS.

4. The objective function of LTS is similar to the

objective function of LS. The only difference is that

the largest squared residuals are not used in the

summation, thus, allowing the fit to stay away from

the outliers.

5. The LTS also has a 50% breakdown point.

Robust Regression

19

References:

Rousseeuw, P. J., & Leroy, A. M. (1987). Robust

Regression and Outlier Detection. Canada.

Jacoby, Bill. Regression III: Advanced Methods.

Michigan State University

Garner, Will. Robust Regression

Simons, Kenneth. (2013). Useful Stata Commands

(for Stata version 12)

Robust Regression

20

- Wal-mart Sales ForecastingUploaded byMithun Mathew
- An Introduction to Robust RegressionUploaded bycrcalub
- MMP using ACEUploaded byMaqsood Iqbal
- Methods in Applied EconometricsUploaded bysravankumar66
- Question Paper Unsolved - Eco No MetricsUploaded byAbhijeet Kulshreshtha
- A Two-stage Robust Power System State Estimation Method With Unkmown Measurement NoiseUploaded byJesus Ortiz Lopez
- Stats Notes l2ss3los10Uploaded bymohuna
- Margins 01Uploaded byhubik38
- Regression With Life DataUploaded byvishp7
- Course_ Decision ScienceUploaded byShrutiKulkarni
- Hetero TestUploaded byLâm Lê Văn
- (P3) Inequality ConvergenceUploaded byNobodycl
- 3.Principals__Leadership_Style_and_School_Perfonnance.pdfUploaded byAmy Lau
- untitledUploaded byFimaIei10
- 1-s2.0-S0261560606000891-mainUploaded byKhairul
- Ch3Uploaded byHarsh bariya
- Dm 26785792Uploaded byAnonymous 7VPPkWS8O
- PARALLEL WISARD OBJECT TRACKER: A RAMBASED TRACKING SYSTEMUploaded bycseij
- Huth ProgramaUploaded byEugenio Fernández Sarasola
- Courseoutline SE(ECO00037M) Term1 201415Uploaded byJoe Ogle
- jurnal internasional 1Uploaded byAzieer Keira Hhelohs
- l25Uploaded bySabbir Pathan
- US Federal Trade Commission: vogtUploaded byftc
- pdfUploaded bycaitlynharvey
- Statistical Analysis to Identify the Main Parameters ToUploaded byInternational Journal of Research in Engineering and Technology
- 01048486 28-20-12Uploaded byIkram Chebout
- 9436Uploaded byNurisra
- BananaUploaded byAYU

- DIRECT SUPERVISORY ADAPTIVE FUZZY CONTROLLER APPLIED TO pH CONTROLUploaded byCarlos Andres
- DualityUploaded byjeaner2008
- Algorithms in DAAUploaded bySajan Antony
- Mutual InformationUploaded bybalasha_937617969
- Inference LogicUploaded byhelperforeu
- Lectures 1-13 +Outlines-6381-2016-Work Book Version.pdfUploaded byAnonymous Y4ajGZ
- AlkalinityUploaded byNeil Nadua
- 273846-strategi-pengembangan-desa-mantar-sebaga-bd22236d.pdfUploaded bysastro
- 3 29 17 notes for combining and composition of functionsUploaded byapi-325344728
- Relating function and 1st derivativeUploaded byMrsFerg7
- Composite FunctionsUploaded byLee Ming Sheng
- Example for Multiple Linear RegressionUploaded bym210192
- Gradient Based Optimization PptUploaded byLuis
- Gauss-jordan Elimination MethodUploaded byRoleine Asegurado Paracuelles
- InformeUploaded byEnmitaDC
- fmipw1-6.pdfUploaded byNguyễn Hữu Phấn
- Demonstrating PID Control Principles Using an Air Heater and LabVIEWUploaded byByron Xavier Lima Cedillo
- DOC316.53.01166_8edUploaded byalonrab
- Fem QuestionpapersUploaded bybookdigit
- ee602 term paper.pdfUploaded byvikram_908116002
- Principalof ExperimentalUploaded byPRASHANTH
- Derivative of SineUploaded byAbhay Sinha
- Alevel C3C4Uploaded bynuddin123
- m62 Stat SampleUploaded byterrygoh6972
- 115706-165Uploaded byVibhanshu Verma
- CantorUploaded byJonathan Bouvier
- MARKOV CHAIN chap4Uploaded byRasull Saimon
- Stock & Watson - Intro to Eco No Metrics (Brief)Uploaded byRhys Souza
- Catalogue v 16Uploaded bypercysearch
- Multiple Dependent Hypothesis Tests in Geographically Weighted RegressionUploaded byGRUNDOPUNK