Ch7 Moving Beyond Linearity

Mutahira Khalid Chaudry & Syed Shaheryar Zahur

Convergent Business Technologies. Confidential.

 Moving Beyond Linearity
• Approach
• Extension of linear model
• Polynomial Regression
• Step Functions
• Regression Splines
• Smoothing Splines
• Local Regression
• Generalized Additive models

Convergent Business Technologies. Confidential.

 Polynomial Regression
• Approach
• Measurement function

Convergent Business Technologies. Confidential.

 Step Functions
• Approach
• Measurement function

Convergent Business Technologies. Confidential.

 Basis Function
• Approach
• Measurement Function

• For polynomial Regression

• For piecewise constant function

• Alternative approaches
• Regression splines

Convergent Business Technologies. Confidential.

 Regression Splines
• Approach
• Piecewise polynomials
• Measurement function

• Adding constraints and splines

Convergent Business Technologies. Confidential.

 Regression Splines
• Can use basis model for representation
• Start with basis function
• Add truncated power basis function per knot

• Problem:
• High variance at outer range of predictors
• To cater - use natural splines

Convergent Business Technologies. Confidential.

 Regression Splines
• Choosing the number and location of knots:
• Regions with more knots, more flexible
• How to find best number of knots:
• Use cross-validation

Convergent Business Technologies. Confidential.

 Regression Splines
• Comparison to Polynomial Regression:
• Superior results

Convergent Business Technologies. Confidential.

 Smoothing Splines
• Want to fit a smooth curve to dataset
• Use smoothing spline

λ – tuning parameter
• Larger λ, smoother the function
• Smaller λ, more jumpy the function

Convergent Business Technologies. Confidential.

 Smoothing Splines
• Choosing the Smoothing parameter λ:
• λ controls the effective degrees of freedom
• Measure of flexibility of smoothing spline

• Knot at each training observation

• Problem : finding value of λ
• Use cross-validation

Convergent Business Technologies. Confidential.

 Local Regression
• Fit at target point, using only nearby observations

• Most important choice – span

Convergent Business Technologies. Confidential.

 Generalized Additive Models
• Framework for extending linear model
• Allows non-linear functions of each variable
• Maintains additivity

Convergent Business Technologies. Confidential.

 Generalized Additive Models

• GAM allows us to model non-linear relationships

• Potentially make better predictions
• Allows better inference
• But also possible to miss important interactions
Convergent Business Technologies. Confidential.
 Thank
Thank you.
you.

©2019 Convergent Business Technologies

Convergent Business Technologies. Confidential.