Scribd Logo
Upload

Welcome to Scribd!

  • EE769-3 Linear Regression

    Uploaded by

    vinit gurav
    11 views14 pages
    This document summarizes an introductory machine learning course on linear regression. It outlines the course objectives, including writing the objective of regression, decomposing loss into bias and variance, and adding regularization to least squares. It also provides an overview of linear regression utilities, problem setup assuming ideal predictions are linear, bias-variance decomposition, regularization techniques like L2 penalty, and iterative optimization algorithms like recursive least squares.

    Original Description:

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    This document summarizes an introductory machine learning course on linear regression. It outlines the course objectives, including writing the objective of regression, decomposing loss into bias and variance, and adding regularization to least squares. It also provides an overview of linear regression utilities, problem setup assuming ideal predictions are linear, bias-variance decomposition, regularization techniques like L2 penalty, and iterative optimization algorithms like recursive least squares.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    11 views14 pages

    EE769-3 Linear Regression

    Uploaded by

    vinit gurav
    This document summarizes an introductory machine learning course on linear regression. It outlines the course objectives, including writing the objective of regression, decomposing loss into bias and variance, and adding regularization to least squares. It also provides an overview of linear regression utilities, problem setup assuming ideal predictions are linear, bias-variance decomposition, regularization techniques like L2 penalty, and iterative optimization algorithms like recursive least squares.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 14

    EE769 Intro to ML

    Linear Regression
    Amit Sethi
    Faculty member, IIT Bombay
    Learning outcomes

    • Write the objective of regression


    • Write the expression for analytical solution
    • Decompose the loss into bias and variance
    • Add regularization to least square
    • Write the algorithm for computational solution
    Outline
    • Utility of linear regression

    • Problem setup

    • Bias-variance decomposition

    • Regularization

    • Iterative solution
    Utility of linear regression
    • Simplest to code

    • Simplest to understand mathematically

    • Nonlinear feature extraction can be used with linear


    solution
    Problem setup
    • Assuming ideal prediction is linear w.r.t. input

    • Or linear with respect to features

    Source: PRML book by C Bishop


    Measured output has stochastic noise
    • Assuming Gaussian noise

    Source: PRML book by C Bishop


    Likelihood of observation
    • Assuming identically independently distributed
    samples

    Source: PRML book by C Bishop


    Maximizing likelihood
    • Set gradients w.r.t. w to zero

    Source: PRML book by C Bishop


    Expected loss (MSE) (1/3)
    Bias-variance decomposition

    Source: PRML book by C Bishop


    Expected loss (MSE) (2/3)
    Bias-variance decomposition

    Source: PRML book by C Bishop


    Expected loss (MSE) (3/3)
    Bias-variance decomposition

    Source: PRML book by C Bishop


    Regularization using L2 penalty

    Source: PRML book by C Bishop


    Geometry of Lp regularization

    Source: PRML book by C Bishop


    Iterative optimization
    • Iterative minimization of convex objectives

    • Recursive least square update

    Source: PRML book by C Bishop

    You might also like