Machine Learning for Finance

UCEMA 2020

Pablo A Macri 1
Lecture 2
Bias-variance tradeoff
Training and test sets
Model capacity and overfitting
Learning in linear regressions
Regularizations
Hyperparameters and cross validation
Gradient descent-like algorithms
Probabilistic models, MLE, MAP
Probabilistic classifications
Logistic regressions
Scikit-Learn demo
Pablo A Macri 2
Lectures Structure
1st block
~1h: Presentation of subjects + code explanation.
~15’: Discussion + Q&A.
2nd block
~1h: Presentation of subjects + code explanation.
~15’: Discussion + Q&A.

Pablo A Macri 3
Generalization as a Goal of Learning

Pablo A Macri 4
Generalization Error in Regression

Pablo A Macri 5
Generalization Error in Regression

Pablo A Macri 5
Generalization Error in Regression

Pablo A Macri 5
Generalization Error in Regression

Pablo A Macri 5
Generalization Error in Regression

Pablo A Macri 5
Generalization Error in Regression

Pablo A Macri 5
Generalization Error in Regression

Pablo A Macri 5
Generalization Error in Regression

noise (σ2)
Pablo A Macri 5
Generalization Error in Regression

noise (σ2) variance

Pablo A Macri 5
Generalization Error in Regression

noise (σ2) variance bias

Pablo A Macri 5
The Bias-Variance Decomposition

Pablo A Macri 6
The Bias-Variance Decomposition

Pablo A Macri 6
The Bias-Variance Decomposition

Pablo A Macri 6
The Bias-Variance Decomposition

Pablo A Macri 6
The Bias-Variance Decomposition

Pablo A Macri 6
The Bias-Variance Tradeoff

Pablo A Macri 7
The Bias-Variance Tradeoff

Pablo A Macri 7
The Bias-Variance Tradeoff

Pablo A Macri 7
The Bias-Variance Tradeoff

Pablo A Macri 7
The Bias-Variance Tradeoff

Pablo A Macri 7
Data Representation

Pablo A Macri 8
Structured Data

n instances

n targets
m features

Pablo A Macri 9
Unstructured Data (Images)

Pablo A Macri 10
Training and Test Sets

Pablo A Macri 11
Training and Test Sets

Pablo A Macri 11
Training and Test Sets

Pablo A Macri 11
Training and Test Sets

Pablo A Macri 11
Training and Test Sets

Pablo A Macri 11
Training and Test Sets

Pablo A Macri 11
Training and Test Sets

Pablo A Macri 11
The Bias-Variance Tradeoff

Pablo A Macri 12
Overfitting

Pablo A Macri 13
Overfitting

Pablo A Macri 13
Overfitting

Pablo A Macri 13
Overfitting

Pablo A Macri 13
Overfitting

Pablo A Macri 13
Model Capacity and Overfitting

Pablo A Macri 14
Linear Regression as a ML Task

Pablo A Macri 15
Linear Regression as a ML Task

Pablo A Macri 15
Linear Regression as a ML Task

Pablo A Macri 15
Linear Regression as a ML Task

Pablo A Macri 15
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 16
Learning in Linear Regression

Pablo A Macri 17
Learning in Linear Regression

Pablo A Macri 17
Learning in Linear Regression

Sir R. Penrose
Nobel Prize in Physics 2020

Pablo A Macri 17
Learning in Linear Regression

Sir R. Penrose
Nobel Prize in Physics 2020

Pablo A Macri 17
Learning in Linear Regression

Sir R. Penrose
Nobel Prize in Physics 2020

Pablo A Macri 17
Regularization

Pablo A Macri 18
Regularization

Pablo A Macri 18
Regularization

Pablo A Macri 18
Hyperparameters and Validation Set

Pablo A Macri 19
Hyperparameters and Validation Set

Pablo A Macri 19
Hyperparameters and Validation Set

Pablo A Macri 19
Hyperparameters and Validation Set

Pablo A Macri 19
Hyperparameters and Validation Set

Pablo A Macri 19
Cross-Validation

Pablo A Macri 20
Cross-Validation

Pablo A Macri 20
Cross-Validation

Pablo A Macri 20
Cross-Validation

Pablo A Macri 20
Supervised Learning Diagram

Pablo A Macri 21
Analytical Learning in Linear Reg

Pablo A Macri 22
Analytical Learning in Linear Reg

Pablo A Macri 22
Analytical Learning in Linear Reg

Pablo A Macri 22
Analytical Learning in Linear Reg

Pablo A Macri 22
Analytical Learning in Linear Reg

Pablo A Macri 22
Gradient Descent

Pablo A Macri 23
Gradient Descent

Pablo A Macri 23
Gradient Descent

Pablo A Macri 23
Gradient Descent

Pablo A Macri 23
Gradient Descent

Pablo A Macri 24
Gradient Descent

Pablo A Macri 24
Choice of Learning Rate

Pablo A Macri 25
Choice of Learning Rate

Pablo A Macri 25
Choice of Learning Rate

Pablo A Macri 25
Choice of Learning Rate

Pablo A Macri 25
Choice of Learning Rate

Pablo A Macri 25
Stochastic Gradient Descent

Pablo A Macri 26
Stochastic Gradient Descent

Pablo A Macri 26
Stochastic Gradient Descent

Pablo A Macri 26
Stochastic Gradient Descent

Pablo A Macri 26
Stochastic Gradient Descent

Pablo A Macri 26
Stochastic Gradient Descent

Pablo A Macri 26
Stochastic Gradient Descent

Pablo A Macri 26
Advanced Variants of SGD

Pablo A Macri 27
The Bayesian Recipe

Pablo A Macri 28
ML Training for Probabilistic Models

Pablo A Macri 29
ML Training for Probabilistic Models

Pablo A Macri 29
ML Training for Probabilistic Models

Pablo A Macri 29
ML Training for Probabilistic Models

Pablo A Macri 29
ML Training for Probabilistic Models

Pablo A Macri 29
Maximum Likelihood Estimation MLE

Pablo A Macri 30
Why MLE

Pablo A Macri 31
Maximum A-Posteriori Estimation

Pablo A Macri 32
MLE and Least Squares Loss

Pablo A Macri 33
MLE and Least Squares Loss

Pablo A Macri 33
MLE and Least Squares Loss

Pablo A Macri 33
MLE and Least Squares Loss

Pablo A Macri 33
Kullback-Leible (KL) Divergence

Pablo A Macri 34
MAP and Regularization

Pablo A Macri 35
MAP and Regularization

Pablo A Macri 35
MAP and Regularization

Pablo A Macri 35
Supervised Learning Algorithms

Pablo A Macri 36
Probabilistic Classification Models

Pablo A Macri 37
Probabilistic Classification Models

Pablo A Macri 37
Probabilistic Classification Models

Pablo A Macri 37
Probabilistic Classification Models

Pablo A Macri 37
Logistic Regression

Pablo A Macri 38
Logistic Regression

Pablo A Macri 38
Logistic Regression

Pablo A Macri 38
Logistic Regression

Pablo A Macri 38
MLE for Logistic Regression

Pablo A Macri 39
Regression and Classification Demo

We will use the scikit-learn package to illustrate

all we have seen during this lecture
Pablo A Macri 40