11/5/2020 DEEP LEARNING · Deep Learning
DEEP LEARNING
DS-GA 1008 · SPRING 2020 · NYU CENTER FOR DATA SCIENCE
INSTRUCTORS Yann LeCun & Alfredo Canziani
LECTURES Mondays 16:55 – 18:35, GCASL C95
PRACTICA Tuesdays 19:10 – 20:00, GCASL C95
PIAZZA Access code: DLSP20
MATERIAL Google Drive, Notebooks
Description
This course concerns the latest techniques in deep learning and representation learning,
focusing on supervised and unsupervised deep learning, embedding methods, metric
learning, convolutional and recurrent nets, with applications to computer vision, natural
language understanding, and speech recognition. The prerequisites include: DS-GA 1001
Intro to Data Science or a graduate-level machine learning course.
Lectures
Legend: 🖥 slides, 📓 Jupyter notebook, 🎥 YouTube video.
Week Format Title Resources
History and motivation
Lecture 🖥🎥
① Evolution and DL
Practicum Neural nets (NN) 📓📓🎥
SGD and backprop
Lecture 🖥🎥
② Backprop in practice
Practicum NN training 🖥📓📓🎥
Parameter transformation
Lecture 🖥🎥
③ CNN
Practicum Natural signals' properties 🖥📓🎥
④ Practicum 1D convolutions 📓🎥
https://atcold.github.io/pytorch-Deep-Learning/ 1/3
11/5/2020 DEEP LEARNING · Deep Learning
Week Format Title Resources
Optimisation I
Lecture 🖥🎥
⑤ Optimisation II
Practicum CNN, autograd 📓📓🎥
CNN applications
Lecture 🖥🖥🎥
⑥ RNNs and attention
Practicum Training RNNs 📓📓🖥🎥
Energy-Based Models
Lecture 🖥🎥
⑦ SSL, EBM
Practicum Autoencoders 🖥📓🎥
Contrastive methods
Lecture 🖥🎥
⑧ Regularised latent
Practicum Training VAEs 🖥📓🎥
Sparsity
Lecture 🖥🎥
⑨ World model, GANs
Practicum Training GANs 🖥📓🎥
CV SSL I
Lecture 🖥🎥
⑩ CV SSL II
Practicum Predictive Control 🖥📓🎥
Activations
Lecture 🖥🖥🖥🎥
⑪ Losses
Practicum PPUU 🖥📓🎥
DL for NLP I
Lecture 🖥🎥
⑫ DL for NLP II
Practicum Attention & transformer 🖥📓🎥
GCNs I
Lecture 🖥🎥
⑬ GCNs II
Practicum GCNs III 🖥📓🎥
Structured Prediction
Lecture 🖥🎥
⑭ Graphical methods
Practicum Regularisation and Bayesian 🖥📓🖥📓🎥
https://atcold.github.io/pytorch-Deep-Learning/ 2/3
11/5/2020 DEEP LEARNING · Deep Learning
People
Role Photo Contact About
Silver Professor in CS at
Yann LeCun
Instructor NYU
yann@cs.nyu.edu
and Turing Award winner
Alfredo Canziani
Instructor Asst. Prof. in CS at NYU
canziani@nyu.edu
Mark Goldstein
Assistant PhD student in CS at NYU
goldstein@nyu.edu
Zeming Lin
Webmaster PhD student in CS at NYU
zl2799@nyu.edu
Disclaimer
All other texts found on this site are lecture notes taken by students of the New York
University during lectures given by Yann Le Cun, Alfredo Canziani, Ishan Misra, Mike Lewis
and Xavier Bresson. Thus the texts in English were written by about 130 people, which has
an impact on the homogeneity of the texts (some write in the past tense, others in the
present tense; the abbreviations used are not always the same; some write short
sentences, while others write sentences of up to 5 or 6 lines, etc.). It is possible that there
may be some omissions: typing errors, spelling mistakes, etc. If you notice any, we invite
you to submit a PR on the GitHub directory of the site specifying with an [EN] that it
concerns the English translation.
Wishing you a deep reading !
https://atcold.github.io/pytorch-Deep-Learning/ 3/3
