Deep Learning [RCS-086]

• Books:
1. Cosma Rohilla Shalizi, Advanced Data Analysis
from an Elementary Point of View, 2015.
2. Deng & Yu, Deep Learning: Methods and
Applications, Now Publishers, 2013.
3. Ian Goodfellow, Yoshua Bengio, Aaron Courville,
Deep Learning, MIT Press, 2016.
4. Michael Nielsen, Neural Networks and Deep
Learning, Determination Press, 2015.
Unit-I
INTRODUCTION : Introduction to machine learning- Linear models (SVMs and Perceptrons,
logistic regression)- Intro to Neural Nets: What a shallow network computes- Training a network:
loss functions, back propagation and stochastic gradient descent- Neural networks as universal
function approximates
Unit-II
DEEP NETWORKS : History of Deep Learning- A Probabilistic Theory of Deep Learning-
Backpropagation and regularization, batch normalization- VC Dimension and Neural Nets-Deep
Vs Shallow Networks-Convolutional Networks- Generative Adversarial Networks (GAN),
Semisupervised Learning
Unit-III
DIMENTIONALITY REDUCTION 9 Linear (PCA, LDA) and manifolds, metric learning - Auto
encoders and dimensionality reduction in networks - Introduction to Convnet - Architectures –
AlexNet, VGG, Inception, ResNet - Training a Convnet: weights initialization, batch
normalization, hyperparameter optimization
Unit-IV
OPTIMIZATION AND GENERALIZATION : Optimization in deep learning– Non-convex
optimization for deep networks- Stochastic Optimization Generalization in neural networks-
Spatial Transformer Networks- Recurrent networks, LSTM - Recurrent Neural Network Language
Models- Word-Level RNNs & Deep Reinforcement Learning - Computational & Artificial
Neuroscience
Unit-V
CASE STUDY AND APPLICATIONS : Imagenet- Detection-Audio WaveNet-Natural Language
Processing Word2Vec - Joint Detection-Bioinformatics- Face Recognition- Scene Understanding-
Gathering Image Captions