AD3501 DEEP LEARNING LTPC

3003
COURSE OBJECTIVES:

To understand and need and principles of deep neural networks

To understand CNN and RNN architectures of deep neural networks
To comprehend advanced deep learning models
To learn the evaluation metrics for deep learning models

UNIT I DEEP NETWORKS BASICS 9

Linear Algebra: Scalars -- Vectors -- Matrices and tensors; Probability Distributions -- Gradient-
based Optimization – Machine Learning Basics: Capacity -- Overfitting and underfitting --
Hyperparameters and validation sets -- Estimators -- Bias and variance -- Stochastic gradient
descent -- Challenges motivating deep learning; Deep Networks: Deep feedforward networks;
Regularization -- Optimization.

UNIT II CONVOLUTIONAL NEURAL NETWORKS 9

Convolution Operation -- Sparse Interactions -- Parameter Sharing -- Equivariance -- Pooling --

Convolution Variants: Strided -- Tiled -- Transposed and dilated convolutions; CNN Learning:
Nonlinearity Functions -- Loss Functions -- Regularization -- Optimizers --Gradient Computation.

UNIT III RECURRENT NEURAL NETWORKS 10

Unfolding Graphs -- RNN Design Patterns: Acceptor -- Encoder --Transducer; Gradient

Computation -- Sequence Modeling Conditioned on Contexts -- Bidirectional RNN -- Sequence
to Sequence RNN – Deep Recurrent Networks -- Recursive Neural Networks -- Long Term
Dependencies; Leaky Units: Skip connections and dropouts; Gated Architecture: LSTM.

UNIT IV MODEL EVALUATION 8

Performance metrics -- Baseline Models -- Hyperparameters: Manual Hyperparameter --

Automatic Hyperparameter -- Grid search -- Random search -- Debugging strategies.

UNIT V AUTOENCODERS AND GENERATIVE MODELS 9

Autoencoders: Undercomplete autoencoders -- Regularized autoencoders -- Stochastic

encoders and decoders -- Learning with autoencoders; Deep Generative Models: Variational
autoencoders – Generative adversarial networks.

TOTAL: 45 PERIODS
COURSE OUTCOMES
After the completion of this course, students will be able to:

CO1:Explain the basics in deep neural networks

CO2:Apply Convolution Neural Network for image processing
CO3:Apply Recurrent Neural Network and its variants for text analysis
CO4:Apply model evaluation for various applications
CO5:Apply autoencoders and generative models for suitable applications
TEXT BOOK

1. Ian Goodfellow, Yoshua Bengio, Aaron Courville, ``Deep Learning'', MIT Press, 2016.
2. Andrew Glassner, “Deep Learning: A Visual Approach”, No Starch Press, 2021.

REFERENCES

1. Salman Khan, Hossein Rahmani, Syed Afaq Ali Shah, Mohammed Bennamoun, ``A Guide to
Convolutional Neural Networks for Computer Vision'', Synthesis Lectures on Computer Vision,
Morgan & Claypool publishers, 2018.
2. Yoav Goldberg, ``Neural Network Methods for Natural Language Processing'', Synthesis
Lectures on Human Language Technologies, Morgan & Claypool publishers, 2017.
3. Francois Chollet, ``Deep Learning with Python'', Manning Publications Co, 2018.
4. Charu C. Aggarwal, ``Neural Networks and Deep Learning: A Textbook'', Springer
International Publishing, 2018.
5. Josh Patterson, Adam Gibson, ``Deep Learning: A Practitioner's Approach'', O'Reilly Media,
2017.