DMIS 2022

Tutorial: Fundamentals of Deep Learning

Pr. Abdelouahab Moussaoui
Duration: 3 half days.

Prerequisites: An understanding of fundamental programming concepts in Python 3, such as functions,

loops, dictionaries, and arrays; familiarity with Pandas data structures; and an understanding of how to
compute a regression line.

Suggested materials to satisfy prerequisites: Python Beginner’s Guide.

Technologies: Tensorflow 2 with Keras, Pandas

Assessment Type: Skills-based coding assessments evaluate students’ ability to train a deep learning model
to high accuracy.

Hardware Requirements:

Desktop or laptop computer capable of running the latest version of Chrome or Firefox and
anaconda distribution.
Internet connection.
Operating System: Windows with 3 GB space or more to install anaconda distribution and databases.

Installation required:

Anaconda Distribution 64-bit with Python 3: latest version

TensorFlow: latest version
Karas: latest version
Number of trainees: 30 trainees max.

Languages: French.

Tutorial Outline

Class 0 — Jupyter notebooks manipulating and learning

 Episode 1 — Linear regression with direct resolution

 Low-level implementation, using numpy, of a direct resolution for a linear regression

 Episode 2 — Linear regression with gradient descent

 Low level implementation of a solution by gradient descent. Basic and stochastic approach.

 Episode 3 — Logistic regression

 Simple example of logistic regression with a sklearn solution

Class 1 — Basic regression using DNN

 Episode 1 — Regression with a Dense Network (DNN)

 Simple example of a regression with the dataset Boston Housing Prices Dataset (BHPD)

 Episode 2 — Regression with a Dense Network (DNN) - Advanced code

  A more advanced implementation of the precedent example

Class 2 — Basic classification using a DNN

 Episode 1 — Simple classification with DNN

 An example of classification using a dense neural network for the famous MNIST dataset

 Episode 2 — Simple classification with CNN

 An example of classification using a convolutional neural network for the famous MNIST dataset

Class 3 — Working with Convolutional Neural Networks (CNN) : Case of Images classification

 Episode 1 — Dataset analysis and preparation

 Analysis of the German Traffic Sign Recognition Benchmark (GTSRB) dataset and creation of an
enhanced dataset

 Episode 2 — First convolutions

 First convolutions and first classification of our traffic signs

 Episode 3 — Training monitoring

 Monitoring, analysis and check points during a training session

 Episode 4 — Data augmentation

 Adding data by data augmentation when we lack it, to improve our results

 Episode 5 — Full convolutions

 A lot of models, a lot of datasets and a lot of results.

Class 4 — Working with Recurrent Neural Networks (RNN): Case of Sentiment Analysis

 Episode 1 — Sentiment analysis with hot-one encoding

 A basic example of sentiment analysis with sparse encoding, using a dataset from Internet Movie
Database (IMDB)

 Episode 2 — Sentiment analysis with text embedding

 A very classical example of word embedding with a dataset from Internet Movie Database (IMDB)

 Episode 3 — Reload and reuse a saved model

 Retrieving a saved model to perform a sentiment analysis (movie review)

 Episode 4 — Reload embedded vectors

 Retrieving embedded vectors from our trained model

 Episode 5 — Sentiment analysis with a RNN network

 Still the same problem, but with a network combining embedding and RNN

Class 5 — Unsupervised learning with an autoencoder neural network (AE)

 Episode 1 — Prepare a noisy MNIST dataset

  Preparation of a noisy MNIST dataset

 Episode 2 — Building and training an AE denoiser model

 Construction of a denoising autoencoder and training of it with a noisy MNIST dataset.

 Episode 3 — Playing with our denoiser model

 Using the previously trained autoencoder to denoise data

 Episode 4 — Denoiser and classifier model

 Construction of a denoiser and classifier model

 Episode 5 — Advanced denoiser and classifier model

 Construction of an advanced denoiser and classifier model

Class 6 — Generative network with Variational Autoencoder (VAE)

 Episode 1 — First VAE, using functional API (MNIST dataset)

 Construction and training of a VAE, using functional APPI, with a latent space of small dimension.

 Episode 2 — VAE, using a custom model class (MNIST dataset)

 Construction and training of a VAE, using model subclass, with a latent space of small dimension.

 Episode 3 — Analysis of the VAE's latent space of MNIST dataset

 Visualization and analysis of the VAE's latent space of the dataset MNIST

Class 7 — Generative Adversarial Networks (GANs)

 Episode 1 — A first DCGAN to Draw a Sheep

 Draw me a sheep, revisited with a DCGAN

 Episode 2 — A WGAN-GP to Draw a Sheep

 Draw me a sheep, revisited with a WGAN-GP