Development of Deep Learning Architecture: Pantech Solutions & The Institution of Electronics and Telecommunication

Development of Deep
Learning Architecture
Organized by
Pantech Solutions
&
The Institution of Electronics and
Telecommunication
Instruction for Attendees
All Participants, Please fill the

Attendance form, Link given in
the description
www.pantechsolutions.net
About IETE
Profile
Founded in 1953|1,25,000 members | 64 Centers
IETE Focus on
IETE conducts Technical Meetings | Conferences | Symposia
| Exhibitions | delivering career advancement opportunity
and publishes Technical journals
IETE Exams
Computer Science & Engineering and Information
Technology streams and Diploma DIPIETE Exams|
ALCCS Post graduate source
About Pantech
Profile
Established on 2004 | 7 Branches | 100+ Team
R&D
Manufacturer of Lab equipment‘s & Development boards |
Industrial &Funded projects | Online retail store of
engineering products and projects
Online Retail Store

Lab equipment, Engineering Kits, Components, Sensors
and All level Projects
Training
1500+ Workshops | 250+ FDP | 100+ Seminars
Pantech Technology
Electronics AI
Manufacturer of Embedded Machine Learning | Deep Learning |
development Boards | FPGA Boards | NLP | Chatbot | Block chain | Data
Sensors | Interface Modules science and AI boards
“To Gain Global
Electrical Leadership in Sof tware
providing AR & VR products | Android
E- Vehicle | Renewable energy
Technological development | Cyber security |
systems | Motors and Drivers |Power
Solutions through Cloud & Mobile computing | IoT
electronics | Power Systems
Sustained
Innovation”
Mechanical Brain Computer Interface

Robotics | Mechanical components | 3D Brain wave analysis | Brain controlled
design and printing | applications | EEG headband
Agenda
01
General talk on AI
02 About Deep Learning & Libraries
Deep Learning Algorithm

03 ANN | RNN | CNN
04 Application of Deep Learning
Demo on Character recognition &

05
Emotion Recognition www.pantechsolutions.net
What is AI
Intelligence demonstrated
by the Machines in
contrast to Natural
Intelligence delivered by
Humans
“AI WILL BE THE BEST OR
WORST THING EVER FOR
HUMANITY.” – ELON MUSK
Intelligence is given by
“SUCCESS IN CREATING AI feeding the experience
WOULD BE THE BIGGEST EVENT of human to machines
IN HUMAN HISTORY. in the form of data
UNFORTUNATELY, IT MIGHT
ALSO BE THE LAST, UNLESS WE
LEARN HOW TO AVOID THE
RISKS.” – STEPHEN HAWKING www.pantechsolutions.net
Deep Learning
Subset of Machine learning, Algorithms inspired by the
structure and function of Human Brain
Recognition
Brain
Vision
Training Model
Dataset Recognition
Why Deep Learning ?
Input Data Automatic Feature
Extraction
Prediction
AI
Its an Apple
Model
Why GPU
“It Can train more number

GPU – Graphical Processing Unit
of data in short time
period & Better for real High Bandwidth | Thread Parallelism |
time AI based application” 100% Easily programmable registers |
Bandwidth Optimized
CPU –Central Processing Unit

Less Bandwidth | Latency Optimised | 50%
Not suitable for Real time application
where performance is the main Factor
Deep Learning Libraries ?
Tensor Flow
Keras
Numerical computation using data flow
Framework for Deep learning | Same
graphs | Backend for Keras |
code for CPU & GPU | Uses Theano/TF
Distributed computing on multiple GPU
as Backend | CNN | not support multi
GPU
Numerical Python & Scientific
Python Torch
Basic operation such as Sorting, Old ML & DL library | Supports
Reshaping, indexing | Scientific CUDA for parallel computation |
computing | Most new feature Supervised image problem with
belong in SciPy rather than CNN
Numpy
Mxnet Caffe
More number of Language Very high performance | Tuning Hyper
Bindings | Distributed computing parameters | CNN & RNN(Facebook
extends) | | CPU & GPU
Neuron & Activation Fn.
Neuron
• Like a Human Brain, here a Neuron takes input and do some
function to give the output
Cell Body Synapse
• Function going to be the Mathematical function Dendrites
• Those Function is known as ACTIVATION function Axon
Activation Function
• Step Function
• Sigmoid Function
Neuron
• Tanh Function
Inputs OUTPUT
• ReLU Function
Activation Fn.
Activation Function
STEP Function
• If value of X is greater than or equal to 0, then output is 1,

If value of X is less than 0, then output is 0
• NN uses back propagation & Gradient descent method to
calculate weight of different layers
• Since step Function is non differentiable to zero, it can’t do
the gradient descent method, so it can’t update weights.
SIGMOID Function Image source from Towards Data Science
• If value of X is infinity, then output is 1,

If value of X is negative infinity, then output is 0
• It captures non-linearity in the data
• It can use Gradient descent & Back propagation method to
calculate weights.
• Output range [0,1]
Activation Function
Tanh Function
• Rescaled of Sigmoid Function

• Output range [-1,1]
• Better learning rate requires, higher gradient. In some times,
for the data is centred around 0, derivatives are higher.
ReLU Function
• Rectified Linear Unit. if any negative input is detected, it

returns 0, otherwise it returns the value back.
Leaky ReLU Function
• Same as ReLU, like returning same for Positive. But for

negative values instead of returning zero, it has constant slope
Activation Function
SOFTMAX Function
• Softmax activation function will be applied in the last layer

of Neural network, instead of ReLU, tanh, Sigmoid.
• It is used to map the non-normalized output of a network to

a probability distribution over predicted output class. That is
it converts output of last layer into a essential probability
distribution.
Deep Learning Algorithm
ARTIFICIAL NEURAL
01 NETWORK (ANN)
RECURRENT NEURAL
02 NETWORK (ANN)
CONVOLUTIONAL NEURAL
03 NETWORK (ANN)
Vanishing & Exploding Gradient
• It is very common problem in every Neural Network, which is associated with Backpropagation.
• Weights of network are updated through backpropagation by finding gradients.
• When the number of hidden layer is high, then the gradient vanishes or explodes as it
propagates backward. It leads instability in network, unable to learn from training
• The explosion occurs through exponential growth by repeatedly multiplying gradients through
the network layers that have values larger than 1.0
• It can be fixed by redesigning the network, using Long Short Term Memory networks, Gradient
clipping, etc.
ANN
• Learns any Non-Linear Function, It is known as Universal
Function Approximators
• Activation Function introduce non linear property to network,

so it will identify complex relationship between input &
output
• Output of each neuron is the activation of weighted sum of

Input, If there is no Activation function, network can't learn
non-linear function
• Feed Forward Neural Network – Input processed in one Output

direction, When hidden layer is more than one, that is Deep
Neural Network
Input
Hidden
RNN
• Looping system in hidden layer of ANN is known as RNN
• It captures sequential info of input data, that is dependency

between words to make prediction. Whereas, ANN cannot
capture sequential information
• RNN shares parameters across different time steps, so that

there will be few parameter to train
• It is the time series version of ANN. Common Recurrent

layers are LSTM(Long Short Term Memory) & GRU (Grated
Recurrent Units)
Output
• GRU is used to how much pass data needed to flow through
model
Input
• It is mostly used in NLP (Natural Language Processing) Hidden
CNN
• CNN learns the filter automatically to extract the right features from the data
• It captures spatial features (Arrangement of pixels) whereas ANN can’t.
• It also follows parameter sharing like RNN, applies single filter in different part of single image.
Whereas ANN can’t.
• It don’t have recurrent connections like RNN, instead it has convolution type of hidden layers
• Convolution and pooling functions are used as activation functions
• CONVOLUTION: Input image and other as Filter on input image(Kernel) produces output image.
• POOLING: picking maximum value from selected region is Max pooling and vice versa.
CNN Architecture
FC_4
Fully Connected
FC_3 Neural Network
Conv_2
Max-pooling
(2x2)
Fully Connected
ReLU Activation
0
Convolution
Conv_1
Max-pooling
(2x2)
(5x5)
1
Convolution
(5x5) .
. 2
.
.
9
.
. .
.
.
. Output
.
.
Input .
28x28x1 n1 channels n1 channels n2 channels Flattened

n3 units
(24 x 24 x n1) (12 x 12 x n1) (8 x 8 x n2) n2 channels
(4 x 4 x n2) www.pantechsolutions.net
Simple Softmax Classification
784 Pixels
….
.
.
.
.
Input
28x28x1
.
.
.
.
0 1 2 3 9
100 image at a time Weights
784x10 Matrix
W0,0 W0,1 W0,2 ……W0,9
W1,0 W1,1 W1,2 ……W1,9
W2,0 W2,1 W2,2 ……W2,9
W3,0 W3,1 W3,2 ……W3,9
X
W4,0 W4,1 W4,2 ……W4,9
Input Image .
100x784 (100 images Flattened) .
.
X W783,0 W783,1 W783,2 ……W783,9 Bias
1x10 Matrix
… 100x10 Matrix
.
… L0,0 W0,1 W0,2 ……W0,9 b0 b1 b2 b3 ….. b9
. L1,0 W1,1 W1,2 ……W1,9
… L2,0 W2,1 W2,2 ……W2,9
. L3,0 W3,1 W3,2 ……W3,9
+
. L4,0 W4,1 W4,2 ……W4,9
. .
. .
.
… L99,0 L99,1 L99,2 ……W99,9
. www.pantechsolutions.net
In Tensor Flow
W[784x10]
X[100x784]
Y[100x10]
b[10]
Y =
tf.nn.softmax(tf.matmul(X,W)+b)
Cross Entropy 0 1 2 3 4 5 6 7 8 9
Actual Probability
0 0 1 0 0 0 0 0 0 0
0 1 2 3 4 5 6 7 8 9
Computed Probability 0.1 0.2 0.9 0.5 0.3 0.1 0.2 0.1 0.3 0.1
Application of Deep Learning
AI Doctor
It is kind of machine analyses the symptoms and cause and
suggesting the Medicine for the patient. Those knowledge is fed by
tons of Medical data, based on the applications.
Prediction of Disease from Medical Image

It is the application uses medical images, to classify the diseased and
healthy like Diabetic Retinopathy
Voice recognition for ALS Patient

ALS (amyotrophic lateral sclerosis) is a kind of disease which causes speech
problems, the voice of the patients completely non understandable. AI helps
to train the voice of ALP patients to deliver voice based assistance for them.
Application of Deep Learning
Fruit & vegetable classification
After cultivation, Fruits and Vegetables needed to be segregated based
on the quality like Rotten or Healthy. AI application will done this with
great accuracy
AI in Cooking
Now AI is used for studying and identifying the pattern of molecular structure of
Food Items to deliver a new dish recipes.
AI in voice Assistance
AI in Autonomous Vehicle
Demo
Session
Q&A
Session
THANK YOU
Upcoming Session
May 30 – Conversational Bot design
April 2 – Brain Computer Interface

Development of Deep Learning Architecture: Pantech Solutions & The Institution of Electronics and Telecommunication

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Development of Deep Learning Architecture: Pantech Solutions & The Institution of Electronics and Telecommunication

Uploaded by

Copyright:

Available Formats

Development of Deep

All Participants, Please fill the

Online Retail Store

Mechanical Brain Computer Interface

02 About Deep Learning & Libraries

Deep Learning Algorithm

04 Application of Deep Learning

Demo on Character recognition &

“It Can train more number

CPU –Central Processing Unit

• Those Function is known as ACTIVATION function Axon

• If value of X is greater than or equal to 0, then output is 1,

SIGMOID Function Image source from Towards Data Science

• If value of X is infinity, then output is 1,

• Rescaled of Sigmoid Function

• Rectified Linear Unit. if any negative input is detected, it

• Same as ReLU, like returning same for Positive. But for

• Softmax activation function will be applied in the last layer

• It is used to map the non-normalized output of a network to

• Weights of network are updated through backpropagation by finding gradients.

• Activation Function introduce non linear property to network,

• Output of each neuron is the activation of weighted sum of

• Feed Forward Neural Network – Input processed in one Output

• It captures sequential info of input data, that is dependency

• RNN shares parameters across different time steps, so that

• It is the time series version of ANN. Common Recurrent

• It captures spatial features (Arrangement of pixels) whereas ANN can’t.

• Convolution and pooling functions are used as activation functions

28x28x1 n1 channels n1 channels n2 channels Flattened

Prediction of Disease from Medical Image

Voice recognition for ALS Patient

You might also like