Program: B.

Tech, CSE, 6th sem, 3rd year

CS 601: Machine Learning

Unit-4
Machine Learning: Translation, Reinforcement LearningLayer
Jan-July, 2021-22 / Lecture No. 4

Dr. Deepak Gupta

Associate Professor, Computer Science & Engineering
1
 Outlines
• Prerequisite
• Objective
• Concept of translation, beam search and width
• Bleu score, attention model
• Reinforcement Learning
• Exercise
• Learning Outcome
• Student Effective Learning Outcome

CS 601 • Text & Reference Books 2

 Prerequisite

•Basics understanding of RNN

• Basics understanding of LSTM

3
CS 601
 Objective

• To acquire Basic understanding of translation, beam search

and width, Bleu score, attention model.
• To Understand the working of reinforcement learning.

4
CS 601
 Machine Translation
• Neural machine translation (NMT) is an approach to
machine translation that uses an artificial neural network
to predict the likelihood of a sequence of words, typically
modeling entire sentences in a single integrated model.
• In machine translation, encoder-decoder architecture as
shown above. RNN Encoder-Decoder model consists of
two recurrent neural networks.
• Encoder and decoder both use the same neural network
model but play a somewhat different role.
• The encoder is used to encode all the word embeddings
and extract context and long term dependencies which are
then passed over to decoder to generate output sentence.
• There are different types of natural language processing
models that can be used for this purpose.
• One RNN encodes a sequence of words into the fixed-
length vector by first passing it through an embedding
layer which is used to form an embedding matrix and then
it’s fed into a recurrent neural network.
 Attention model
• Attention enables our decoder to look at our encoded
source words while translating.
• Attention is most commonly used in sequence-to-
sequence models to attend to encoder states, but can also
be used in any sequence model to look back at past states.
• dot-product attention, which means the dot product of our
intermediate decoder output and our encoder output.
• We then take a weighted sum of our encoder vectors,
using this dot product as the weight.
Beam Search and width
 Beam Search and width
• It is popular heuristic that expands upon the greedy search
and returns a list of most likely output sequences.

• Instead of greedily choosing the most likely next step as

the sequence is constructed, the beam search expands all
possible next steps and keeps the k most likely, where k is
a user-specified parameter and controls the number of
beams or parallel searches through the sequence of
probabilities.
 Beam Search and width
• Common beam width values are 1 for a greedy search and
values of 5 or 10 for common benchmark problems in
machine translation.
• Larger beam widths result in better performance of a
model as the multiple candidate sequences increase the
likelihood of better matching a target sequence.
• This increased performance results in a decrease in
decoding speed.
Bleu score
Bleu score
Reinforcement Learning
• Reinforcement learning (RL) is an area of machine
learning concerned with how intelligent agents ought to
take actions in an environment in order to maximize the
notion of cumulative reward.
• Reinforcement learning is one of three basic machine
learning paradigms, alongside supervised learning and
unsupervised learning.
• Input: The input should be an initial state from which the
model will start
• Output: There are many possible output as there are
variety of solution to a particular problem
• Training: The training is based upon the input, The model
will return a state and the user will decide to reward or
punish the model based on its output.
• The model keeps continues to learn.
• The best solution is decided based on the maximum
reward.
 Various Practical applications of Reinforcement
Learning –
• RL can be used in robotics for industrial automation.
• RL can be used in machine learning and data processing
• RL can be used to create training systems that provide
custom instruction and materials according to the
requirement of students.
 Exercises
1. What is a neural machine translation?
2. How bleu scores improve the performance of the system?
3. How Reinforcement learning is different from supervised
learning.

24
CS 601
 Learning Outcomes

• Understand the basic concept of Translation

• Be able to understand Reinforcement learning?

25
CS 601
 Student Effective Learning Outcomes

1. Ability to solve problems through application of

theoretical & practical concept.

6. Ability to observe and develop sense making, logical

skills for abstract concepts.

12. Computational thinking ability (including to translate

vast data into concepts and to understand database
reasoning.

26
CS 601
 Text & Reference Books
Text Book:
1. Tom M. Mitchell, “Machine Learning”, McGraw Hill Education, First
edition, 2017
2. Ian Goodfellow and Yoshua Bengio and Aaron Courville, “Deep
Learning”, MIT Press, 2016
Reference Book:
3. Andreas Muller, "Introduction to Machine Learning with Python: A
Guide for Data Scientists", Shroff/O'Reilly; First edition (2016).
4. Aurelien Geon, “Hands-On Machine Learning with Scikit-Learn and
Tensorflow: Concepts, Tools, and Techniques to Build Intelligent
Systems”, Shroff/O'Reilly; First edition (2017).
27
CS 601
 28
CS 601