NPTEL Online Certification Courses

Indian Institute of Technology Kharagpur

Deep Learning
Assignment- Week 12
TYPE OF QUESTION: MCQ/MSQ
Number of questions: 10 Total mark: 10 X 1 = 10
______________________________________________________________________________

QUESTION 1:
Which one of the following graphical models fully represents a Variational Auto-encoder (VAE)
realization?

Correct Answer: a

Detailed Explanation:
For practical realization of VAE, we have an encoder 𝑄(∙) which receives an input signal, 𝑥 and
generates a latent code, 𝑧. This part of the network can be denoted by 𝑄(𝑧|𝑥) and directed from
𝑥 to 𝑧. Next, we have a decoder section which takes the encoded z vector to reconstruct the input
signal, 𝑥. This part of the network is represented by 𝑃(𝑥|𝑧) and should be directed from 𝑧 to 𝑥.
______________________________________________________________________________

QUESTION 2:
Which one of the following computational graphs correctly depict the re-parameterization trick
deployed for practical Variational Auto-encoder (VAE) implementation? Circular nodes
represent random nodes in the models and the quadrilateral nodes represent deterministic
nodes.
 NPTEL Online Certification Courses
Indian Institute of Technology Kharagpur

a b c d

Correct Answer: a

Detailed Solution:
With the re-parameterization trick, the only random component in the network is the node of ∊
which is sampled from 𝑁(0, 𝐼). The other nodes of μ and σ are deterministic. Since ∊ is sampled
from outside the computational graph, the overall z vector also becomes deterministic
component for a given set of μ, σ and ∊. Also, if z is not deterministic, we cannot back propagate
gradients through it. Also, in the computation graph, the forward arrows will emerge from μ,σ,ϵ
towards z for computing the z vector.
______________________________________________________________________________

QUESTION 3:
When the GAN game has converged to its Nash equilibrium (when the Discriminator randomly
makes an error in distinguishing fake samples from real samples), what is the probability (of
belongingness to real class) given by the Discriminator to a fake generated sample?

a. 1
b. 0.5
c. 0
d. 0.25

Correct Answer: b

Detailed Solution:
 NPTEL Online Certification Courses
Indian Institute of Technology Kharagpur

Nash equilibrium is reached when the generated distribution, 𝑝𝑔 (𝑥) equals the original data
distribution, 𝑝𝑑𝑎𝑡𝑎 (𝑥), which leads to 𝐷(𝑥) = 0.5 for all 𝑥.
______________________________________________________________________________

QUESTION 4:
Why is re-parameterization trick used in VAE?

a. Without re-parameterization, the mean vector of latent code of VAE encoder

with tend towards zero
b. Sampling from a VAE encoder latent space is non-differentiable and thus we
cannot back propagate gradient during optimization using gradient descent
c. We need to re-parameterize Normal distribution over latent space to Bernoulli
distribution
d. None of the above

Correct Answer: b

Detailed Solution:
We cannot sample in a differentiable manner from within a computational graph present in a
neural network. Re-parameterization enables the sampling function to be present outside the
main computational graph which enables us to do regular gradient descent optimization.
______________________________________________________________________________
 NPTEL Online Certification Courses
Indian Institute of Technology Kharagpur

QUESTION 5:
Which of the following losses can be used to optimize for generator’s objective (while training a
Generative Adversarial network) by MINIMIZING with gradient descent optimizer? Consider
cross-entropy loss,

CE(a, b) = - [ a*log(b) + (1-a)*log(1-b)]

and D(G(z)) = probability of belonging to real class as output by the Discriminator for a given
generated sample G(z).

a. CE(1, D(G(z)))
b. CE(1, -D(G(z)))
c. CE(1, 1 - D(G(z)))
d. CE(1, 1 / D(G(z)))

Correct Answer: a

Detailed Solution:
Except for option (a) none of the other objective function are minimized at D(G(z)) = 1 which is
the goal of the Generator, i.e. to force the Discriminator to output probability=1 for a generated
sample. Loss function in option (a) is the only choice which keeps on decreasing as D(G(z))
increases. Also, it is required that D(G(z)) ∈ [0,1].
____________________________________________________________________________

QUESTION 6:
What will happen if we do not enforce KL divergence loss in VAE latent code space?

a. The latent code distribution will be mimic zero mean and unit variance Normal
distribution
b. Network will learn to form distinctive clusters with high standard deviation for
each cluster
c. Network will learn to form distinctive clusters with low standard deviation for
each cluster
d. None of the above

Correct Answer: c
Detailed Solution:

With zero KL loss, the encoder part of VAE will try to form separated clusters (by
increasing the distance of the mean vectors) and simultaneously reduce the standard
 NPTEL Online Certification Courses
Indian Institute of Technology Kharagpur

deviation of each cluster to reduce confusion for the decoder part of the network. This will
allow to efficiently reduce the reconstruction loss only present as the loss component in the
network. So, without the KL loss, the network will reduce to a simple autoencoder network.

____________________________________________________________________________

QUESTION 7:
Which of the following is an INVALID activation function inside a neural network?

a. 𝑓(𝑥) = 𝑚𝑎𝑥(0, 2𝑥)

b. 𝑓(𝑥) = 𝑚𝑖𝑛(0, 2𝑥)
c. 𝑓(𝑥) = 𝑡𝑎𝑛ℎ(𝑥)
d. None of the above

Correct Answer: d

Detailed Solution:

Options (a), (b) and (c) are the valid activation functions.

______________________________________________________________________________

QUESTION 8:
Which one of the following statements is True about Variational Autoencoder (VAE)?

a. VAE can only be applied on monochrome images

b. VAE reconstructions tend to be blurry
c. VAE reconstructions always have high frequency preserving details
d. VAE latent space is designed to be NOT smooth

Correct Answer: b

Detailed Solution:

All other options are False. One of the primary reasons for the blurry reconstructions by VAE is
the explicit reconstruction loss component present in the loss function.

____________________________________________________________________________
 NPTEL Online Certification Courses
Indian Institute of Technology Kharagpur

QUESTION 9:
Figure shows latent vector addition of two concepts of “man without a hat” and “hat”. What is
expected from the resultant vector?

a. Hat without man

b. Man with hat
c. Woman with hat
d. Woman without hat

Correct Answer: b
Detailed Solution:

It is expected that VAE latent space follows semantic vector arithmetic. Thus the resultant
vector is a vector addition of the two semantic concepts which will result in the final vector
to represent a MAN WITH HAT.

______________________________________________________________________________

QUESTION 10:
Which one of the following statements is preferable situation for GAN training?

a. Discriminator loss should monotonically increase

b. Generator loss should monotonically increase
c. Discriminator loss should monotonically go down while the Discriminator loss
should monotonically go up
d. Neither of the Discriminator or Generator monotonically loss goes up or down
monotonically

Correct Answer: d

Detailed Solution:
 NPTEL Online Certification Courses
Indian Institute of Technology Kharagpur

Since GAN game is played under a zero-sum non-cooperative game; if one of the wins, the
opponent losses and the Nash equilibrium is not reached.

____________________________________________________________________________

______________________________________________________________________________

************END*******