ADDITIONAL MATERIAL FOR MICRO-LECTURE TOPICS

1.​ NN Architecture III

List 1:
https://en.wikipedia.org/wiki/Multilayer_perceptron
https://missinglink.ai/guides/neural-network-concepts/perceptrons-and-multi-layer-perceptrons-t
he-artificial-neuron-at-the-core-of-deep-learning/

List 2:
https://www.youtube.com/watch?v=u5GAVdLQyIg

2.​NN Learning II

List 1:
https://en.wikipedia.org/wiki/Backpropagation
https://towardsdatascience.com/how-does-back-propagation-in-artificial-neural-networks-work-c
7cad873ea7

List 2:
http://neuralnetworksanddeeplearning.com/chap2.html
https://www.youtube.com/watch?v=Ilg3gGewQ5U
https://www.youtube.com/watch?v=tIeHLnjs5U8

3.​Introduction to PyTorch

List 1:
https://heartbeat.fritz.ai/introduction-to-pytorch-for-deep-learning-5b437cea90ac
https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html
https://cs230.stanford.edu/blog/pytorch/
https://pytorch.org/tutorials/
https://pytorch.org/tutorials/beginner/nlp/pytorch_tutorial.html

List2:
https://pytorch.org/tutorials/​ (Contains both beginner and more advanced tutorials)

4. ​Neural Networks Basics

List 1:
http://jalammar.github.io/visual-interactive-guide-basics-neural-networks/
https://towardsdatascience.com/neural-networks-basics-29cc093b82be
https://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html

List 2:
https://towardsdatascience.com/introduction-to-math-behind-neural-networks-e8b60dbbdeba
https://www.youtube.com/watch?v=vT1JzLTH4G4&list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8s
Yv

5 .​Finetuning

List 1:
https://pytorch.org/tutorials/beginner/data_loading_tutorial.html
https://pytorch.org/docs/stable/torchvision/models.html
https://towardsdatascience.com/how-to-fine-tune-your-neural-network-for-your-data-image-class
ification-d0f01c92300b

List 2:
https://stats.stackexchange.com/questions/255364/fine-tuning-vs-joint-training-vs-feature-extrac
tion

6.​Overfitting

List​ ​1:
https://www.cs.toronto.edu/~lczhang/360/lec/w05/overfit.html
https://atcold.github.io/pytorch-Deep-Learning/en/week14/14-3/
https://machinelearningmastery.com/early-stopping-to-avoid-overtraining-neural-network-models
/
https://www.geeksforgeeks.org/dropout-in-neural-networks/
https://towardsdatascience.com/batch-normalization-and-dropout-in-neural-networks-explained-
with-pytorch-47d7a8459bcd

List 2:
https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf

7.NN Architecture V

List1:
Forward and Backward Convolution Passes as Matrix Multiplication (danieltakeshi.github.io)

List2:
Intuitively Understanding Convolutions for Deep Learning | by Irhum Shafkat | Towards Data
Science

8.NN Learning IV

List1​:
Gradient Descent With Momentum (C2W2L06) - YouTube
Deep Learning — All Optimizers Explained | by Vishal Mishra | The Startup | Nov, 2020 |
Medium
Stochastic Gradient Descent with momentum | by Vitaly Bushaev | Towards Data Science
Gradient Descent With Momentum. The problem with vanilla gradient… | by Rauf Bhat | Oct,
2020 | Towards Data Science

List2:
https://youtu.be/wrEcHhoJxjM

9. NN ARCHITECTURE IV

List 1:
https://towardsdatascience.com/activation-functions-neural-networks-1cbd9f8d91d6

List 2:
https://missinglink.ai/guides/neural-network-concepts/7-types-neural-network-activation-function
s-right/#:~:text=Activation%20functions%20are%20mathematical%20equations,relevant%20for
%20the%20model's%20prediction.

10. NN Learning III

List 1:
​http://tensorlab.cms.caltech.edu/users/anima/slides/nips2015-nonconvex.pdf​ (Introduction
slides.)
https://medium.com/learn-love-ai/the-curious-case-of-the-vanishing-exploding-gradient-bf58ec6
822eb

List 2: ​https://www.cs.cornell.edu/courses/cs6787/2017fa/Lecture7.pdf​ ​(First 13 slides)

https://towardsdatascience.com/the-vanishing-exploding-gradient-problem-in-deep-neural-netwo
rks-191358470c11

11. BSL I

List 1:
https://towardsdatascience.com/supervised-vs-unsupervised-learning-14f68e32ea8d

List 2:
https://towardsdatascience.com/the-blurry-lines-of-supervised-and-unsupervised-learning-b8a2a
a04c8b0
https://machinelearningmastery.com/types-of-learning-in-machine-learning/

12. BSL II

List 1:
https://sites.google.com/site/dataclusteringalgorithms/mst-based-clustering-algorithm
https://www.cs.cmu.edu/~ckingsf/class/02713/lectures/lec05-mstclust.pdf

List 2:
https://static.aminer.org/pdf/PDF/000/219/731/computing_hierarchies_of_clusters_from_the_eu
clidean_minimum_spanning_tree.pdf

13. BSL III

List 1:
https://en.wikipedia.org/wiki/K-means_clustering
https://stanford.edu/~cpiech/cs221/handouts/kmeans.html

List 2:
https://towardsdatascience.com/k-means-clustering-algorithm-applications-evaluation-methods-
and-drawbacks-aa03e644b48a

14. BSL IV

List 1:
https://medium.com/inside-machine-learning/placeholder-3557ebb3d470

List 2:
http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf

15. ​NN Architecture VI

List1:
https://www.jeremyjordan.me/autoencoders/
https://www.tensorflow.org/tutorials/generative/autoencoder

List2:
https://blog.keras.io/building-autoencoders-in-keras.html
http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/

16. NN Architecture VII

List1:
https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks

List2:
https://www.tensorflow.org/tutorials/text/text_generation
https://colah.github.io/posts/2015-08-Understanding-LSTMs/