UNIT-I: Introduction to Neural Networks
Neural networks mimic the human brain's functioning to solve complex computational problems.
This unit introduces key components and concepts.
1. Artificial Neurons and Perceptron
An artificial neuron is the basic computational unit in a neural network.
The perceptron, introduced by Frank Rosenblatt, is a simple model for binary classification.
It consists of weights, bias, and an activation function.
Example: A perceptron can classify whether an email is spam or not based on inputs like subject
line and email content.
2. Computational Models of Neurons
Neurons perform weighted summation of inputs and apply activation functions like sigmoid or
ReLU to introduce non-linearity.
Example: Predicting house prices using inputs like area, location, and number of bedrooms.
3. Structure of Neural Networks
Neural networks consist of layers: input, hidden, and output layers.
Each neuron in one layer is connected to neurons in the subsequent layer.
4. Multilayer Feedforward Neural Networks (MLFFNN)
These networks pass information forward through multiple layers, allowing them to model complex
patterns.
Example: Image recognition in which raw pixel data is processed through layers to identify objects.
5. Back-propagation Learning
This algorithm minimizes the error by propagating it backward through the network to adjust
weights.
6. Empirical Risk Minimization and Bias-Variance Tradeoff
Empirical risk minimization aims to minimize errors on training data, while the bias-variance
tradeoff balances underfitting and overfitting.
7. Regularization
Techniques like L1, L2 regularization, and dropout help reduce overfitting.
8. Output Units and Hidden Units
- Output Units: Linear (for regression), softmax (for classification).
- Hidden Units: Activation functions like tanh and ReLU introduce non-linearity.
Example: ReLU activation can be used in hidden layers to model complex relationships.
UNIT-II: Deep Neural Networks
Deep Neural Networks (DNNs) have multiple hidden layers, allowing them to model more complex
data.
1. Difficulty in Training DNNs
Challenges include vanishing/exploding gradients and computational cost.
2. Greedy Layerwise Training
Training one layer at a time simplifies the process and helps initialize weights efficiently.
3. Optimization Methods
- AdaGrad: Adjusts learning rates for parameters based on their frequency.
- RMSProp: Modifies AdaGrad to perform better on non-stationary data.
- Adam: Combines the benefits of RMSProp and momentum.
Example: Adam optimizer is widely used in NLP tasks for faster convergence.
4. Regularization Techniques
- Dropout: Randomly disables neurons during training to reduce overfitting.
- Batch Normalization: Normalizes inputs to each layer to accelerate learning.
UNIT-III: Convolutional Neural Networks (CNNs)
CNNs are specialized for processing structured grid data like images.
1. Introduction to CNN
- Convolution: Extracts features by applying filters to input data.
- Pooling: Reduces dimensionality to prevent overfitting and improve computational efficiency.
2. Deep CNNs
Architectures like LeNet (for digit recognition) and AlexNet (for ImageNet classification)
revolutionized computer vision.
3. Training CNNs
Proper weight initialization, batch normalization, and optimization methods ensure better training.
Example: Using a pre-trained convnet like ResNet to classify new images with transfer learning.
UNIT-IV: Recurrent Neural Networks (RNNs)
RNNs are designed for sequence data like time series and text.
1. Sequence Modeling
RNNs maintain hidden states to capture temporal dependencies.
Example: Predicting stock prices based on historical data.
2. Backpropagation Through Time (BPTT)
Extends backpropagation for sequential data by unrolling the RNN over time.
3. Advanced Variants
- LSTM: Solves the vanishing gradient problem with gates controlling information flow.
- Bidirectional RNN/LSTM: Processes data in both directions to improve context understanding.
Example: Machine translation using Bidirectional LSTMs.
UNIT-V: Unsupervised Deep Learning
Focuses on extracting patterns from unlabeled data.
1. Autoencoders
Learn efficient representations of data by compressing and reconstructing input.
Example: Image denoising by reconstructing clear images from noisy ones.
2. Generative Adversarial Networks (GANs)
Consist of a generator and discriminator competing to create realistic synthetic data.
Example: Generating synthetic images of faces or artwork.
3. Applications
- Computer Vision: Object detection, image segmentation.
- Speech Recognition: Transcribing spoken words into text.
- Natural Language Processing: Sentiment analysis, language translation.
