Enrollment No : 2020103103510280

Practical:9
Aim: Design and implement a neural network with Pima Indian
diabetes dataset.
A neural network is a computational model inspired by the structure and functioning
of the human brain. It consists of interconnected nodes, or artificial neurons,
organized into layers. Neural networks are used for various machine learning tasks,
such as pattern recognition, classification, regression, and decision-making.

Basic Components of a Neural Network:

1. Neuron (Node):
• The basic building block of a neural network.
• Represents a unit of computation that takes input, processes it.
• Artificial neurons mimic the behavior of biological neurons, but in a
simplified form.

2. Layer:
• A collection of neurons arranged together in a specific way.
• Three main types of layers:
• Input Layer: Receives input features and passes them to the next layer.
• Hidden Layer(s): Layers between the input and output layers where
computations and transformations occur.
• Output Layer: Produces the final output or prediction.

3. Connection:
• Links between neurons in different layers.
• Each connection has an associated weight, which determines the strength
of the connection.
• Weights are adjusted during training to optimize the network's
performance.

4. Activation Function:
• A mathematical operation applied to the weighted sum of inputs
at each neuron.
• Introduces non-linearity to the model, allowing it to learn
complex patterns.
• Common activation functions include sigmoid, hyperbolic tangent (tanh), and
rectified linear unit (ReLU).

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

5. Bias:

• An additional term added to the weighted sum before applying the activation
function. Allows the model to capture patterns even when all input features are
zero.

Different Processes in a Neural Network:

1. Feedforward:
• The process of passing input data through the network to generate an output.
• Information travels in one direction, from the input layer through the hidden layers to the
output layer.

2. Weights and Bias Adjustment (Training):

• During training, the network learns by adjusting the weights and biases based on
the error between predicted and actual output.
• Backpropagation algorithm is commonly used for this purpose.

3. Loss Function:
• Measures the difference between the predicted output
anactual target.
• The goal during training is to minimize the loss.

Types of Neural Networks:

1. Feedforward Neural Network (FNN):
• Standard neural network where information flows in one direction.
• Commonly used for classification and regression tasks.

2. Recurrent Neural Network (RNN):

• Allows information to be passed across layers and used in
subsequent iterations.
• Suitable for sequence data, such as time series or natural
language.

3. Convolutional Neural Network (CNN):

• Specifically designed for processing grid-like data, such as images.
• Uses convolutional layers to learn spatial hierarchies of features.

4. Generative Adversarial Network (GAN):

• Consists of a generator and a discriminator network that are
trained adversarially.
• Used for generating synthetic data, such as images.

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

5. Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU):

• Specialized RNN architectures designed to capture long-term dependencies
in sequence data.
• Mitigate the vanishing gradient problem.

6. Autoencoder:
• Consists of an encoder and a decoder.
• Used for unsupervised learning and dimensionality reduction.
Here, we are implementing neural network with the Pima Indian diabetes
dataset using Python.

Step 1: Import necessary libraries

import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from keras.models import Sequential

from keras.layers import Dense

This code block imports necessary libraries for building and training a neural
network. It uses the pandas library to handle data in a DataFrame, matplotlib for
plotting, train_test_split from scikit-learn to split the data into training and testing
sets, StandardScaler for feature scaling, and Sequential and Dense classes from
Keras to initialize and define the architecture of a feedforward neural network,
respectively. The neural network will be used for a machine learning task, and the
subsequent code will involve loading and preprocessing data, building the network,
training it, and evaluating its performance.

Step 2: Load and Preprocess the Data

url = "https://raw.githubusercontent.com/npradaschnor/Pima-Indians
Diabetes-Dataset/master/diabetes.csv"
diabetes_df = pd.read_csv(url)
# Separate features (X) and target (y)
X = diabetes_df.drop(columns=['Outcome'])
y = diabetes_df['Outcome']
# Standardize the features
scaler = StandardScaler()
X_std = scaler.fit_transform(X)

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

This code block loads the Pima Indians Diabetes dataset from a given URL using
pandas. It then separates the features (X) and the target variable (y). The features are
stored in the DataFrame X by dropping the 'Outcome' column, and the target
variable is stored in the Series y. Subsequently, the features are standardized using
StandardScaler, a preprocessing technique that scales the data to have a mean of 0
and a standard deviation of 1, ensuring that each feature contributes equally to the
model. The standardized features are stored in the variable X_std.

Step 3: Initialize Neural Network Model

# Initialize the model
model = Sequential()
# Add input layer (8 features) and hidden layers
model.add(Dense(64, input_dim=8, activation='relu'))
model.add(Dense(32, activation='relu'))
# Add output layer (1 neuron, sigmoid activation for binary
classification)
model.add(Dense(1, activation='sigmoid'))

This code block initializes a feedforward neural network model using the Keras
library. The model is Sequential, indicating a linear stack of layers. The input layer
consists of 8 neurons, corresponding to the number of features in the dataset. Two
hidden layers follow with 64 and 32 neurons, respectively, both utilizing the
rectified linear unit (ReLU) activation function. ReLU is a commonly used
activation function that introduces non-linearity to the model. The output layer
contains a single neuron with a sigmoid activation function, suitable for binary
classification problems like the Pima Indians Diabetes dataset, where the goal is
to predict the presence or absence of diabetes.

Step 4: Compile the Model

# Compile the model
model.compile(optimizer='adam', loss='binary_crossentropy',
metrics=['accuracy'])

In this code block, the neural network model is compiled, configuring the essential
settings for training. The Adam optimizer is chosen for efficient gradient descent,
and binary crossentropy is used as the loss function, suitable for binary classification
tasks. The chosen metric for evaluation during training is accuracy, which measures
the proportion of correctly predicted instances. The compilation prepares the model
for the training phase, where it will learn to minimize the specified loss function and
improve its predictive performance.

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

Step 5: Train the Model

# Train the model and store the history
history = model.fit(X_std, y, epochs=10, batch_size=32, validation_split=0.2)

Output:

In this code block, the compiled neural network model is trained on the standardized
feature data (X_std) and target labels (y). The training is performed for 50 epochs
with a batch size of 32, meaning the model iterates through the entire dataset 50
times, updating its parameters after processing each batch of 32 samples. The
validation_split=0.2 parameter indicates that 20% of the training data will be used as
a validation set to monitor the model's performance during training. The training
history, including information about accuracy and loss over epochs, is stored in the
history variable for later analysis and visualization.

Step 6: Print the Accuracy

# Evaluate the model on the entire dataset
test_loss, test_accuracy = model.evaluate(X_std, y)
print(f"Test Accuracy: {test_accuracy:.2f}")

Output:

In this code block, the trained neural network model is evaluated on the entire dataset
(both training and validation sets). The evaluate method computes the loss and
accuracy of the model based on the standardized features (X_std) and target labels
(y). The results, including the test accuracy, are printed to the console for assessment
of the model's performance on the entire dataset. The accuracy provides a measure of
how well the model generalizes to unseen data.

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

Step 7: Plot Training and Validation Accuracy and Loss

# Get training history from the model
accuracy_key = 'accuracy' if 'accuracy' in history.history.keys() else
'acc'
val_accuracy_key = 'val_accuracy' if 'val_accuracy' in
history.history.keys() else 'val_acc'
plt.figure(figsize=(12, 5))
# Plot training accuracy
plt.subplot(1, 2, 1)
plt.plot(history.history[accuracy_key], label='Training Accuracy')
plt.plot(history.history[val_accuracy_key], label='Validation Accuracy')
plt.title('Training and Validation Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
# Plot training loss
plt.subplot(1, 2, 2)
plt.plot(history.history['loss'], label='Training Loss')
plt.plot(history.history['val_loss'], label='Validation Loss')
plt.title('Training and Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.tight_layout()
plt.show()

Output:

This code block visualizes the training history of the neural network model. It checks
whether the accuracy and validation accuracy keys are present in the history
dictionary and selects the appropriate ones. Then, it plots two subplots side by side:

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

1. Training and Validation Accuracy: This subplot displays the training

accuracy and validation accuracy over epochs, helping to assess how well the
model is learning from the training data and generalizing to the validation set.

2. Training and Validation Loss: This subplot shows the training loss and
validation loss over epochs. Monitoring loss is crucial to ensure that the model is
minimizing errors during training and not overfitting or underfitting. The plots
provide insights into the training progress and help identify potential issues such as
overfitting or insufficient learning.

Practical:10

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

Aim: To implement multiclass classification with neural network

on Iris flower species dataset.

Multiclass classification with neural networks involves training a model to

categorize instances into three or more classes using a neural network architecture.
Neural networks, especially deep learning models, are well-suited for handling
complex relationships in high-dimensional data. Here is an explanation of key
aspects of multiclass classification with neural networks:

1. Neural Network Architecture:

• The architecture of the neural network is designed to handle multiple classes. It
includes an input layer, one or more hidden layers, and an output layer. The
number of neurons in the output layer matches the number of classes.

2. Activation Functions:
• The activation function in the output layer is typically a softmax activation.
Softmax converts the raw output of the model into probability scores, providing a
probability distribution over all classes.

3. Loss Function:
• The choice of the loss function is critical for training the model. Cross-
entropy loss (either categorical or sparse categorical, depending on label
encoding) is commonly used for multiclass classification. It measures the
difference between predicted and true class probabilities.

4. Label Encoding:
• Labels are often one-hot encoded, especially when using neural networks.
One-hot encoding represents each class as a binary vector, where each
element corresponds to a class.

5. Training Process:
• During training, the neural network adjusts its weights and biases to minimize the
chosen loss function. This is done through backpropagation and optimization
algorithms like stochastic gradient descent (SGD) or its variants.

6. Forward and Backward Pass:

• In each training iteration, the forward pass computes the model's output, and the
backward pass calculates gradients for each parameter. These gradients are then
used to update the model's weights.

7. Epochs and Batch Size:

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

• Training is conducted over multiple epochs, where each epoch represents one pass
through the entire training dataset. Batch size determines the number of instances
processed in each iteration.

8. Validation and Testing:

• The model's performance is assessed on validation data during training, and final
evaluation is done on a separate test set. Metrics such as accuracy, precision, recall,
and F1 score are used for evaluation.

9. Overfitting Prevention:
• Techniques like dropout and regularization are employed to prevent overfitting,
ensuring that the model generalizes well to new, unseen data.

10. Visualization and Interpretation:

• Tools like confusion matrices, classification reports, and learning curves are used to
visualize and interpret the model's performance. Multiclass classification with neural
networks is applied in various domains, including image classification, text
categorization, and speech recognition, where instances can belong to multiple
classes. The flexibility and expressive power of neural networks make them suitable
for handling complex relationships in diverse datasets.

Here, we are implementing multiclass classification with neural network on Iris

flower species dataset using Python.

Step 1: Import necessary libraries

# Import necessary libraries

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import load_iris
from keras.models import Sequential
from keras.layers import Dense
from keras.utils import to_categorical

This code block imports the required libraries for implementing a neural network-
based multiclass classification using the Iris dataset. The libraries include NumPy for
numerical operations, pandas for data manipulation, matplotlib for plotting, scikit-
learn for dataset loading and preprocessing, and Keras for building and training
neural networks. The dataset used is the Iris dataset, loaded from scikit- learn's
datasets module. The neural network model is constructed using the Sequential API
from Keras, consisting of an input layer, one hidden layer with rectified linear unit

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

(ReLU) activation, and an output layer with a softmax activation function. To

facilitate multiclass classification, the target labels are one-hot encoded using the
to_categorical function from Keras utils. This encoding transforms categorical labels
into binary vectors, allowing the neural network to predict probabilities across
multiple classes.

Step 2: Load and Preprocess the Data

# Load the Iris dataset

iris = load_iris()
X = iris.data
y = iris.target
# Preprocess the data
scaler = StandardScaler()
X_std = scaler.fit_transform(X)
# Convert labels to one-hot encoding
y_encoded = to_categorical(y)

In this code block, the Iris dataset is loaded using the load_iris function from the
sklearn.datasets module. The dataset contains features (X) representing sepal and
petal dimensions of iris flowers and labels (y) indicating the species of each iris. The
features are then preprocessed by standardizing them using StandardScaler from
sklearn.preprocessing. Standardization ensures that the features have zero mean and
unit variance, making them suitable for neural network training. Additionally, the
labels (species) are converted into one-hot encoding using the to_categorical function
from keras.utils. One hot encoding transforms the categorical labels into binary
vectors, where each element corresponds to a unique class. This encoding is
necessary for multiclass classification with neural networks, enabling the model to
predict probabilities for each class independently. The resulting X_std contains the
standardized features, and y_encoded holds the one-hot encoded labels, preparing the
data for training a neural network.

Step 3: Split Data

# Split the data into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X_std, y_encoded,
test_size=0.2, random_state=42)

In this code block, the dataset is split into training and testing sets using the
train_test_split function from scikit-learn. The standardized features (X_std) and

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

one-hot encoded labels (y_encoded) are divided into two sets: X_train and y_train
for training, and X_test and y_test for testing. The parameter test_size=0.2
indicates that 20% of the data will be reserved for testing, and random_state=42
ensures reproducibility by fixing the random seed for the split. This separation is
crucial for training the neural network on one subset and evaluating its performance
on an independent subset to assess generalization to new, unseen data.

Step 4: Build Neural Network Model

# Build the neural network model
model = Sequential()
model.add(Dense(8, input_dim=4, activation='relu'))
model.add(Dense(3, activation='softmax')) # Output layer with softmax
activation for multiclass classification

In this code block, a neural network model for multiclass classification is constructed
using the Keras Sequential API. The model consists of two layers. The first layer is a
densely connected (fully connected) layer with 8 neurons and a ReLU activation
function. It takes input with a dimension of 4, representing the features of the input
data. The second layer is the output layer with 3 neurons, corresponding to the three
classes in the Iris dataset. The activation function used in the output layer is softmax,
which converts the raw model outputs into probability scores, providing a probability
distribution over the three classes. This architecture is well-suited for the multiclass
classification task, where the goal is to classify Iris flowers into three species based
on their features.

Step 5: Compile the Model

# Compile the model

model.compile(optimizer='adam', loss='categorical_crossentropy',
metrics=['accuracy'])

In this code block, the neural network model is compiled before training. The
compile function configures the model for training by specifying the optimizer, loss
function, and evaluation metric. In this case, the 'adam' optimizer, which is a popular
choice for its adaptive learning rates, is utilized. The loss function is set to
'categorical_crossentropy,' suitable for multiclass classification tasks with one-hot
encoded labels. The metric to be monitored during training is 'accuracy,' providing a
measure of the model's performance. This step prepares the model for the training
process, allowing it to be optimized for the specified task.

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

Step 6: Train the Model

# Train the model

history = model.fit(X_train, y_train, epochs=50, batch_size=32,
validation_split=0.2, verbose=0)
print(f"Test Accuracy: {test_accuracy:.2f}")

This code block trains the neural network model using the training data (X_train and
y_train) with 50 epochs, a batch size of 32, and a validation split of 20%. The fit
method adjusts the model's weights based on the training data, and the training
process is monitored by the history variable, which stores metrics such as training
and validation accuracy and loss over each epoch. The verbose=0 parameter
suppresses the progress output during training. After training, the model's
performance is evaluated on the test set, and the test accuracy is printed.

Step 7: Print the Accuracy

# Evaluate the model on the test set

test_loss, test_accuracy = model.evaluate(X_test, y_test)
print(f"Test Accuracy: {test_accuracy:.2f}")

In this code block, the trained neural network model is evaluated on the previously
unseen test set (X_test). The evaluate method computes the loss and accuracy of the
model's predictions on the test data. The resulting values, namely test_loss and
test_accuracy, are then printed to the console. The test accuracy provides an
assessment of how well the neural network generalizes to new, unseen instances,
giving an indication of its performance on real-world data beyond the training set.

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

Step 8: Plot Training and Validation Metrics

# Plot training and validation accuracy and loss
accuracy_key = 'accuracy' if 'accuracy' in history.history.keys() else 'acc'
val_accuracy_key = 'val_accuracy' if 'val_accuracy' in
history.history.keys() else 'val_acc'
plt.figure(figsize=(12, 5))
# Plot training accuracy
plt.subplot(1, 2, 1)
plt.plot(history.history[accuracy_key], label='Training Accuracy')
plt.plot(history.history[val_accuracy_key], label='Validation Accuracy')
plt.title('Training and Validation Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
# Plot training loss
plt.subplot(1, 2, 2)
plt.plot(history.history['loss'], label='Training Loss')
plt.plot(history.history['val_loss'], label='Validation Loss')
plt.title('Training and Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.tight_layout()
plt.show()

Output:

This code block is responsible for plotting the training and validation accuracy as well as the
training and validation loss over the training epochs. Let's break down the code step by step:

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

1. Key Selection:
• The variables accuracy_key and val_accuracy_key are assigned the appropriate
keys for training accuracy and validation accuracy based on whether 'accuracy' or
'acc' is present in the history's keys.

2. Plot Initialization:
• A figure with a size of (12, 5) is initialized for the plots. The figure is divided
into two subplots arranged in a 1x2 grid.

3. Training Accuracy Plot (Left Subplot):

• The left subplot (plt.subplot(1, 2, 1)) is designated for plotting training accuracy.
The training accuracy (history.history[accuracy_key]) and validation accuracy
(history.history[val_accuracy_key]) curves are plotted against the number of epochs.
• The plot is labeled with "Training and Validation Accuracy," and the axes are
labeled as "Epochs" (x-axis) and "Accuracy" (y-axis).
• A legend is added to distinguish between the training and validation
accuracy curves.

4. Training Loss Plot (Right Subplot):

• The right subplot (plt.subplot(1, 2, 2)) is reserved for plotting training loss. The
training loss (history.history['loss']) and validation loss (history.history['val_loss'])
curves are plotted against the number of epochs.
• The plot is labeled with "Training and Validation Loss," and the axes are labeled
as "Epochs" (x - axis) and "Loss" (y-axis).
• A legend is added to distinguish between the training and
validation loss curves.

5. Layout Adjustment and Display:

• plt.tight_layout() ensures that subplots do not overlap, improving visualization.
• Finally, plt.show() displays the figure with both subplots. This code block provides
a visual representation of the training and validation performance of the neural
network over multiple epochs. Monitoring these curves helps in assessing the
model's learning behavior, identifying overfitting, and making informed decisions
about model training and architecture adjustments.

Practical:11

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

Aim: To implement convolutional neural network (CNN) for

object recognition in an image with CIFAR 10 dataset.

Step 1: Import necessary libraries

import tensorflow as tf
from tensorflow.keras import datasets, layers, models
import matplotlib.pyplot as plt

This code block imports TensorFlow and necessary modules from TensorFlow's
Keras API, which is a high-level neural networks API running on top of TensorFlow.
It also imports the datasets module, which provides access to commonly used
datasets for training machine learning models. Additionally, it imports the layers and
models modules, which contain functions and classes to define and build neural
network architectures. Finally, it imports the matplotlib.pyplot module for
visualization purposes.

Step 2: Load and Preprocess the CIFAR-10 Dataset

(train_images, train_labels), (test_images, test_labels) =

datasets.cifar10.load_data() train_images, test_images =
train_images / 255.0, test_images / 255.0\

This code block loads the CIFAR-10 dataset using TensorFlow's datasets module,
which provides a convenient interface to access preprocessed versions of the dataset.
The dataset consists of images divided into training and test sets, along with
corresponding labels indicating the class of each image. After loading the dataset,
the pixel values of the images are normalized to be between 0 and 1 by dividing
each pixel value by 255.0. This normalization step ensures that the input data is
within a consistent range, which can improve the training process and convergence
of the neural network model.

Step 3: Verify and Visualize the Data

class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer','dog', 'frog',

'horse', 'ship', 'truck'] plt.figure(figsize=(10,10))
for i in range(25):
plt.subplot(5,5,i+1)
plt.xticks([])
plt.yticks([])
plt.grid(False)
plt.imshow(train_images[i])

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

plt.xlabel(class_names[train_labels[i][0]])
plt.show()

Output:

This code block visualizes a subset of the CIFAR-10 dataset to verify the data and ensure it
was loaded correctly. It creates a grid of 25 subplots using Matplotlib, with each subplot
displaying an image from the training set along with its corresponding class name. The
class_names list contains the names of the 10 classes in the dataset, such as 'airplane',
'automobile', 'bird', etc. The loop iterates over the first 25 images in the training set, plots
each image using imshow, and assigns the corresponding class name as the xlabel for better
interpretation of the displayed images.

Step 4: Build the Convolutional Neural Network (CNN) Model

model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu',
input_shape=(32,32, 3))) model.add(layers.MaxPooling2D((2,
2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))

This code block builds a Convolutional Neural Network (CNN) model using the
Keras Sequential API. The model consists of a sequence of convolutional and
pooling layers, which are fundamental building blocks of CNNs. The first layer is a
2D convolutional layer with 32 filters of size 3x3, using the ReLU activation
function, and it expects input images of size 32x32 pixels with 3 color channels
(RGB). After each convolutional layer, a max-pooling layer is added to reduce the
spatial dimensions of the feature maps. This process helps in extracting and learning

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

hierarchical features from the input images, making the model capable of
recognizing patterns and structures in the data.

Step 5: Add Dense Layers

model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(10))

In this code block, dense layers are added to the convolutional neural network (CNN)
model. The Flatten() layer transforms the 2D feature maps produced by the
convolutional layers into a 1D array, which can be fed into the subsequent dense
layers. The first dense layer consists of 64 neurons with the ReLU activation
function, allowing the network to learn complex patterns from the flattened feature
vectors.The final dense layer has 10 neurons, corresponding to the 10 classes in the
CIFAR- 10 dataset, without an explicit activation function. This layer represents the
output layer of the model, and its activations are interpreted as class scores.

Step 6: Compile the Model

model.compile(optimizer='adam',
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logi
ts=True),
metrics=['accuracy'])

In this code block, the model is compiled with specific configurations necessary for
training. The 'adam' optimizer is chosen for optimizing the model parameters during
training, which is a popular choice due to its efficiency and adaptiveness. The loss
function used is the Sparse Categorical Crossentropy, which is suitable for multi-
class classification problems like CIFAR-10. Setting 'from_logits' to True indicates
that the model's output is not normalized with softmax activation, and thus the cross-
entropy loss function should handle logits directly. Additionally, the 'accuracy' metric
is specified to monitor the performance of the model during training and evaluation.
Overall, this step prepares the model for training by defining how it should optimize
its parameters and evaluate its performance.

Step 7: Train the Model

history = model.fit(train_images, train_labels, epochs=10,

validation_data=(test_images, test_labels))

Output:
UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

This code block trains the CNN model using the training data (train_images and
train_labels) for a specified number of epochs (in this case, 10). During training, the
model adjusts its parameters to minimize the loss function, which measures the
difference between the predicted and actual labels. Additionally, the validation data
(test_images and test_labels) are provided to evaluate the model's performance on
unseen data after each epoch. The fit method returns a history object containing
information about the training process, such as the loss and accuracy metrics
recorded during each epoch. This information can be used for analyzing the model's
performance and diagnosing potential issues like overfitting or underfitting.

Step 8: Evaluate the Model

plt.plot(history.history['accuracy'], label='accuracy')
plt.plot(history.history['val_accuracy'], label = 'val_accuracy')
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.ylim([0.5, 1])
plt.legend(loc='lower right')
plt.show()
test_loss, test_acc = model.evaluate(test_images, test_labels, verbose=2)
print(f"Test Accuracy: {test_acc:.2f}

Output:

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

This code block is responsible for evaluating the performance of the trained
convolutional neural network (CNN) model. Firstly, it plots the training and
validation accuracy over epochs using the Matplotlib library, which helps visualize
how the model's accuracy changes during training and whether it is overfitting or
underfitting. After plotting, the model is evaluated on the test dataset using the
evaluate method, which computes the loss and accuracy of the model on the test data.
The test accuracy is then printed to the console, providing a quantitative measure of
the model's performance on unseen data. This step is crucial for assessing how well
the model generalizes to new, unseen images and helps determine its real-world
applicability.

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

Practical:12
Aim: To implement sequence classification of movie reviews with LSTM
network.
Step 1: Import necessary libraries

import tensorflow as tf
from tensorflow.keras.datasets import imdb
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, LSTM, Embedding
from tensorflow.keras.preprocessing import sequence

This code block imports necessary libraries and modules from TensorFlow and Keras for
building a sentiment analysis model using LSTM networks on the IMDB movie review
dataset. It imports TensorFlow as tf, IMDB dataset from tensorflow.keras.datasets,
Sequential model and various layers (Dense, LSTM, Embedding) from
tensorflow.keras.models and tensorflow.keras.layers respectively, and preprocessing utilities
from tensorflow.keras.preprocessing. These imports provide functionalities for loading the
dataset, constructing the neural network model, and preprocessing the text data by padding
sequences to a uniform length.

Step 2: Set a random seed for reproducibility

tf.random.set_seed(7

In this code block, tf.random.set_seed(7) sets the random seed for TensorFlow's random
number generator to 7. Setting a random seed ensures that the sequence of random numbers
generated by TensorFlow operations, such as weight initialization and dropout, remains
consistent across runs. This is crucial for reproducibility, as it ensures that the model's
behavior is deterministic and can be replicated across different executions. By fixing the
random seed, you can obtain consistent results when training and evaluating your model,
which is particularly important for experimentation and debugging purposes.

Step 3: Load the IMDB dataset and limit the number of words to the
top 5000 top_words = 5000
(X_train, y_train), (X_test, y_test) = imdb.load_data(num_words=top_words)

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

In this code block, the IMDB movie review dataset is loaded using the imdb.load_data()
function from the Keras dataset module. By setting num_words=top_words as a parameter,
the dataset is limited to include only the top 5000 most frequently occurring words in the
reviews. This restriction helps manage the vocabulary size and reduces computational
complexity. The loaded dataset is split into training and testing sets, denoted as (X_train,
y_train) and (X_test, y_test) respectively, where X_train and X_test contain the movie
reviews encoded as sequences of word indices, and y_train and y_test contain
corresponding binary sentiment labels (0 for negative and 1 for positive). This step prepares
the dataset for subsequent preprocessing and model training.

Step 4: Preprocess the data: truncate sequences to a fixed length (500

words)

max_review_length = 500
X_train = sequence.pad_sequences(X_train, maxlen=max_review_length)
X_test = sequence.pad_sequences(X_test, maxlen=max_review_length)

In this code block, the data preprocessing step involves truncating and padding sequences to
ensure a fixed length of 500 words for each movie review. First, the variable
max_review_length is set to 500, specifying the desired length for the sequences. Then, the
sequence.pad_sequences() function from Keras preprocessing module is used to truncate or
pad each sequence of movie reviews to match this fixed length. X_train and X_test are
updated with the truncated or padded sequences, ensuring uniformity in the length of input
data, which is necessary for feeding it into the neural network model for training and
evaluation. This step prepares the data for further processing and model fitting.

Step 5: Define the LSTM model architecture

embedding_vector_length = 32
model = Sequential()
model.add(Embedding(top_words, embedding_vector_length,
input_length=max_review_length))
model.add(LSTM(100))
model.add(Dense(1, activation='sigmoid'))

In this code block, the architecture of the LSTM model is defined using the Keras Sequential
API. The embedding layer is added as the first layer, with top_words as the input dimension,
embedding_vector_length as the output dimension, and max_review_length as the input
length. This layer converts the input sequences of word indices into dense vectors of fixed

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

size, which are learned during training. Next, an LSTM layer with 100 units is added, which
processes the input sequences and captures their sequential dependencies. Finally, a dense
layer with a single neuron and sigmoid activation function is added to produce the output,
representing the predicted sentiment score (0 for negative and 1 for positive). This
architecture defines the flow of information through the model, from input to output, and
prepares it for compilation and training.

Step 6: Compile the model

model.compile(loss='binary_crossentropy', optimizer='adam',
metrics=['accuracy'])

In this code block, the compiled model is configured with specific settings for training. The
compile() function from the Keras API is used to specify the loss function, optimizer, and
evaluation metrics. In this case, the binary cross-entropy loss function is chosen since it is
suitable for binary classification tasks like sentiment analysis. The Adam optimizer is
selected for its efficiency and effectiveness in updating model weights during training.
Additionally, the accuracy metric is specified to monitor the performance of the model
during training and evaluation. By compiling the model with these settings,

it is prepared for training on the training data, where it will iteratively adjust its
parameters to minimize the defined loss function and improve its predictive
performance.

Step 7: Print the model summary

print(model.summary())

Output:

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

In this code block, the summary() method of the model object is called to print a summary of
the model architecture. This summary provides a concise overview of the model's layers,
their output shapes, and the number of parameters in each layer. The summary includes
information such as the layer type (e.g., Embedding, LSTM, Dense), the output shape of
each layer (excluding the batch size), the number of trainable parameters in each layer, and
the total number of trainable parameters in the model. Printing the model summary allows
for easy inspection of the model's structure and parameter count, aiding in understanding its
complexity and potential for overfitting or underfitting. It is a helpful diagnostic tool for
ensuring that the model architecture matches the intended design and for identifying any
issues before training begins.

Step 8: Train the model on the training data

model.fit(X_train, y_train, epochs=3, batch_size=64)

Output:

In this code block, the fit() method of the model object is called to train the model on the
training data. The X_train and y_train variables represent the input features
(padded/truncated sequences of movie reviews) and the corresponding target labels
(sentiment scores) respectively. The epochs

parameter is set to 3, indicating the number of times the entire training dataset will be passed
through the model for training. The batch_size parameter specifies the number of samples
to be used in each training iteration. During training, the model updates its parameters
(weights and biases) using optimization techniques like stochastic gradient descent (SGD)
or its variants, such as Adam, to minimize the defined loss function (binary cross-entropy in
this case) and improve its accuracy in predicting sentiment. This step involves iteratively
adjusting the model's parameters based on the training data, allowing it to learn the
underlying patterns and relationships in the data.

Step 9: Evaluate the model on the test data and print the accuracy

scores = model.evaluate(X_test, y_test, verbose=0)

print("Accuracy: %.2f%%" % (scores[1] * 100))

Output:

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)
Enrollment No : 2020103103510280

In this code block, the evaluate() method of the model object is used to evaluate the trained
model's performance on the test data. The X_test and y_test variables represent the input
features and target labels of the test dataset, respectively. The verbose parameter is set to 0
to suppress the progress bar during evaluation. The evaluate() method computes the loss
value and metrics (specified during
compilation, in this case, accuracy) for the test dataset. The evaluation results are stored in
the scores variable, which typically contains the loss value and the values of specified
metrics. Finally, the accuracy metric is extracted from the scores variable and printed as a
percentage using string formatting, providing a measure of the model's performance in
correctly predicting sentiment labels on unseen data.

UTU/CGPIT/B-TECH/CE/SEM-6/MI (CE5008)