ADDITIONAL ASSIGNMENT

Devdutt Gautam
1904796
B.Tech (Mechanical)

Observation Table:

S.NO Architecture Learning Momentum Final Final Final

Rate Number of Training Testing
epochs error error

1. 2-3-1 0.8 0.01 10 0.008912 0.009574

2. 2-3-1 0.9 0.02 11 0.010835 0.008636
3. 2-4-1 0.8 0.01 8 0.010520 0.009973
4. 2-4-1 0.9 0.02 7 0.008543 0.007890
5. 2-6-1 0.8 0.01 8 0.007419 0.010180
6. 2-6-1 0.95 0.02 4 0.018448 0.009537
7. 2-6-1 0.6 0.01 9 0.010569 0.008167
8. 2-10-1 0.7 0.01 5 0.012636 0.008974
9. 2-10-1 0.9 0.02 4 0.012149 0.007680
10. 2-10-1 0.5 0.01 6 0.012909 0.008784

Observation:
The presented table shows that, each characterized by distinct architectures, learning rates, and momentum.
Notably, the number of neurons in hidden layers, represented as "2-3-1," "2-4-1," "2-6-1," and "2-10-1,"
demonstrates that increasing layer complexity does not consistently translate to improved performance. The
range of learning rates (from 0.5 to 0.95) reveals that higher rates do not universally yield superior
outcomes, exemplified by a higher final testing error in the case with a learning rate of 0.95. Momentum
values at 0.01 and 0.02 show no clear pattern of impact on performance. The variable number of training
epochs, spanning 4 to 11, indicates that a lower epoch count does not consistently lead to optimal results,
suggesting possible convergence issues. Interestingly, the relationship between training and testing errors is
not straightforward, highlighting the importance of considering overfitting. Overall, the observations
emphasize the nuanced interplay of architecture and hyperparameters, suggesting that more nuanced
analyses and fine-tuning may be necessary to optimize neural network performance.
Modified Code:
clear all test_error = 2 * tol;
% creating data set
x = linspace(-2,2,25); format long
y = x;
k = 1; train_errors = []; % store training errors for
for i = 1:25 each epoch
for j = 1:25 test_errors = []; % store testing errors for
pattern(k,:) = [x(i) y(j) exp(-(x(i)^2 + each epoch
y(j)^2))];
k = k +1; while (test_error > tol)
end epoch = epoch + 1;
end train_error = 0;
for k = 1:Q
index = randperm(625);
trainpattern = pattern(1:500,:); %80 percent % Forward propagation
testpattern = pattern(501:625,:); %20 Zh = Si(k,:) * Wih;
percent Sh = [1 1./(1 + exp(-Zh))];

eta = 0.8; % learning rate Yj = Sh * Whj;

alpha = 0.01; % momentum Sy = 1./(1 + exp(-Yj));
epsilon = 0.01;
% Backpropagation
Q = 500; % total no. of the patterns to be Ek = D(k) - Sy;
input deltaO = Ek .* Sy .* (1 - Sy);
n = 2;
q = 3; % neurons in hidden layer for h = 1:q+1
p = 1; % output neurons DeltaWhj(h,:) = deltaO * Sh(h);
end
% Weights initialization
% Compute delta for the hidden layer
Wih = 2 * rand(n+1,q) - 1;
for h = 2:q+1
Whj = 2 * rand(q+1,p) - 1;
deltaH(h) = (deltaO * Whj(h,:)') *
Sh(h) * (1 - Sh(h));
DeltaWih = zeros(n+1,q);
end
DeltaWhj = zeros(q+1,p);
for i = 1:n+1
DeltaWihOld = zeros(n+1,q);
DeltaWih(i,:) = deltaH(2:q+1) *
DeltaWhjOld = zeros(q+1,p);
Si(k,i);
end
Si = [ones(Q,1) trainpattern(:,1:2)];
D = trainpattern(:,3); % Update weights
Wih = Wih + eta * DeltaWih + alpha *
testSi = [ones(125,1) testpattern(:,1:2)]; DeltaWihOld;
testD = testpattern(:,3); Whj = Whj + eta * DeltaWhj + alpha *
DeltaWhjOld;
Sh = [1 zeros(1,q)];
Sy = zeros(1,p); % Update previous deltas
DeltaWihOld = DeltaWih;
deltaO = zeros(1,p); DeltaWhjOld = DeltaWhj;
deltaH = zeros(1,q+1);
% Compute training error
epoch = 0; train_error = train_error + sum(Ek.^2);
tol = 0.01; % tolerance fprintf('epoch=%d \t %9.6f \t %9.6f \
end n',epoch, train_error, test_error)
 train_error = train_error / Q; end
train_errors(epoch) = train_error; % Store % Plotting the error curves
training error for this epoch epochs = 1:epoch;
figure;
% Compute test error plot(epochs, train_errors, 'b', 'LineWidth',
test_error = 0; 2);
for l = 1:125 hold on;
Zh = testSi(l,:) * Wih; plot(epochs, test_errors, 'r', 'LineWidth', 2);
Sh = [1 1./(1 + exp(-Zh))]; xlabel('Epochs');
Yj = Sh * Whj; ylabel('Error');
Sy = 1./(1 + exp(-Yj)); title('Training and Testing Error Curves');
testEk = testD(l) - Sy; legend('Training Error', 'Testing Error');
test_error = test_error + sum(testEk^2); grid on;
end
test_error = test_error / 125;
test_errors(epoch) = test_error; % Store
testing error for this epoch

Results:

S.n Architechtur Learnin Momentu Graph

o e g Rate m
1. 2-3-1 0.8 0.01
2. 2-3-1 0.9 0.02

3. 2-4-1 0.8 0.01

4. 2-4-1 0.9 0.02

5. 2-6-1 0.8 0.01

6. 2-6-1 0.95 0.02

7. 2-6-1 0.6 0.01

8. 2-10-1 0.7 0.01

9. 2-10-1 0.9 0.02

10. 2-10-1 0.5 0.01