School of Computer Science Engineering and Technology

Assignment-7
Course-B. Tech. Type- Core
Course Code- CSET 301 Course Name-Artificial Intelligence and
Machine Learning
Year- 2024 Semester- Even
Date- 12/03/2024 –15/03/2024 Batch-

CO-Mapping
CO1 CO2 CO3
Q1 √ √
Q2 √ √

Objectives (Q1)
To understand the process of constructing a decision tree using the ID3 algorithm and apply it to a given
dataset.

Question 1:
Consider a dataset consisting of the following points with their corresponding labels:

Point x1 x2 Label
P1 1 2 +1
P2 2 3 -1
P3 3 2 +1
P4 4 3 -1
P5 2 1 +1

You are required to implement the AdaBoost algorithm to classify the data points. Follow the steps below
to complete the implementation:

1. Initialize the weights of each data point to 1/N, where N is the total number of data points.
2. For the first iteration, choose a weak classifier that minimizes the error relative to the weights. For
simplicity, consider using decision stumps (one-level decision trees) that threshold a single feature.
Calculate the error rate of this classifier.
3. Update the weights of the data points based on the classification error. Increase the weights of the
misclassified points and decrease the weights of the correctly classified points.
4. Compute the classifier's weight (α) using the formula α = 0.5 *ln((1 – error)/error).
5. Normalize the weights of the data points so that they sum up to 1.
6. Repeat the steps for a second iteration to find another weak classifier and update the weights again.
7. Provide the final hypothesis H(x) as the weighted majority vote of the two weak classifiers.
 School of Computer Science Engineering and Technology

Tasks:

a) Write a program to perform the first iteration of AdaBoost: select the weak classifier, calculate its error,
update the weights, and compute the classifier's weight (α).

b) Write a program to perform the second iteration of AdaBoost using the updated weights: select another
weak classifier, update the weights, and compute the classifier's weight (α).

c) Write a program to combine the two weak classifiers to form the final hypothesis H(x) and describe how
it would classify a new point x = (3, 3).

Objectives (Question 2)

Given the following dataset, implement the first two iterations of the Gradient Boosting algorithm to
perform regression. The dataset consists of five data points:

Point Feature (x) Target Value (y)

P1 1 2.5
P2 2 0.5
P3 3 3.5
P4 4 1.5
P5 5 2.0

The objective is to create a sequence of weak models, specifically decision trees, that successively improve
upon the residuals of the previous model. Follow these steps to perform the first two iterations of Gradient
Boosting:

1. Initialize the model with a constant prediction that minimizes the mean squared error (MSE) for the
target values.
2. For the first iteration, calculate the residuals by subtracting the initial prediction from the target values.
3. Fit a decision tree to these residuals with a depth of 1 (i.e., a tree with a single decision node and two
leaves).
4. Update the model by adding this tree's predictions, multiplied by a learning rate (assume a learning
rate of 0.1), to the initial prediction.
5. For the second iteration, calculate the new residuals by subtracting the updated prediction from the
target values.
6. Fit a new decision tree to these new residuals, again with a depth of 1.
7. Update the model by adding the second tree's predictions, multiplied by the learning rate, to the updated
prediction from step 4.

Tasks:

a) Write a program to determine the initial prediction that minimizes the MSE for the target values.

b) Write a program to perform the first iteration of Gradient Boosting: calculate residuals, fit a decision
tree to these residuals, and update the model.

c) Write a program to perform the second iteration: calculate new residuals, fit another decision tree, and
update the model.
d) What would be the predicted value for a new data point with feature x =3.5 after two iterations?
School of Computer Science Engineering and Technology