Decision tree classifier

Dishant Kumar Yadav 2021BCS0136

Implementation:

General Terms: Let us first discuss a few statistical concepts used in this post.

Entropy: The entropy of a dataset, is a measure the impurity, of the dataset Entropy can also be
thought, as a measure of uncertainty. We should try to minimize, the Entropy. The goal of
machine learning models is to reduce uncertainty or entropy, as far as possible.

Information Gain: Information gain, is a measure of, how much information, a feature gives us
about the classes. Decision Trees algorithm, will always try, to maximize information gain.
Feature, that perfectly partitions the data, should give maximum information. A feature, with the
highest Information gain, will be used for split first.

keyboard_arrow_down Import Libraries:

We are going to import NumPy and the pandas library.

# Import the required libraries

import pandas as pd
import numpy as np

from google.colab import files

uploaded = files.upload()

Choose Files diabetes11.csv

diabetes11.csv(text/csv) - 7491 bytes, last modified: 17/1/2024 - 100% done
Saving diabetes11.csv to diabetes11.csv

import shutil

# Assuming the file name is 'diabetes.csv'

shutil.move('diabetes11.csv', '/content/diabetes11.csv')
 '/content/diabetes11.csv'

import os

# List files in the /content directory

os.listdir('/content')

['.config',
'diabetes (1).csv',
'diabetes.csv',
'diabetes11.csv',
'sample_data']

import pandas as pd

# Read the CSV file into a DataFrame

df = pd.read_csv('/content/diabetes11.csv')

# Display the first few rows of the DataFrame

df.head()
 1 to 5 of 5 entries Filter
index Glucose BloodPressure diabetes
0 148 72 1
1 85 66 0
2 183 64 1
3 89 66 0
4 137 40 1
Show 25 per page

Like what you see? Visit the data table notebook to learn more about interactive tables.

Distributions

2-d distributions

Time series

# Define the calculate entropy function

def calculate_entropy(df_label):
classes,class_counts = np.unique(df_label,return_counts = True)
entropy_value = np.sum([(-class_counts[i]/np.sum(class_counts))*np.log2(class_counts
for i in range(len(classes))])
return entropy_value
# Define the calculate information gain function
def calculate_information_gain(dataset,feature,label):
# Calculate the dataset entropy
dataset_entropy = calculate_entropy(dataset[label])
values,feat_counts= np.unique(dataset[feature],return_counts=True)

# Calculate the weighted feature entropy # Call the ca

weighted_feature_entropy = np.sum([(feat_counts[i]/np.sum(feat_counts))*calculate_ent
==values[i]).dropna()[label]) for i in range(len(values))]
feature_info_gain = dataset_entropy - weighted_feature_entropy
return feature_info_gain
# Set the features and label
features = df.columns[:-1]
label = 'diabetes'
parent=None
features

Index(['Glucose', 'BloodPressure'], dtype='object')

import numpy as np

def create_decision_tree(dataset, df, features, label, parent=None):

datum = np.unique(df[label], return_counts=True)
unique_data = np.unique(dataset[label])

if len(unique_data) <= 1:
return unique_data[0]

elif len(dataset) == 0:
return unique_data[np.argmax(datum[1])]

elif len(features) == 0:
return parent

else:
parent = unique_data[np.argmax(datum[1])]

# call the calculate_information_gain function

item_values = [calculate_information_gain(dataset, feature, label) for feature in
optimum_feature = features[np.argmax(item_values)]