PKM Educational Trust ®

R. R. Institute of Technology
Affiliated to VTU Belgaum and Approved by AICTE, New Delhi, Recognized by Govt. of
Karnataka,
Accredited by NAAC with ‘B+

LAB MANUAL
(2023-24) ODD SEM

18CSL76

AIML
LABORATORY

VII Semester

DEPARTMENT OF INFORMATION SCIENCE & ENGINEERING

Name:

USN:

Batch:
 VISVESVARAYA TECHNOLOGICAL UNIVERSITY
JnanaSangama, Belagavi, Karnataka–590018

VII SEMESTER / B.E.

AIML LABORATORY

18CSL76

Prepared By:
Prof. NISHCHITTHA T S
Assistant Professor, Dept. of ISE
 VISION AND MISSION OF INSTITUTE

VISION
To be a best institution imparting quality engineering education to deal with community
needs through learning and performance.

MISSION
To implement path breaking student centric education methods.
To augment talent, nurture teamwork to transform to develop individual as responsible
citizen.
To educate the students and faculties about entrepreneurship to meet vibrant
requirements of the society.
Strengthen Industry-Institute Interaction for knowledge sharing

VISION AND MISSION OF DEPARTMENT

Vision

To be a potential quality source of expert computer engineers, addressing the

requirements of industry and civic demands at large.

Mission
To enable learning in emerging technologies adopting innovative methods.
To enhance ability, cultivate collaboration to bring change in building students
approach towards society.
To make available platform for harnessing entrepreneurial and leadership qualities.
To partner continuously with industry to inculcate practical knowledge

PROGRAM EDUCATIONAL OBJECTIVES (PEO)

The program educational objectives of Bachelor of Engineering in Computer Science &

Engineering at Sapthagiri College of Engineering are broadly defined on following four
counts.

PEO 1: Graduates will have the expertise in analyzing real time problems and providing
appropriate solutions related to Computer Science & Engineering.

PEO 2: Graduates will have the knowledge of fundamental principles and innovative
technologies to succeed in higher studies, and research.

PEO 3: Graduates will continue to learn and to adapt technology developments combined
with deep awareness of ethical responsibilities in profession
 PROGRAM SPECIFIC OUTCOMES(PSO)

The graduates of Computer Science and Engineering program at Sapthagiri College of

engineering will be able to attain the following at the time of graduation.

PSO 1: Apply the knowledge gained from Mathematics, Basic Computing, Basic Sciences
and Social Sciences in general and all computer science courses in particular to
identify, formulate and solve real life engineering problems.

PSO2: Identify the various analysis & design methodologies for facilitating development of
high quality system software products with focus on performance optimization.

PROGRAM OUTCOMES(POs)

Graduation of Bachelor of Computer Science and Engineering program at R R Institute of

Technologywill attain the following program outcomes in the field of Information
Science & Engineering.

PO1. Engineering knowledge: Apply the knowledge of mathematics, science, engineering

fundamentals, and an engineering specialization to the solution of complex
engineering problems.

PO2. Problem analysis: Identify, formulate, research literature, and analyze complex
engineering problems reaching substantiated conclusions using first principles of
mathematics, natural sciences, and engineering sciences.

PO3. Design/development of solutions: Design solutions for complex engineering

problems and design system components or processes that meet the specified needs
with appropriate consideration for the public health and safety, and the cultural,
societal, and environmental considerations.

PO4. Conduct investigations of complex problems: Use research-based knowledge and

research methods including design of experiments, analysis and interpretation of data,
and synthesis of the information to provide valid conclusions.

PO5. Modern tool usage: Create, select, and apply appropriate techniques, resources, and
modern engineering and IT tools including prediction and modeling to complex
engineering activities with an understanding of the limitations.

PO6. The engineer and society: Apply reasoning informed by the contextual knowledge to
assess societal, health, safety, legal and cultural issues and the consequent
responsibilities relevant to the professional engineering practice.

PO7. Environment and sustainability: Understand the impact of the professional

engineering solutions in societal and environmental contexts, and demonstrate the
knowledge of, and need for sustainable development.
PO8. Ethics: Apply ethical principles and commit to professional ethics and responsibilities
and norms of the engineering practice.

PO9. Individual and team work: Function effectively as an individual, and as a member or
leader in diverse teams, and in multidisciplinary settings.

PO10. Communication: Communicate effectively on complex engineering activities with

the engineering community and with society at large, such as, being able to
comprehend and write effective reports and design documentation, make effective
presentations, and give and receive clear instructions.

PO11. Project management and finance: Demonstrate knowledge and understanding of

the engineering and management principles and apply these to one’s own work, as a
member and leader in a team, to manage projects and in multidisciplinary
environments.

PO12. Life-long learning: Recognize the need for, and have the preparation and ability to
engage in independent and life-long learning in the broadest context of technological
change.
 DO’S AND DON’TS

Do’s

Do wear ID card and follow dress code. Do log off the computers when you finish.
Be on time to LAB sessions.
Do ask for assistance if you need help.
Do keep your voice low when speaking to others in the LAB.
Do make suggestions as to how we can improve the LAB.
In case of any hardware related problem, ask LAB in charge for solution.
If you are the last one leaving the LAB, make sure that the staff in charge of the
LABis informed to close the LAB
Do keep the LAB as clean as possible.
Do prepare for lab programs before entering the lab

Don’ts
Do not use mobile phone inside the lab.
Don’t do anything that can make the LAB dirty (like eating, throwing waste
papers etc).
Do not carry any external devices without permission.
Don’t move the chairs of the LAB.
Don’t interchange any part of one computer with another.
Don’t leave the computers of the LAB turned on while leaving the LAB.
Do not install or download any software or modify or delete any system files onany
lab computers.
Do not damage, remove, or disconnect any labels, parts, cables, or equipment.
Don’t attempt to bypass the computer security system.
Do not read or modify other user’s file.
If you leave the lab, do not leave your personal belongings unattended.
We are not responsible for any theft.
 i

INDEX
SL NO. Program Title

1 Implement A* Search algorithm.

2 Implement AO* Search algorithm.

3 For a given set of training data examples stored in a .CSV file, implement
and demonstrate the Candidate-Elimination algorithm to output a
description of the set of all hypotheses consistent with the training
examples.

4 Write a program to demonstrate the working of the decision tree-based

ID3 algorithm. Use an appropriate data set for building the decision tree and
apply this knowledge to classify a new sample.

5 Build an Artificial Neural Network by implementing the Backpropagation

algorithm and test the same using appropriate data sets.

6 Write a program to implement the naïve Bayesian classifier for a sample

training data set stored as a .CSV file. Compute the accuracy of the
classifier, considering few test data sets.

7 Apply EM algorithm to cluster a set of data stored in a .CSV file. Use the
same data set for clustering using the k-Means algorithm. Compare the
results of these two algorithms and comment on the quality of clustering.
You can add Java/Python ML library classes/API in the program.

8 Write a program to implement the k-Nearest Neighbor algorithm to classify

the iris data set. Print both correct and wrong predictions. Java/Python ML
library classes can be used for this problem.

9 Implement the non-parametric Locally Weighted Regression algorithm in

order to fit data points. Select the appropriate data set for your experiment
and draw graphs.
 1. Implement A* Search algorithm.

def aStarAlgo(start_node, stop_node):

open_set = set(start_node)
closed_set = set()
g = {}
parents = {}

g[start_node] = 0
parents[start_node] = start_node
while len(open_set) > 0:
n = None
for v in open_set:
if n == None or g[v] + heuristic(v) < g[n] + heuristic(n):
n=v
if n == stop_node or Graph_nodes[n] is None:
pass
else:
for (m, weight) in get_neighbours(n):
if m not in open_set and m not in closed_set:
open_set.add(m)
parents[m] = n
g[m] = g[n] + weight
else:
if g[m] > g[n] + weight:
g[m] = g[n] + weight
parents[m] = n
if m in closed_set:
closed_set.remove(m)
open_set.add(m)
if n is None:
print('path does not exist!')
return None
if n == stop_node:
path = []
while parents[n] != n:
path.append(n)
n = parents[n]
path.append(start_node)
path.reverse()
print('path found: {}'.format(path))
return path
open_set.remove(n)
closed_set.add(n)
print('path does not exist!')
return None
def get_neighbours(v):
if v in Graph_nodes:
return Graph_nodes[v]
else:
return None

def heuristic(n):
H_dist = {
'A': 10,
'B': 8,
'C': 5,
'D': 7,
'E': 3,
'F': 6,
'G': 5,
'H': 3,
'I': 1,
'J': 0
}
return H_dist[n]

Graph_nodes = {
'A': [('B', 6), ('F', 3)],
'B': [('C', 3), ('D', 2)],
'C': [('D', 1), ('E', 5)],
'D': [('C', 1), ('E', 8)],
'E': [('I', 5), ('J', 5)],
'F': [('G', 1), ('H', 7)],
'G': [('I', 3)],
'H': [('I', 2)],
'I': [('E', 5), ('J', 3)]
}

aStarAlgo('A', 'J')

Output:
path found: ['A', 'F', 'G', 'I', 'J']

2. Implement AO* Search algorithm.

class Graph:
def __init__(self, graph, heuristicNodeList, startNode): # instantiate
graph object with graph topology,
# heuristic values, start node

self.graph = graph
self.H = heuristicNodeList
self.start = startNode
self.parent = {}
self.status = {}
self.solutionGraph = {}

def applyAOStar(self): # starts a recursive AO* algorithm

self.aoStar(self.start, False)

def getNeighbors(self, v): # gets the Neighbors of a given node

return self.graph.get(v, '')

def getStatus(self, v): # return the status of a given node

return self.status.get(v, 0)

def setStatus(self, v, val): # set the status of a given node

self.status[v] = val

def getHeuristicNodeValue(self, n):

return self.H.get(n, 0) # always return the heuristic value of a given
node

def setHeuristicNodeValue(self, n, value):

self.H[n] = value # set the revised heuristic value of a given node

def printSolution(self):
print("FOR GRAPH SOLUTION, TRAVERSE THE GRAPH FROM THE
START NODE:", self.start)
print("------------------------------------------------------------")
print(self.solutionGraph)
print("------------------------------------------------------------")

def computeMinimumCostChildNodes(self, v): # Computes the Minimum

Cost of child nodes of a given node v
minimumCost = 0
costToChildNodeListDict = {}
costToChildNodeListDict[minimumCost] = []
flag = True
 for nodeInfoTupleList in self.getNeighbors(v): # iterate over all the set
of child node/s
cost = 0
nodeList = []
for c, weight in nodeInfoTupleList:
cost = cost + self.getHeuristicNodeValue(c) + weight
nodeList.append(c)

if flag == True: # initialize Minimum Cost with the cost of first set of
child node/s
minimumCost = cost
costToChildNodeListDict[minimumCost] = nodeList # set the
Minimum Cost child node/s
flag = False
else: # checking the Minimum Cost nodes with the current Minimum
Cost
if minimumCost > cost:
minimumCost = cost
costToChildNodeListDict[minimumCost] = nodeList # set the
Minimum Cost child node/s

return minimumCost, costToChildNodeListDict[minimumCost] #

return Minimum Cost and Minimum Cost child node/s

def aoStar(self, v, backTracking): # AO* algorithm for a start node and

backTracking status flag

print("HEURISTIC VALUES :", self.H)

print("SOLUTION GRAPH :", self.solutionGraph)
print("PROCESSING NODE :", v)
print("-----------------------------------------------------------------------------------------")

if self.getStatus(v) >= 0: # if status node v >= 0, compute Minimum

Cost nodes of v
minimumCost, childNodeList =
self.computeMinimumCostChildNodes(v)
self.setHeuristicNodeValue(v, minimumCost)
self.setStatus(v, len(childNodeList))

solved = True # check the Minimum Cost nodes of v are solved

for childNode in childNodeList:
self.parent[childNode] = v
if self.getStatus(childNode) != -1:
solved = solved & False
 if solved == True: # if the Minimum Cost nodes of v are solved, set
the current node status as solved(-1)
self.setStatus(v, -1)
self.solutionGraph[
v] = childNodeList # update the solution graph with the solved
nodes which may be a part of
# solution

if v != self.start: # check the current node is the start node for

backtracking the current node value
self.aoStar(self.parent[v],
True) # backtracking the current node value with
backtracking status set to true

if not backTracking: # check the current call is not for backtracking

for childNode in childNodeList: # for each Minimum Cost child
node
self.setStatus(childNode, 0) # set the status of child node to
0(needs exploration)
self.aoStar(childNode,
False) # Minimum Cost child node is further explored
with backtracking status as false

h1 = {'A': 1, 'B': 6, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 5, 'H': 7, 'I': 7, 'J': 1, 'T': 3}
graph1 = {
'A': [[('B', 1), ('C', 1)], [('D', 1)]],
'B': [[('G', 1)], [('H', 1)]],
'C': [[('J', 1)]],
'D': [[('E', 1), ('F', 1)]],
'G': [[('I', 1)]]
}
G1 = Graph(graph1, h1, 'A')
G1.applyAOStar()
G1.printSolution()

h2 = {'A': 1, 'B': 6, 'C': 12, 'D': 10, 'E': 4, 'F': 4, 'G': 5, 'H': 7} # Heuristic values
of Nodes
graph2 = { # Graph of Nodes and Edges
'A': [[('B', 1), ('C', 1)], [('D', 1)]], # Neighbors of Node 'A', B, C & D with
respective weights
'B': [[('G', 1)], [('H', 1)]], # Neighbors are included in a list of lists
'D': [[('E', 1), ('F', 1)]] # Each sublist indicate a "OR" node or "AND"
nodes
}
 G2 = Graph(graph2, h2, 'A') # Instantiate Graph object with graph, heuristic
values and start Node
G2.applyAOStar() # Run the AO* algorithm
G2.printSolution() # Print the solution graph as output of the AO* algorithm
search

Output:
HEURISTIC VALUES : {'A': 1, 'B': 6, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 5, 'H': 7, 'I': 7, 'J':
1, 'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 10, 'B': 6, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 5, 'H': 7, 'I': 7,
'J': 1, 'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : B
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 10, 'B': 6, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 5, 'H': 7, 'I': 7,
'J': 1, 'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 10, 'B': 6, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 5, 'H': 7, 'I': 7,
'J': 1, 'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : G
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 10, 'B': 6, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 8, 'H': 7, 'I': 7,
'J': 1, 'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : B
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 10, 'B': 8, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 8, 'H': 7, 'I': 7,
'J': 1, 'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 12, 'B': 8, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 8, 'H': 7, 'I': 7,
'J': 1, 'T': 3}
SOLUTION GRAPH : {}
PROCESSING NODE : I
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 12, 'B': 8, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 8, 'H': 7, 'I': 0,
'J': 1, 'T': 3}
SOLUTION GRAPH : {'I': []}
PROCESSING NODE : G
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 12, 'B': 8, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0,
'J': 1, 'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I']}
PROCESSING NODE : B
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 12, 'B': 2, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0,
'J': 1, 'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I'], 'B': ['G']}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 6, 'B': 2, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0, 'J':
1, 'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I'], 'B': ['G']}
PROCESSING NODE : C
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 6, 'B': 2, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0, 'J':
1, 'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I'], 'B': ['G']}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 6, 'B': 2, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0, 'J':
1, 'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I'], 'B': ['G']}
PROCESSING NODE : J
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 6, 'B': 2, 'C': 2, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0, 'J':
0, 'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I'], 'B': ['G'], 'J': []}
PROCESSING NODE : C
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 6, 'B': 2, 'C': 1, 'D': 12, 'E': 2, 'F': 1, 'G': 1, 'H': 7, 'I': 0, 'J':
0, 'T': 3}
SOLUTION GRAPH : {'I': [], 'G': ['I'], 'B': ['G'], 'J': [], 'C': ['J']}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
FOR GRAPH SOLUTION, TRAVERSE THE GRAPH FROM THE START NODE: A
------------------------------------------------------------
{'I': [], 'G': ['I'], 'B': ['G'], 'J': [], 'C': ['J'], 'A': ['B', 'C']}
------------------------------------------------------------
HEURISTIC VALUES : {'A': 1, 'B': 6, 'C': 12, 'D': 10, 'E': 4, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 11, 'B': 6, 'C': 12, 'D': 10, 'E': 4, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {}
PROCESSING NODE : D
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 11, 'B': 6, 'C': 12, 'D': 10, 'E': 4, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 11, 'B': 6, 'C': 12, 'D': 10, 'E': 4, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {}
PROCESSING NODE : E
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 11, 'B': 6, 'C': 12, 'D': 10, 'E': 0, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {'E': []}
PROCESSING NODE : D
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 11, 'B': 6, 'C': 12, 'D': 6, 'E': 0, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {'E': []}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 7, 'B': 6, 'C': 12, 'D': 6, 'E': 0, 'F': 4, 'G': 5, 'H': 7}
SOLUTION GRAPH : {'E': []}
PROCESSING NODE : F
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 7, 'B': 6, 'C': 12, 'D': 6, 'E': 0, 'F': 0, 'G': 5, 'H': 7}
SOLUTION GRAPH : {'E': [], 'F': []}
PROCESSING NODE : D
-----------------------------------------------------------------------------------------
HEURISTIC VALUES : {'A': 7, 'B': 6, 'C': 12, 'D': 2, 'E': 0, 'F': 0, 'G': 5, 'H': 7}
SOLUTION GRAPH : {'E': [], 'F': [], 'D': ['E', 'F']}
PROCESSING NODE : A
-----------------------------------------------------------------------------------------
FOR GRAPH SOLUTION, TRAVERSE THE GRAPH FROM THE START NODE: A
------------------------------------------------------------
{'E': [], 'F': [], 'D': ['E', 'F'], 'A': ['D']}
-----------------------------------------------------------

3. For a given set of training data examples stored in a .CSV file,

implement and demonstrate the Candidate-Elimination algorithm to
output a description of the set of all hypotheses consistent with the
training examples.

import csv

with open("CandidateElimination.csv") as f:
csv_file = csv.reader(f)
 data = list(csv_file)

s = data[1][:-1]
g = [['?' for i in range(len(s))] for j in range(len(s))]

for i in data:
if i[-1] == "Yes":
for j in range(len(s)):
if i[j] != s[j]:
s[j] = '?'
g[j][j] = '?'

elif i[-1] == "No":

for j in range(len(s)):
if i[j] != s[j]:
g[j][j] = s[j]
else:
g[j][j] = "?"
print("\nSteps of Candidate Elimination Algorithm", data.index(i) + 1)
print(s)
print(g)
gh = []
for i in g:
for j in i:
if j != '?':
gh.append(i)
break
print("\nFinal specific hypothesis:\n", s)

print("\nFinal general hypothesis:\n", gh)

Datasheets:
CSV FILE: CandidateElimanation.csv
sky,airtemp,Humidity,Wind,Water,forecast,Enjoy sport
Sunny,Warm,Normal,Strong,Warm,Same,Yes
Sunny,Warm,high,Strong,Warm,Same,Yes
Rainy,Cold,high,Strong,Warm,Change,No
Sunny,Warm,high,Strong,Cool,Change,Yes

Output:
Steps of Candidate Elimination Algorithm 1 ['Sunny', 'Warm', 'Normal', 'Strong',
'Warm', 'Same'] [['?', '?', '?',
'?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?',
'?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?']]

Steps of Candidate Elimination Algorithm 2 ['Sunny', 'Warm', 'Normal', 'Strong',

'Warm', 'Same'] [['?', '?', '?',
'?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?',
'?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?']]

Steps of Candidate Elimination Algorithm 3 ['Sunny', 'Warm', '?', 'Strong', 'Warm',

'Same'] [['?', '?', '?', '?',
'?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?',
'?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?']]

Steps of Candidate Elimination Algorithm 4 ['Sunny', 'Warm', '?', 'Strong', 'Warm',

'Same'] [['Sunny', '?', '?', '?',
'?', '?'], ['?', 'Warm', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?',
'?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', 'Same']]

Steps of Candidate Elimination Algorithm 5 ['Sunny', 'Warm', '?', 'Strong', '?', '?']
[['Sunny', '?', '?', '?', '?',
'?'], ['?', 'Warm', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?',
'?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?']]

Final specific hypothesis:

['Sunny', 'Warm', '?', 'Strong', '?', '?']

Final general hypothesis:

[['Sunny', '?', '?', '?', '?', '?'], ['?', 'Warm', '?', '?', '?', '?']]

4. Write a program to demonstrate the working of the decision tree-based

ID3 algorithm. Use an appropriate data set for building the decision tree and
apply this knowledge to classify a new sample.

import numpy as np
import pandas as pd
PlayTennis = pd.read_csv("PlayTennis.csv")
PlayTennis
from sklearn.preprocessing import LabelEncoder
Le = LabelEncoder()
PlayTennis['outlook'] = Le.fit_transform(PlayTennis['outlook'])
PlayTennis['temp'] = Le.fit_transform(PlayTennis['temp'])
PlayTennis['humidity'] = Le.fit_transform(PlayTennis['humidity'])
PlayTennis['windy'] = Le.fit_transform(PlayTennis['windy'])
PlayTennis['play'] = Le.fit_transform(PlayTennis['play'])
PlayTennis
y = PlayTennis['play']
X = PlayTennis.drop(['play'],axis=1)
from sklearn import tree
clf = tree.DecisionTreeClassifier(criterion = 'entropy')
clf = clf.fit(X, y)
tree.plot_tree(clf)
X_pred = clf.predict(X)
X_pred == y

Datasets: Filename: PlayTennis.csv

outlook,temp,humidity,windy,play
Sunny,Hot,High,Weak,No
Sunny,Hot,High,Strong,No
Overcast,Hot,High,Weak,Yes
Rain,Mild,High,Weak,Yes
Rain,Cool,Normal,Weak,Yes
Rain,Cool,Normal,Strong,No
Overcast,Cool,Normal,Strong,Yes
Sunny,Mild,High,Weak,No
Sunny,Cool,Normal,Weak,Yes
Rain,Mild,Normal,Weak,Yes
Sunny,Mild,Normal,Strong,Yes
Overcast,Mild,High,Strong,Yes
Overcast,Hot,Normal,Weak,Yes
Rain,Mild,High,Strong,No

Output:
0 True
1 True
2 True
3 True
4 True
5 True
6 True
7 True
8 True
9 True
10 True
11 True
12 True
13 True
Name: play, dtype: bool

5. Build an Artificial Neural Network by implementing the Backpropagation

algorithm and test the same using appropriate data sets.
----------------------------------------------------------------------------------------------------------------

import numpy as np

X = np.array(([2, 9], [1, 5], [3, 6]), dtype=float)

# X = (hours sleeping, hours studying)

y = np.array(([92], [86], [89]), dtype=float)

# y = score on test

# scale units
X = X/np.amax(X, axis=0) # maximum of X array

y = y/100 # max test score is 100

class Neural_Network(object):

def __init__(self):

# Parameters

self.inputSize = 2

self.outputSize = 1

self.hiddenSize = 3

# Weights

self.W1 = np.random.randn(self.inputSize,
self.hiddenSize)

# (3x2) weight matrix from input to hidden layer

self.W2 = np.random.randn(self.hiddenSize,
self.outputSize)

# (3x1) weight matrix from hidden to output layer

def forward(self, X):

#forward propagation through our network

self.z = np.dot(X, self.W1)

# dot product of X (input) and first set of 3x2 weights

self.z2 = self.sigmoid(self.z)

# activation function

self.z3 = np.dot(self.z2, self.W2)

# dot product of hidden layer (z2) and second set of 3x1 weights
 o = self.sigmoid(self.z3)

# final activation function

return o

def sigmoid(self, s):

return 1/(1+np.exp(-s)) # activation function

def sigmoidPrime(self, s):

return s * (1 - s) # derivative of sigmoid

def backward(self, X, y, o):

# backward propgate through the network

self.o_error = y - o

# error in output

self.o_delta = self.o_error*self.sigmoidPrime(o)

# applying derivative of sigmoid to

self.z2_error = self.o_delta.dot(self.W2.T)

# z2 error: how much our hidden layer weights contributed to

output error

self.z2_delta = self.z2_error*self.sigmoidPrime(self.z2)
# applying derivative of sigmoid to z2 error

self.W1 += X.T.dot(self.z2_delta)

# adjusting first set (input --> hidden) weights

self.W2 += self.z2.T.dot(self.o_delta)

# adjusting second set (hidden --> output) weights

def train (self, X, y):

o = self.forward(X)
 self.backward(X, y, o)

NN = Neural_Network()

print ("\nInput: \n" + str(X))

print ("\nActual Output: \n" + str(y))

print ("\nPredicted Output: \n" + str(NN.forward(X)))

print ("\nLoss: \n" + str(np.mean(np.square(y -

NN.forward(X)))))

# mean sum squared loss)

NN.train(X, y)

---------------------------------------------------------------------------------------------------------------------

Output………..
Input:
[[0.66666667 1. ]
[0.33333333 0.55555556]
[1. 0.66666667]]

Actual Output:
[[0.92]
[0.86]
[0.89]]

Predicted Output:
[[0.47300011]
[0.4970386 ]
[0.48683558]]

Loss:
0.16469714028370117
6. Write a program to implement the naïve Bayesian classifier for a sample
training data set stored as a .CSV file. Compute the accuracy of the
classifier, considering few test data sets.
-----------------------------------------------------------------------------------------------------------------------------------------

# import necessary libarities

import pandas as pd

from sklearn import tree

from sklearn.preprocessing import LabelEncoder

from sklearn.naive_bayes import GaussianNB

# load data from CSV

data = pd.read_csv('tennisdata.csv')

print("THe first 5 values of data is :\n",data.head())

# obtain Train data and Train output

X = data.iloc[:,:-1]

print("\nThe First 5 values of train data is\n",X.head())

y = data.iloc[:,-1]

print("\nThe first 5 values of Train output is\n",y.head())

# Convert then in numbers

le_outlook = LabelEncoder()

X.Outlook = le_outlook.fit_transform(X.Outlook)

le_Temperature = LabelEncoder()

X.Temperature = le_Temperature.fit_transform(X.Temperature)

le_Humidity = LabelEncoder()

X.Humidity = le_Humidity.fit_transform(X.Humidity)
le_Windy = LabelEncoder()

X.Windy = le_Windy.fit_transform(X.Windy)

print("\nNow the Train data is :\n",X.head())

le_PlayTennis = LabelEncoder()

y = le_PlayTennis.fit_transform(y)

print("\nNow the Train output is\n",y)

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X,y,

test_size=0.20)

classifier = GaussianNB()

classifier.fit(X_train,y_train)

from sklearn.metrics import accuracy_score

print("Accuracy
is:",accuracy_score(classifier.predict(X_test),y_test))

Datasheets
Csvfile: tennisdata.csv
Outlook Temperature Humidity Windy PlayTennis
Sunny Hot High FALSE No
Sunny Hot High TRUE No
Overcast Hot High FALSE Yes
Rainy Mild High FALSE Yes
Rainy Cool Normal FALSE Yes
Rainy Cool Normal TRUE No
Overcast Cool Normal TRUE Yes
Sunny Mild High FALSE No
Sunny Cool Normal FALSE Yes
Rainy Mild Normal FALSE Yes
Sunny Mild Normal TRUE Yes
Overcast Mild High TRUE Yes
Overcast Hot Normal FALSE Yes
Rainy Mild High TRUE No
output:
THe first 5 values of data is :
Outlook Temperature Humidity Windy PlayTennis
0 Sunny Hot High False No
1 Sunny Hot High True No
2 Overcast Hot High False Yes
3 Rainy Mild High False Yes
4 Rainy Cool Normal False Yes

The First 5 values of train data is

Outlook Temperature Humidity Windy
0 Sunny Hot High False
1 Sunny Hot High True
2 Overcast Hot High False
3 Rainy Mild High False
4 Rainy Cool Normal False

The first 5 values of Train output is

0 No
1 No
2 Yes
3 Yes
4 Yes
Name: PlayTennis, dtype: object

Now the Train data is :

Outlook Temperature Humidity Windy
0 2 1 0 0
1 2 1 0 1
2 0 1 0 0
3 1 2 0 0
4 1 0 1 0

Now the Train output is

[0 0 1 1 1 0 1 0 1 1 1 1 1 0]
Accuracy is: 1.0

7. Apply EM algorithm to cluster a set of data stored in a .CSV file. Use the
same data set for clustering using the k-Means algorithm. Compare the
results of these two algorithms and comment on the quality of clustering.
You can add Java/Python ML library classes/API in the program.
--------------------------------------------------------------------------------------------------------

from sklearn.cluster import KMeans

from sklearn import preprocessing

from sklearn.mixture import GaussianMixture

from sklearn.datasets import load_iris

import sklearn.metrics as sm

import pandas as pd

import numpy as np

import matplotlib.pyplot as plt

dataset=load_iris()

# print(dataset)

X=pd.DataFrame(dataset.data)

X.columns=['Sepal_Length','Sepal_Width','Petal_Length','Petal_Wi
dth']

y=pd.DataFrame(dataset.target)

y.columns=['Targets']

# print(X)

plt.figure(figsize=(14,7))

colormap=np.array(['red','lime','black'])

# REAL PLOT

plt.subplot(1,3,1)

plt.scatter(X.Petal_Length,X.Petal_Width,c=colormap[y.Targets],s
=40)

plt.title('Real')

# K-PLOT

plt.subplot(1,3,2)
model=KMeans(n_clusters=3)

model.fit(X)

predY=np.choose(model.labels_,[0,1,2]).astype(np.int64)

plt.scatter(X.Petal_Length,X.Petal_Width,c=colormap[predY],s=40)

plt.title('KMeans')

# GMM PLOT

scaler=preprocessing.StandardScaler()

scaler.fit(X)

xsa=scaler.transform(X)

xs=pd.DataFrame(xsa,columns=X.columns)

gmm=GaussianMixture(n_components=3)

gmm.fit(xs)

y_cluster_gmm=gmm.predict(xs)

plt.subplot(1,3,3)

plt.scatter(X.Petal_Length,X.Petal_Width,c=colormap[y_cluster_gm
m],s=40)

plt.title('GMM Classification')

------------------------------------------------------------------------------------------------------------------------------
output

8. Write a program to implement the k-Nearest Neighbor algorithm to classify

the iris data set. Print both correct and wrong predictions. Java/Python ML
library classes can be used for this problem.
----------------------------------------------------------------------------------------------------------------
from sklearn.model_selection import train_test_split

from sklearn.neighbors import KNeighborsClassifier

from sklearn.metrics import classification_report, confusion_matrix

from sklearn import datasets

iris=datasets.load_iris()

iris_data=iris.data

iris_labels=iris.target

print(iris_data)

print(iris_labels)

x_train, x_test, y_train, y_test=train_test_split(iris_data,iris_labels,test_size=0.30)

classifier=KNeighborsClassifier(n_neighbors=5)

classifier.fit(x_train,y_train)

y_pred=classifier.predict(x_test)

print('confusion matrix is as follows')

print(confusion_matrix(y_test,y_pred))

print('Accuracy metrics')

print(classification_report(y_test,y_pred))

----------------------------------------------------------------------------------------------------

output………………….
[[5.1 3.5 1.4 0.2]
[4.9 3. 1.4 0.2]
[4.7 3.2 1.3 0.2]
[4.6 3.1 1.5 0.2]
[5. 3.6 1.4 0.2]
[5.4 3.9 1.7 0.4]
[4.6 3.4 1.4 0.3]
[5. 3.4 1.5 0.2]
[4.4 2.9 1.4 0.2]
[4.9 3.1 1.5 0.1]
[5.4 3.7 1.5 0.2]
[4.8 3.4 1.6 0.2]
[4.8 3. 1.4 0.1]
[4.3 3. 1.1 0.1]
[5.8 4. 1.2 0.2]
[5.7 4.4 1.5 0.4]
[5.4 3.9 1.3 0.4]
[5.1 3.5 1.4 0.3]
[5.7 3.8 1.7 0.3]
[5.1 3.8 1.5 0.3]
[5.4 3.4 1.7 0.2]
[5.1 3.7 1.5 0.4]
[4.6 3.6 1. 0.2]
[5.1 3.3 1.7 0.5]
[4.8 3.4 1.9 0.2]
[5. 3. 1.6 0.2]
[5. 3.4 1.6 0.4]
[5.2 3.5 1.5 0.2]
[5.2 3.4 1.4 0.2]
[4.7 3.2 1.6 0.2]
[4.8 3.1 1.6 0.2]
[5.4 3.4 1.5 0.4]
[5.2 4.1 1.5 0.1]
[5.5 4.2 1.4 0.2]
[4.9 3.1 1.5 0.2]
[5. 3.2 1.2 0.2]
[5.5 3.5 1.3 0.2]
[4.9 3.6 1.4 0.1]
[4.4 3. 1.3 0.2]
[5.1 3.4 1.5 0.2]
[5. 3.5 1.3 0.3]
[4.5 2.3 1.3 0.3]
[4.4 3.2 1.3 0.2]
[5. 3.5 1.6 0.6]
[5.1 3.8 1.9 0.4]
[4.8 3. 1.4 0.3]
[5.1 3.8 1.6 0.2]
[4.6 3.2 1.4 0.2]
[5.3 3.7 1.5 0.2]
[5. 3.3 1.4 0.2]
[7. 3.2 4.7 1.4]
[6.4 3.2 4.5 1.5]
[6.9 3.1 4.9 1.5]
[5.5 2.3 4. 1.3]
[6.5 2.8 4.6 1.5]
[5.7 2.8 4.5 1.3]
[6.3 3.3 4.7 1.6]
[4.9 2.4 3.3 1. ]
[6.6 2.9 4.6 1.3]
[5.2 2.7 3.9 1.4]
[5. 2. 3.5 1. ]
[5.9 3. 4.2 1.5]
[6. 2.2 4. 1. ]
[6.1 2.9 4.7 1.4]
[5.6 2.9 3.6 1.3]
[6.7 3.1 4.4 1.4]
[5.6 3. 4.5 1.5]
[5.8 2.7 4.1 1. ]
[6.2 2.2 4.5 1.5]
[5.6 2.5 3.9 1.1]
[5.9 3.2 4.8 1.8]
[6.1 2.8 4. 1.3]
[6.3 2.5 4.9 1.5]
[6.1 2.8 4.7 1.2]
[6.4 2.9 4.3 1.3]
[6.6 3. 4.4 1.4]
[6.8 2.8 4.8 1.4]
[6.7 3. 5. 1.7]
[6. 2.9 4.5 1.5]
[5.7 2.6 3.5 1. ]
[5.5 2.4 3.8 1.1]
[5.5 2.4 3.7 1. ]
[5.8 2.7 3.9 1.2]
[6. 2.7 5.1 1.6]
[5.4 3. 4.5 1.5]
[6. 3.4 4.5 1.6]
[6.7 3.1 4.7 1.5]
[6.3 2.3 4.4 1.3]
[5.6 3. 4.1 1.3]
[5.5 2.5 4. 1.3]
[5.5 2.6 4.4 1.2]
[6.1 3. 4.6 1.4]
[5.8 2.6 4. 1.2]
[5. 2.3 3.3 1. ]
[5.6 2.7 4.2 1.3]
[5.7 3. 4.2 1.2]
[5.7 2.9 4.2 1.3]
[6.2 2.9 4.3 1.3]
[5.1 2.5 3. 1.1]
[5.7 2.8 4.1 1.3]
[6.3 3.3 6. 2.5]
[5.8 2.7 5.1 1.9]
[7.1 3. 5.9 2.1]
[6.3 2.9 5.6 1.8]
[6.5 3. 5.8 2.2]
[7.6 3. 6.6 2.1]
[4.9 2.5 4.5 1.7]
[7.3 2.9 6.3 1.8]
[6.7 2.5 5.8 1.8]
[7.2 3.6 6.1 2.5]
[6.5 3.2 5.1 2. ]
[6.4 2.7 5.3 1.9]
[6.8 3. 5.5 2.1]
[5.7 2.5 5. 2. ]
[5.8 2.8 5.1 2.4]
[6.4 3.2 5.3 2.3]
[6.5 3. 5.5 1.8]
[7.7 3.8 6.7 2.2]
[7.7 2.6 6.9 2.3]
[6. 2.2 5. 1.5]
[6.9 3.2 5.7 2.3]
[5.6 2.8 4.9 2. ]
[7.7 2.8 6.7 2. ]
[6.3 2.7 4.9 1.8]
[6.7 3.3 5.7 2.1]
[7.2 3.2 6. 1.8]
[6.2 2.8 4.8 1.8]
[6.1 3. 4.9 1.8]
[6.4 2.8 5.6 2.1]
[7.2 3. 5.8 1.6]
[7.4 2.8 6.1 1.9]
[7.9 3.8 6.4 2. ]
[6.4 2.8 5.6 2.2]
[6.3 2.8 5.1 1.5]
[6.1 2.6 5.6 1.4]
[7.7 3. 6.1 2.3]
[6.3 3.4 5.6 2.4]
[6.4 3.1 5.5 1.8]
[6. 3. 4.8 1.8]
[6.9 3.1 5.4 2.1]
[6.7 3.1 5.6 2.4]
[6.9 3.1 5.1 2.3]
[5.8 2.7 5.1 1.9]
[6.8 3.2 5.9 2.3]
 [6.7 3.3 5.7 2.5]
[6.7 3. 5.2 2.3]
[6.3 2.5 5. 1.9]
[6.5 3. 5.2 2. ]
[6.2 3.4 5.4 2.3]
[5.9 3. 5.1 1.8]]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
2 2]
confusion matrix is as follows
[[13 0 0]
[ 0 14 0]
[ 0 3 15]]
Accuracy metrics
precision recall f1-score support

0 1.00 1.00 1.00 13

1 0.82 1.00 0.90 14
2 1.00 0.83 0.91 18

accuracy 0.93 45
macro avg 0.94 0.94 0.94 45
weighted avg 0.95 0.93 0.93 45

9. Implement the non-parametric Locally Weighted Regression algorithm in

order to fit data points. Select the appropriate data set for your experiment
and draw graphs.
----------------------------------------------------------------------------------------------------------------

from math import ceil

import numpy as np

from scipy import linalg

def lowess(x, y, f, iterations):

n = len(x)

r = int(ceil(f * n))

h = [np.sort(np.abs(x - x[i]))[r] for i in range(n)]

w = np.clip(np.abs((x[:, None] - x[None, :]) / h), 0.0, 1.0)

 w = (1 - w ** 3) ** 3

yest = np.zeros(n)

delta = np.ones(n)

for iteration in range(iterations):

for i in range(n):

weights = delta * w[:, i]

b = np.array([np.sum(weights * y), np.sum(weights * y * x)])

A = np.array([[np.sum(weights), np.sum(weights *
x)],[np.sum(weights * x), np.sum(weights * x * x)]])

beta = linalg.solve(A, b)

yest[i] = beta[0] + beta[1] * x[i]

residuals = y - yest

s = np.median(np.abs(residuals))

delta = np.clip(residuals / (6.0 * s), -1, 1)

delta = (1 - delta ** 2) ** 2

return yest

import math

n = 100

x = np.linspace(0, 2 * math.pi, n)

y = np.sin(x) + 0.3 * np.random.randn(n)

f =0.25

iterations=3

yest = lowess(x, y, f, iterations)

import matplotlib.pyplot as plt

plt.plot(x,y,"r.")

plt.plot(x,yest,"b-")

------------------------------------------------------------------------------------------------------------- -------------------------------

output……………………..