Artificial Intelligence

and
Machine Learning
Syllabus
• UNIT - I
Introduction : Basics of problem solving, problem representation; Search Techniques: Problem size,
complexity; Uninformed search techniques- Depth, Breadth, Uniform Cost, Depth Limited, Iterative
deepening DFS.
• UNIT - II
Informed search techniques : Heuristic Based Search, Greedy Based First Search, A* Search;
Local
Search algorithms: Hill-climbing, Simulated Annealing, Genetic Algorithms.
• UNIT - IIIThe concept learning task, General-to-specific ordering of hypotheses. Version spaces.
Inductive bias.Decision Tree Learning. Rule Learning: Propositional and First-Order, Over-fitting,
Cross-Validation.
Experimental Evaluation of Learning Algorithms.
• UNIT - IV
Instance-Based Learning : k-Nearest neighbor algorithm, Radial basis functions. Case- based
learning,
K-means and Hierarchical Clustering.
• UNIT - V
Artificial Neural Networks : Linear threshold units, Perceptions, Multilayer networks and back-
propagation, recurrent networks.
• UNIT - VI
Probabilistic Machine Learning : Maximum Likelihood Estimation, MAP, Bayes Classifiers Naive
Bayes, Bayes optimal classifiers, Minimum description length principle
Text Books
• Stuart Russel and Peter Norvig; Artificial Intelligence: A Modern
Approach; Third Edition;Pearson Education, 2009

• Tom Mitchell; Machine Learning – an Artificial Intelligence

Approach, Volume-II

• Ethem Alpaydin; Introduction to machine Learning, Third

Edition
Course Outcomes

• On successful completion of the course, students will be able to:

1. Represent given problem using state space representation.
2. Apply uninformed and informed search techniques on it.
3. Analyze learning systems in Artificial Intelligence.
4. Apply uncertainty theory based on techniques like probability theory in
AI and Machine Learning system.
Intelligence
Intelligence

The ability of a system to calculate, reason, perceive relationships and analogies,

learn from experience, store and retrieve information from memory, solve problems,
comprehend complex ideas, use natural language fluently, classify, generalize, and
adapt new situations.
Types of Intelligence
Artificial Intelligence
Artificial
Intelligence
• making computers that think?
• the automation of activities we associate
with human thinking, like decision
making, learning ... ?
• the art of creating machines that perform
functions that require intelligence when
performed by people ?
• the study of mental faculties through the
use of computational models ?
What is Artificial Intelligence?

Systems that Systems that

THOU
think think
GHT
like humans rationally
Systems that Systems that
act act
BEHAV
like humans rationally
IOUR

HUM RATIO
AN NAL
Systems that act like humans:
Turing Test( 1950 by Alen
Turing)
• “The art of creating machines that perform functions that require intelligence when
performed by people.” (Kurzweil)
• “The study of how to make computers do things at which, at the moment, people
are better.” (Rich and Knight)
Systems that act like humans

?
• You enter a room which has a computer terminal. You have a fixed period of time to type
what you want into the terminal, and study the replies. At the other end of the line is either a
human being or a computer system.
• If it is a computer system, and at the end of the period you cannot reliably determine
whether it is a system or a human, then the system is deemed to be intelligent.
Systems that act like humans

• The Turing Test approach

• a human questioner cannot tell if
• there is a computer or a human answering his question, via
teletype (remote communication)
• The computer must behave intelligently
• Intelligent behavior
• to achieve human-level performance in all cognitive tasks
Systems that act like humans
• These cognitive tasks include:
– Natural language processing
• for communication with human
– Knowledge representation
• to store information effectively & efficiently
– Automated reasoning
• to retrieve & answer questions using the stored
information
– Machine learning
• to adapt to new circumstances
The total Turing Test

• Includes two more issues:

– Computer vision
• to perceive objects (seeing)
– Robotics
• to move objects (acting)
What is Artificial Intelligence ?

THOUG Systems that Systems that

HT think think
like humans rationally

Systems that Systems that

BEHAVIO act act
UR like humans rationally
HUM RATION
AN AL
Systems that think like humans:
• Humans as observed from ‘inside’
• How do we know how humans think?
– Introspection vs. psychological experiments
• Cognitive Science
• “The exciting new effort to make computers think … machines
with minds in the full and literal sense” (Haugeland)
• “[The automation of] activities that we associate with human
thinking, activities such as decision-making, problem solving,
learning …” (Bellman)
What is Artificial Intelligence ?

THOUG Systems that Systems that

HT think think
like humans rationally

Systems that Systems that

BEHAVIO act act
UR like humans rationally
HUM RATION
AN AL
Systems that think ‘rationally’ "laws of thought"

• Humans are not always ‘rational’

• Rational - defined in terms of logic?
• Logic can’t express everything (e.g. uncertainty)
• Logical approach is often not feasible in terms of computation
time (needs ‘guidance’)
• “The study of mental facilities through the use of computational
models” (Charniak and McDermott)
• “The study of the computations that make it possible to perceive,
reason, and act” (Winston)
What is Artificial Intelligence ?

THOUG Systems that Systems that

HT think think
like humans rationally

Systems that Systems that

BEHAVIO act act
UR like humans rationally
HUM RATION
AN AL
Systems that act rationally: “Rational agent”
• Rational behavior: doing the right thing
• The right thing: that which is expected to maximize goal achievement,
given the available information
• Giving answers to questions is ‘acting’.
• I don't care whether a system:
– replicates human thought processes
– makes the same decisions as humans
– uses purely logical reasoning
What is Intelligence Composed of?
What is Intelligence Composed of?
• Reasoning :
It is the set of processes that enables us to provide basis for judgement, making
decisions, and prediction.
• Learning:
It is the activity of gaining knowledge or skill by studying, practising, being taught,
or experiencing something. Learning enhances the awareness of the subjects of
the study.
• Problem Solving:
It is the process in which one perceives and tries to arrive at a desired solution
from a present situation by taking some path, which is blocked by known or
unknown hurdles.
Problem solving also includes decision making, which is the process of selecting
the best suitable alternative out of multiple alternatives to reach the desired goal
are available.
What is Intelligence Composed of?
• Perception :
It is the process of acquiring, interpreting, selecting, and organizing
sensory information. Perception presumes sensing.
In humans, perception is aided by sensory organs.
In the domain of AI, perception mechanism puts the data acquired by the
sensors together in a meaningful manner.
• Linguistic Intelligence:
It is one’s ability to use, comprehend, speak, and write the verbal and
written language. It is important in interpersonal communication.
Task Classification of AI
Agents
• An agent is an entity that perceives and acts
Problem Solving
• Problem-solving is commonly known as the method to reach the desired goal or find a solution
to a given situation. In computer science, problem-solving refers to artificial intelligence
techniques, including various techniques such as forming efficient algorithms, heuristics, and
performing root cause analysis to find desirable solutions.
• Problem-solving in Artificial Intelligence usually refers to researching a solution to a problem
by performing logical algorithms, utilizing polynomial and differential equations, and executing
them using modeling paradigms.
• There can be various solutions to a single problem, which are achieved by different heuristics.
Also, some problems have unique solutions. It all rests on the nature of the given problem.
Steps of Problem Solving
• Problem definition: Detailed specification of inputs and acceptable system solutions.
• Problem analysis: Analyse the problem thoroughly.

• Knowledge Representation: collect detailed information about the problem and define all
possible techniques.
• Problem-solving: Selection of best techniques.
Measuring Problem Solving Performance:
• Completeness: Is the algorithm guaranteed to find a solution when there is one?
• Optimality: Does the strategy find the optimal ?

• Time Complexity: How long does it take to find a solution?

• Space Complexity: How much memory is needed to perform the search?
Problem Representation
• Problem-solving is commonly known as the method to reach the desired goal or find a solution
to a given situation. In computer science, problem-solving refers to artificial intelligence
techniques, including various techniques such as forming efficient algorithms, heuristics, and
performing root cause analysis to find desirable solutions.
• Problem-solving in Artificial Intelligence usually refers to researching a solution to a problem
by performing logical algorithms, utilizing polynomial and differential equations, and executing
them using modeling paradigms.
• There can be various solutions to a single problem, which are achieved by different heuristics.
Also, some problems have unique solutions. It all rests on the nature of the given problem.
Water jug problem?
Problem
Statement/Definition
The water jug problem: You are given two jugs, a 4-gallons one and a
3-gallons one. Neither has any measuring markers on it. There is a pump
that can be used to fill the jugs with water. How can you get exactly 2
gallons of water into 4-gallons jug.
Explanatio
n Initial state Condition Final/Goal
? state
No Measurement
2
given Anythin
Jug- Jug-B g
4 Jug- Jug-B
Agallon 3 gallon
capacit capacit A
y y
Proposed Solution Gallons in Four Jug Gallons in Three Jug Rules Applied

0 0 0
0 0 0 3 2
3 0 7

0 3 3 2
3
4 2 11
0 2 3
2 0 10
3 0

3 3
Final/Goal state

0
4 2 2 0
2
Exercise

Consider the 3-water jug problem, which is stated as follows:

There are three jugs of capacity 8,5 and 3 liters. There is no mark on any jug.
Initially the 8-liter jug is filled with water and other two jugs are empty. You cannot
get water from any external source and you cannot pour water on the ground. How
to get 4 liter of water in 8-liter jug.
Determine state representation of 3-water-jug problem.
Determine operators (rules) for 3-water-jug problem.
 Searching
Techniques
Uninformed Search
Strategies:
• Also known as “blind search,”
uninformed search strategies use
no information about the likely
“direction” of the goal node(s)
Informed search
strategies
• Aka “heuristic search,” informed
search strategies use information
about the domain to (try to)
(usually) head in the general
direction of the goal node(s)

• Uninformed search methods: • Informed search methods: Hill

Breadth-first, depth-first, climbing, best-first, greedy
depth-limited, uniform-cost, search, beam search, A, A*
depth-first iterative deepening,
bidirectional
Uninformed Search Strategies

Breadth First Search

Depth First Search

Uniform Cost

Depth Limited

Iterative Depending DFS

Graphs
• graph: A data structure containing:
– a set of vertices V, (sometimes called nodes) a b
– a set of edges E, where an edge
represents a connection between 2 vertices.
c d
• Graph G = (V, E )
• an edge is a pair (v, w ) where v, w are in V
• the graph at right:
– V = {a, b, c, d}
– E = {(a, c), (b, c), (b, d), (c, d)}

• degree: number of edges touching a given vertex.

– at right: a=1, b=2, c=3, d=2
 Graph examples
• For each, what are the vertices and what are the edges?
– Web pages with links
– Methods in a program that call each other
– Road maps (e.g., Google maps)
– Airline routes
– Facebook friends
– Course pre-requisites
– Family trees
– Paths through a maze
 Paths
• path: A path from vertex a to b is a sequence of edges that can be
followed starting from a to reach b.
– can be represented as vertices visited, or edges taken
– example, one path from V to Z: {b, h} or {V, X, Z}
– What are two paths from U to Y?
V
a b
• path length: Number of vertices U d X Z
or edges contained in the path. h
c e
W g
• neighbor or adjacent: Two vertices
connected directly by an edge. f
Y
– example: V and X
 Reachability, connectedness
• reachable: Vertex a is reachable from b V
a b
if a path exists from a to b. d
U X Z
• connected: A graph is connected if every c h
e
vertex is reachable from any other. W g
– Is the graph at top right connected? f
Y
• strongly connected: When every vertex
has an edge to every other vertex.
a b
a b c d
e
c d
Loops and cycles
• cycle: A path that begins and ends at the same node.
– example: {b, g, f, c, a} or {V, X, Y, W, U, V}.
– example: {c, d, a} or {U, W, V, U}.

– acyclic graph: One that does V

a b
not contain any cycles. U d X Z
h
c e
W g
• loop: An edge directly from
a node to itself. f Y
– Many graphs don't allow loops.
 Weighted graphs
• weight: Cost associated with a given edge.
– Some graphs have weighted edges, and some are unweighted.
– Edges in an unweighted graph can be thought of as having equal weight
(e.g. all 0, or all 1, etc.)
– Most graphs do not allow negative weights.

• example: graph of airline flights, weighted by miles between

849 cities:
18 4 3 ORD PVD
1 4
SFO 2

802
17 4 LGA
337

2555 3 138 10
HNL
LAX 1233 DF 7
1120 9 9
W MIA
 Directed graphs
• directed graph ("digraph"): One where edges are one-way
connections between vertices.
– If graph is directed, a vertex has a separate in/out degree.
– A digraph can be weighted or unweighted.
– Is the graph below connected? Why or why not?
a b

c d e

f g
 Digraph example
• Vertices = UW CSE courses (incomplete list)
• Edge (a, b) = a is a prerequisite for b
12 13 14 14 45
0 1 2 0 0
14 15
3 4
33 31
34 35 37 37
1 1 33
34 1 1 3 4
40 31 2 35 33
3 2 4 3
2 41 41 41
43 44 44 42 44
1 6 4 1 0 45 0 3 4
41 41
1 5 7
45
2
 Digraph example
• Vertices = UW CSE courses (incomplete list)
• Edge (a, b) = a is a prerequisite for b
12 13 14 14 45
0 1 2 0 0
14 15
3 4
33 31
34 35 37 37
1 1 33
34 1 1 3 4
40 31 2 35 33
3 2 4 3
2 41 41 41
43 44 44 42 44
1 6 4 1 0 45 0 3 4
41 41
1 5 7
45
2
 Digraph example
• Vertices = UW CSE courses (incomplete list)
• Edge (a, b) = a is a prerequisite for b
12 13 14 14 45
0 1 2 0 0
14 15
3 4
33 31
34 35 37 37
1 1 33
34 1 1 3 4
40 31 2 35 33
3 2 4 3
2 41 41 41
43 44 44 42 44
1 6 4 1 0 45 0 3 4
41 41
1 5 7
45
2
Breadth First Search
Example:
Take a weighted graph and start/goal node an input. Your job is to find
goal node. Print the total cost and path.

Basic
Fundamentals
First iteration
s will be popped from the queue
Neighbors of s i.e. 1 and 2 will be traversed
1 and 2, which have not been traversed earlier, are traversed. They will be:
Pushed in the queue
1 and 2 will be marked as visited

Second iteration
1 is popped from the queue
Neighbors of 1 i.e. s and 3 are traversed
s is ignored because it is marked as 'visited'
3, which has not been traversed earlier, is traversed. It is:
Pushed in the queue
Marked as visited

Third iteration
2 is popped from the queue
Neighbors of 2 i.e. s, 3, and 4 are traversed
3 and s are ignored because they are marked as 'visited'
4, which has not been traversed earlier, is traversed. It is:
Pushed in the queue
Marked as visited
Fourth iteration
3 is popped from the queue
Neighbors of 3 i.e. 1, 2, and 5 are traversed
1 and 2 are ignored because they are marked as 'visited'
5, which has not been traversed earlier, is traversed. It is:
Pushed in the queue
Marked as visited

Fifth iteration
4 will be popped from the queue
Neighbors of 4 i.e. 2 is traversed
2 is ignored because it is already marked as 'visited'

Sixth iteration
5 is popped from the queue
Neighbors of 5 i.e. 3 is traversed
3 is ignored because it is already marked as 'visited'
Breadth First
Search
Breadth First
Search
Breadth First
Search
Step 1: Take an Empty Queue.
Step 2: Select a starting node (visiting a node) and insert it into the Queue.
Step 3: Provided that the Queue is not empty, extract the node from the
Queue and insert its child nodes (exploring a node) into the Queue.
Step 4: Print the extracted node.
Don’t worry if you’re confused, we shall understand this with an example.
Take a look at the below graph, we will use the Breadth-First Search
algorithm to traverse through the graph.
Breadth First
Search
• Assign ‘a’ as the root node and insert it into the Queue.
• Extract node ‘a’ from the queue and insert the child nodes of ‘a’, i.e., ‘b’
and ‘c’.
• Print node ‘a’.
• The queue is not empty and has node ‘b’ and ‘c’. Since ‘b’ is the first
node in the queue, let’s extract it and insert the child nodes of ‘b’, i.e.,
node ‘d’ and ‘e’.
• Repeat these steps until the queue gets empty. Note that the nodes
that are already visited should not be added to the queue again.
Applications Of Breadth-First
Search Algorithm
• GPS Navigation
systems
• Broadcasting
Depth First Search
Depth First
Search
• Depth First Search (DFS) algorithm traverses a graph in a depth ward
motion and uses a stack to remember to get the next vertex to start a
search, when a dead end occurs in any iteration.
Depth First
Search
• Rule 1 − Visit the adjacent unvisited vertex. Mark it as visited. Display it.
Push it in a stack.
• Rule 2 − If no adjacent vertex is found, pop up a vertex from the stack. (It
will pop up all the vertices from the stack, which do not have adjacent
vertices.)
• Rule 3 − Repeat Rule 1 and Rule 2 until the stack is empty.
Depth First
Search
Depth First
Search
Depth First
Search
Uniform Cost
Uniform Cost
Search
• Uniform-cost search is a searching algorithm used for traversing a
weighted tree or graph. This algorithm comes into play when a different
cost is available for each edge.
• The primary goal of the uniform-cost search is to find a path to the goal
node which has the lowest cumulative cost.
• Uniform-cost search expands nodes according to their path costs form the
root node.
• It can be used to solve any graph/tree where the optimal cost is in
demand. A uniform-cost search algorithm is implemented by the priority
queue.
• It gives maximum priority to the lowest cumulative cost. Uniform cost
search is equivalent to BFS algorithm if the path cost of all edges is the
same.