21CS54- Artificial intelligence and Machine Learning Module-1

Foundations of Artificial Intelligence

Philosophy e.g., foundational issues (can a machine think?), issues of knowledge and believe,
mutual knowledge

• Psychology and Cognitive Science

e.g., problem solving skills

• Neuro-Science
e.g., brain architecture

• Computer Science and Engineering

e.g., complexity theory, algorithms, logic and inference, programming languages, and
system building

• Mathematics and Physics

e.g., statistical modeling, continuous mathematics,

• Statistical Physics, and Complex Systems.

History of Artificial Intelligence

Important research that laid the groundwork for AI:

• In 1931, Goedel layed the foundation of Theoretical Computer Science1920-30s:

He published the first universal formal language and showed that math itself is
either flawed or allows for unprovable but true statements.

• In 1936, Turing reformulated Goedel’s result and church’s extension thereof.

• In 1956, John McCarthy coined the term "Artificial Intelligence" as the topic of the
Dartmouth Conference, the first conference devoted to the subject.
• In 1957, The General Problem Solver (GPS) demonstrated by Newell, Shaw & Simon  In
1958, John McCarthy (MIT) invented the Lisp language.
• In 1959, Arthur Samuel (IBM) wrote the first game-playing program, for checkers, to
achieve sufficient skill to challenge a world champion.
• In 1963, Ivan Sutherland's MIT dissertation on Sketchpad introduced the idea of
interactive graphics into computing.
• In 1966, Ross Quillian (PhD dissertation, Carnegie Inst. of Technology; now CMU)
demonstrated semantic nets
• In 1967, Dendral program (Edward Feigenbaum, Joshua Lederberg, Bruce Buchanan,
Georgia Sutherland at Stanford) demonstrated to interpret mass spectra on organic
chemical compounds. First successful knowledge-based program for scientific reasoning.

Dr. Vijayashree Budyal, Prof. Dept. of ISE

21CS54- Artificial intelligence and Machine Learning Module-1

• In 1967, Doug Engelbart invented the mouse at SRI  In 1968, Marvin Minsky &
Seymour Papert publish Perceptrons, demonstrating limits of simple neural nets.
• In 1972, Prolog developed by Alain Colmerauer.
• In Mid 80’s, Neural Networks become widely used with the Backpropagation algorithm
(first described by Werbos in 1974).
• 1990, Major advances in all areas of AI, with significant demonstrations in machine
learning, intelligent tutoring, case-based reasoning, multi-agent planning, scheduling,
uncertain reasoning, data mining, natural language understanding and translation, vision,
virtual reality, games, and other topics.
• In 1997, Deep Blue beats the World Chess Champion Kasparov
• In 2002, iRobot, founded by researchers at the MIT Artificial Intelligence Lab, introduced
Roomba, a vacuum cleaning robot. By 2006, two million had been sold.

Dr. Vijayashree Budyal, Prof. Dept. of ISE

21CS54- Artificial intelligence and Machine Learning Module-1

Real world Problem

Consider the airline travel problems that must be solved by a travel-planning Web site:
States: Each state obviously includes a location (e.g., an airport) and the current time.
Furthermore, because the cost of an action (a flight segment) may depend on previous segments,
their fare bases, and their status as domestic or international, the state must record extra
information about these "historical" aspects.
Initial state: This is specified by the user's query.
Actions: Take any flight from the current location, in any seat class, leaving after the current
time, leaving enough time for within-airport transfer if needed.
Transition model: The state resulting from taking a flight will have the flight's desti-nation as
the current location and the flight's arrival time as the current time.
Goal test: Are we at the final destination specified by the user?
Path cost: This depends on monetary cost, waiting time, flight time, customs and im-migration
procedures, seat quality, time of day, type of airplane, frequent-flyer mileage awards, and so
on.
Other Examples such as
• Touring Problem
• VLSI layout
• A VLSI layout problem requires positioning millions of components and connections
on a chip to minimize area, minimize circuit delays, minimize stray capacitances, and
maximize manufacturing yield
• Robot Navigation

Searching for solutions

A solution is an action sequence, so search algorithms work by considering various possible
action sequences. The SEARCH TREE possible action sequences starting at the initial state
form a search tree with the initial state NODE at the root; the branches are actions and the
nodes correspond to states in the state space of the problem. In a Romania example Figure
shows the first few steps in growing the search tree for finding a route from Arad to Bucharest.
The root node of the tree corresponds to the initial state, In(Arad). The first step is to test
whether this is a goal state. (Clearly it is not, but it is important to check so that we can solve
trick problems like “starting in Arad, get to Arad.”) Then we need to consider taking .various
actions. We do this by expanding the current state; that is, applying each legal action to the
current state. thereby generating a new set of states. Inthis case, we add three branches from
the parent node In(Arad) leading to three new child nodes: in(Sibw), In(Timisaara), and
In(Zerind). Now we must choose which of these three possibilities to consider farther.

This is the essence of search—following up one option now and putting the others aside for
later, in case the first choice does not lead to a solution. Suppose we choose Sibiu first. We

Dr. Vijayashree Budyal, Prof. Dept. of ISE

21CS54- Artificial intelligence and Machine Learning Module-1

check to see whether it is a goal state (it is not) and then expand it to get In(Arad), In(Fagaras),
In(Oradea), and In(RimnicuVilcea). We can then choose any of these four or go LEAF NODE
back and choose Timisoara or Zerind. Each of these six nodes is a leaf node, that is, a node
with no children in the tree. The set of all leaf nodes available for expansion at any given
FRONTIER point is called the frontier. (Many authors call it the open list, which is both
geographically OPEN LIST less evocative and less accurate, because other data structures are
better suited than a list.) In Figure, the frontier of each tree consists of those nodes with bold
outlines. The process of expanding nodes on the frontier continues until either a solution is
found or there are no more states to expand.

Searching for Solutions

•A solution is an action sequence and search algorithms consider various possible action
sequences.
• The possible action sequences starting at the initial state form a search tree with the initial
state at the root;
• The branches are actions and the nodes correspond to states in the state space of the problem.
• Expanding the current state is application of each legal action to the current state and
generation of a new set of states
• The current state is the parent node, newly generated states are child nodes
• Leaf node is a node with no children in the tree.

Dr. Vijayashree Budyal, Prof. Dept. of ISE

21CS54- Artificial intelligence and Machine Learning Module-1

• The set of all leaf nodes available for expansion at any given point is called the frontier.
• Search algorithms all share this basic structure; they vary primarily according to how they
choose which state to expand next: search strategy.
Measuring problem-solving performance
We can evaluate a search algorithm’s performance in four ways:
Completeness: Is the algorithm guaranteed to find a solution when there is one?
Optimality: Does the strategy find the optimal solution?
Time complexity: How long does it take to find a solution?
Space complexity: How much memory is needed to perform the search?

Uninformed and Informed search strategy

Uninformed search also called blind search. The strategies have no additional information
about states beyond that provided in the problem definition. All they can do is generate
successors and distinguish a goal state from a non-goal state. As it uses no information about
the problem to guide the search and therefore may not be very efficient. Two types of
Uninformed search are

(i) Breadth First Search

(ii) Depth First Search
Informed Search: Also called heuristic or intelligent search, uses information about the
problem to guide the search, usually guesses the distance to a goal state and therefore efficient,
but the search may not be always possible.
(i) Greedy Best First Search
(ii) A* Algorithm
All search strategies are distinguished by the order in which nodes are expanded

Breadth First Search

Breadth first search algorithm is an algorithm used to find the shortest path from one node to
another in a graph. Breadth-first search is a simple strategy in which the root node is expanded
first, then all the successors of the root node are expanded next, then their successors, and so
on. In general, all the nodes are expanded at a given depth in the search tree before any nodes
at the next level are expanded. BFS (Breadth First Search) finds the shortest path. This is
achieved very simply by using a FIFO queue for the frontier. Thus, new nodes (which are
always deeper than their parents) go to the back of the queue, and old nodes, which are
shallower than the new nodes, get expanded first.

Dr. Vijayashree Budyal, Prof. Dept. of ISE

21CS54- Artificial intelligence and Machine Learning Module-1

BFS is a traversing algorithm where you should start traversing from a selected node (source
or starting node) and traverse the graph layer wise thus exploring the neighbour nodes (nodes
which are directly connected to source node). You must then move towards the next-level
neighbour nodes.
As the name BFS suggests, you are required to traverse the graph breadthwise as follows:
1. First move horizontally and visit all the nodes of the current layer
2. Move to the next layer

Advantages of Breadth First Search Algorithm

There are many advantages to using the Breadth First Search algorithm:
• The main advantage is that it is very simple to implement and understand.
• It is guaranteed to find the shortest path from the starting point to the goal.
• Breadth First Search tends to find paths with fewer steps than other algorithms, such as
Depth First Search.
• Breadth First Search can be easily parallelized, which means that it can take advantage
of multiple processors to speed up the search.
Disadvantages of Breadth First Search Algorithm
• There are a few disadvantages to the Breadth First Search algorithm:
• It can be very memory intensive since it needs to keep track of all the nodes in the
search tree.
• It can be slow since it expands all the nodes at each level before moving on to the next
level.
• It can sometimes find sub-optimal solutions since it doesn’t explore all possible paths
through the search tree.

Applications of Breadth First Search Algorithm

Dr. Vijayashree Budyal, Prof. Dept. of ISE

21CS54- Artificial intelligence and Machine Learning Module-1

The breadth first search algorithm is a powerful tool that can be used to solve various problems.
Potential applications of this algorithm.
1. Graph traversal: Breadth first search can be used to traverse a graph. This can be
useful for tasks such as finding the shortest path between two nodes or determining if
a graph is connected.
2. Network routing: Breadth first search can be used to find the shortest path between
two nodes in a network. This can be useful for tasks such as routing traffic or finding
the quickest route between two points.
3. Pattern matching: Breadth first search can be used to match patterns in data. This can
be useful for tasks such as searching for a specific string in a text file or finding all
instances of a particular word in a document.
4. Data mining: Breadth first search can be used to mine data for insights. This can be
useful for tasks such as finding trends in social media data or uncovering hidden
relationships in large datasets.
Limitations/Disadvantages BFS
• It can be very memory intensive since it needs to keep track of all the nodes in the
search tree.
• It can be slow since it expands all the nodes at each level before moving on to the next
level

Dr. Vijayashree Budyal, Prof. Dept. of ISE

21CS54- Artificial intelligence and Machine Learning Module-1

Depth First Search

Depth-first search always expands the deepest node in the current frontier of the search tree.
The search proceeds immediately to the deepest level of the search tree, where the nodes have
no successors. As those nodes are expanded, they are dropped from the frontier, so then the
search “backs up” to the next deepest node that still has unexplored successors.
Breadth-first-search uses a FIFO queue, depth-first search uses a LIFO queue. A LIFO queue
means that the most recently generated node is chosen for expansion. This must be the deepest
unexpanded node because it is one deeper than its parent—which, in turn, was the deepest
unexpanded node when it was selected.
We may sometimes search the goal along the largest depth of the tree, and move up only when
further traversal along the depth is not possible. We then attempt to find alternative offspring
of the parent of the node (state) last visited. If we visit the nodes of a tree using the above
principles to search the goal, the traversal made is called depth first traversal and consequently
the search strategy is called depth first search.
Example:

Step 1: Initially fringe contains only the node for A

FRINGE: A
Step 2: A is removed from fringe. A is expanded and its children B and C are put in front of
fringe.

Dr. Vijayashree Budyal, Prof. Dept. of ISE

21CS54- Artificial intelligence and Machine Learning Module-1

FRINGE: B C
Step 3: Node B is removed from fringe, and its children D and E are pushed in front of fringe.

FRINGE: D E C
Step 4: Node D is removed from fringe. C and F are pushed in front of fringe.

FRINGE: C F E C
Step 5: Node C is removed from fringe. Its child G is pushed in front of fringe.

FRINGE: G F E C
Step 6: Node G is expanded and found to be a goal node.

Dr. Vijayashree Budyal, Prof. Dept. of ISE

21CS54- Artificial intelligence and Machine Learning Module-1

FRINGE: G F E C
The solution path A-B-D-C-G is returned and the algorithm terminates.
****
Depth first search

1. The algorithm takes exponential time.

2. If N is the maximum depth of a node in the search space, in the worst case the algorithm
will take time O(b d ).

3. The space taken is linear in the depth of the search tree, O(bN).

Note that the time taken by the algorithm is related to the maximum depth of the search tree. If
the search tree has infinite depth, the algorithm may not terminate. This can happen if the search
space is infinite. It can also happen if the search space contains cycles. The latter case can be
handled by checking for cycles in the algorithm. Thus, Depth First Search is not complete.

Dr. Vijayashree Budyal, Prof. Dept. of ISE