Problem Solving

Problem:
 A problem is really a collection of
information that the agent will use to
decide what to do.
 Problem means- trouble or distress.
 For example, merger of company is
the problem as well opportunity.
Problem Solving
 The environment of the problem is
represented by a state space.
 A state space represents the set of
all possible states for a given
problem.
 A path through the state space from
the initial state to a goal state is a
solution.
Problem Solving

A problem consists of four parts:

1. Initial state

2. Operators
3. Goal test
4. Path cost
Problem Solving
 Initial state:-
The initial state that agent knows itself to
be in.
 Operators:-
 The set of possible actions is called
operators.
 The term operator is used to denote the
description of an action in terms of which
state will be reached by carrying out the
action in particular state.
Problem Solving

 Goal test:
 The goal is objective to be achieved,
or final answer, or solution.
 It is final destination to reach from
initial state to final or goal state .
 It is simply solution to be achieved.
Problem Solving

 Path cost:-
 A path cost function a function that
assigns a cost to a path.
 The cost of path is the sum of the
costs of the individual actions along
the path.
Problem Solving

Problem Solving is
 performance/output/feedback of thinking.
 mental activity of finding solution to a
problem.
 it was introduced by mathematicians.
 it is equivalent to decision making in the
world of business.
Problem Solving
 Problem solving is the one of the major
objective of AI expert systems to solve
problems like human do.
 Decision making, or problem solving, is
a difficult task, even for people.
 Reasoning is the major characteristic of
intelligent behavior in problem solving.
The process of Problem Solving

Problem Solving steps:

There are six steps in problem solving.
1. Problem identification and definition
2. Identifying evaluating criteria
3. Generation of Alternatives
4. Search solution and evaluation
5. Choice and Recommendation
6. Implementation
 Problem Identification and Definition

Identifying Evaluating Criteria

Generation of Alternatives

Search for solution and Evaluation

Choice and Recommendation

Implementation

Problem Solving Process

The process of Problem Solving

1. Problem identification
and definition:-
 A problem (or opportunity must first
be recognized.
 Its magnitude and importance are
determined and defined .
The process of Problem Solving

2. Identifying evaluating criteria:-

 Solution to a problem depends on the

criteria used to judge the possible
alternatives.
 For example, finding a good investment
depends on criteria (or objectives) such as
safety, liquidity and rate of return.
The process of Problem Solving

3. Generation of Alternatives:-

 Two or more courses of action (one

of which can be doing nothing) to be
in a decision situation.
 Generation of potential solutions
requires creativity and ingenuity.
The process of Problem Solving

4. Search solution and evaluation:-

 This step involves examination of the
solution candidates in light of the criteria
outlined earlier.
 It is basically a search process since we are
looking for the best or "good enough"
solutions.
 Several search, evaluation, and reasoning
methodologies can be used in this step.
The process of Problem Solving

5. Choice and Recommendation:-

The product of a search is selecting a
solution to recommend as a remedy
for the problem.
6. Implementation:-
To solve the problem, the
recommended solution needs to be
successfully implemented.
The process of Problem Solving

 Applied AI technologies can be used to

support all of the six steps.
 However, most of the AI action takes place in
support of steps 4 and 5.
 Expert systems are used to find a solution
among alternatives.
 Thus the role of AI is basically conducting a
search and an evaluation using some
inference capabilities.
Search strategies

Criteria for Searching Strategies/Algorithms:-

There are four criteria to evaluate search
strategies.
 Completeness:- is the strategy
guaranteed to find a solution when there
is one.
 Time Complexity:- how long does it
take to find a solution?
Search strategies

 Space Complexity:- how much

memory does it need to perform the
search?
 Optimality:- does the strategy find
the highest-quality solution when
there are several different solutions.
Problem Solving in AI
 Applied AI technologies are being
associated primarily with the search
and evaluation steps of the problem-
solving process.
 The objective is to automate these
steps as much as possible.
Search Approaches

Many approaches and strategies are used

to search for an appropriate solution to
problems.

Some of these approaches are

1. Informal Approach
2. Formal Approach
Search Approaches

1. Informal Approach:-

It involve intuition, “gut feeling” or

just acting on impulses
Search Approaches

2. Formal Approach:-
Formal approaches can be classified
into three categories:

1. Optimization
2. Blind Search
3. Heuristic search
Search Approaches

1. Optimization:-
 It involves numeric and quantitative
analysis.
 It attempts to find the best possible
solution by using mathematical
formulas that model specific
situations.
Search Approaches
 The problem domain must be fairly
structured, and the optimization is
conducted by using either a one-step
formula or an algorithm.
 an algorithm is a step-by-step search
process in which solutions are
generated and tested for possible
improvements.
Search Approaches
 Optimization is used extensively in
non-AI technologies such as
operations research (management
science) and mathematics.
 In AI, blind search and heuristic search
are used extensively.
Search Approaches/techniques

Search:-
The process of sifting through
alternative solutions to proceed
from initial state to goal state is
called a search.
In AI, you begin with initial state
and attempt to reach a goal state.
Searching Techniques

Blind Search:-
 A blind search (also called an uninformed search) is a
search that has no information about its domain.
 The only thing that a blind search can do is distinguish
a non-goal state from a goal state.
 The description of a desired solution is given
while conducting this search.
 This is called a goal.
 For example, a goal can be to identify the
best location for a plant or to approve or
disapprove a request for a loan.
Blind Search
 The possible steps leading from initial
conditions (e.g., information about the
alternatives or existing symptoms) to the
goal are viewed as the search steps.
 Problem solving is carried out by searching
through the space of possible solutions.
 Blind search techniques explore the
alternatives and events of a decision
situation, one at a time.
Blind Search
There are two types of blind search:
Complete or Exhaustive Enumeration
In complete (exhaustive) enumeration all
alternatives are considered; therefore, an optimal
solution is discovered.

Incomplete or Partial Search:

In incomplete (partial) search, the search continues
until a good enough solution is found.
Blind Search
 In blind situations the search path is arbitrary.
 No intelligent decision making is employed to direct
the search.

Limitations:
 Time Consuming
 Occupies much more computer storage
 It is not practical to solve the large problems
Combinatorial Explosion
 It is used in many situations.
 In conducting a blind search one may
encounter a situation where number of
alternatives to be considered expands
exponentially.
 Such problems are the combinatorial
problems.
 The major characteristic of combinatorial
problems is that the number of alternative
solutions increase much faster than the size
of the problem.
Combinatorial Explosion

For example, problem of matching

candidates and jobs, by increasing the
number of candidates and jobs.
As a result, a blind search may take
years to accomplish, even with the aid
of sophisticated computers
Heuristic Search
 For many applications, it is possible to find specific
information to guide the search process and thus reduce the
amount of computation.
 This is called heuristic information, and the search
procedures that use I are called heuristic search methods.
 A heuristic is a method that
 might not always find the best solution
 but is guaranteed to find a good solution in reasonable time.
– By sacrificing completeness it increases efficiency.
– Useful in solving tough problems which
 could not be solved any other way.

 solutions take an infinite time or very long time to compute.

 The classic example of heuristic search methods is the traveling salesman
problem.
Heuristic Search

 Heuristic means:
-Helping to discover or learn
-Serving to discover
-an aid to discovery
 A heuristic contributes to reduction of search
in a problem solving activity.
 Heuristic- simply rules of thumb or “rules”
 Heuristics are the decision rules regarding
how problem should be solved.
Heuristic Search
 Heuristics are developed on a basis of solid, rigorous analysis of the
problem and sometimes involve designed experimentation.
 Heuristic problem solving is the implementation of heuristics in
problem-solving or in decision-making situations.

The use of heuristics are:

 hints,
 intuition,
 judgment,
 rules of thumb, and
 inspiration.
 Heuristic search is much faster and cheaper than a blind search.
 The results are considered good enough, and in the case of
quantitative analysis they are very close to optimal solutions.
Search Directions

A search in AI can be goal directed, data

directed, or a combination of both.
Data-Directed (Forward) Search

 A data-directed search starts from

available information (or facts) and
tries to draw conclusions regarding the
situation or the goal attainment.
 For example, if a company is losing
sales volume, the search attempts to
find out why (e.g., because of
insufficient advertising).
Goal-Directed (Backward) Search

 A goal-directed search starts from expectations of what the

goal is or what is to happen; then it seeks evidence that
supports (or contradicts) those expectations (or
hypotheses).
 For example, we expect that sales declined because we think
our advertising budget is insufficient.
 The goal-directed search will confirm or deny our
expectations.
 AI-in contrast with conventional problem-solving techniques
such as operations research, management science, or
decision support systems-employs basically blind and
heuristic approaches.
 The application of these approaches enables Al to tackle
problems that conventional programming cannot handle.
Problem Representation in AI

 To understand how the blind and

the heuristic searches work, it is
necessary first to illustrate how
problems are represented in AI.
State-Space Representation

The general process of solving any problem

using AI involves three major elements:

 problem states,
 a goal, and
 operators.
State-Space Representation

 Problem states:-
Problem states define the problem situation and
existing conditions.
 States:-
 States are snapshots of varying conditions in the
environment.
 For example a state can be "you cannot start your
car," or "there is an oil leak."
 States can also be potential alternative solutions to
problems.
 All states are unique.
State-Space Representation

Goal:-
 The goal is the objective to be
achieved, a final answer, or a solution.
 For example, your goal is finding what is
wrong with your car.
 There may be more than one goal.
State-Space Representation

Operators:-
 Operators are procedures used for changing
from one state to another.
 An operator describes a process whereby
some action is taken to change initial state
into another state that more closely
approaches the goal.
 Operators move the problem from one state
to the next, following the guidance of a
master control strategy, until a goal is
reached.
State-Space Representation

 An operator could be an algorithmic subroutine.

 Following figure shows the relationship between the
initial state, procedures, and goal:

Initial State (s) Procedures Goal (s)

Control
Strategy

Figure:
Relationship between Initial state (s), Procedures, and Goal (s)
State-Space Representation

The initial conditions provide the states that are

manipulated by procedures to achieve the goal.

 Control Strategy:-
A control strategy selects or guides the procedures.

State Space:
A state space represents the set of all
attainable states for a given problem.
Graphic Presentations

Directed State Graph :-

The following state graph for a simple
problem- attempting to find the best path
from one city, the source (S), to another
city, the destination, or goal (G).
The problem is to reach the destination in
the least amount of time , or using the
shortest route.
Graphic Presentations
H

A
6 5
E
4 4 7
Start
S 3 3B 9 G

5 8 goal
C F
6
2
D
Search Tree
 A search tree based on the state graph
in Figure.
 The tree diagram states the same
problem but in slightly different format
 The network thus formed is more like
a hierarchy.
 Some of the nodes are repeated to
eliminate the cyclical loop problem
described earlier.
Search Tree
Root Node Level 0 (Root)
4
S 5
3

A B
6 9 4 2
C 8

E G A D F
5 7 6 6

H G E
5 7
G
Figure: Search Tree
H G
Search Tree

 The initial node (root node)

describes the object or (topic).
 The other nodes branch out from the
root, is called the Successor or
descendents, or sometimes referred to
as children.
 They are intermediate nodes.
Search Tree
 Working backwards through the tree, nodes
are said to have predecessors, ancestors, or
parents.
 Nodes with no children or successors are
called leaf nodes which designate the end
of search, either by arriving at a goal or by
being at a dead end.
 The interconnecting arcs are referred to as
branches.
Search Tree
 AND/OR Branches:- are another aspect
of search trees.
 The branches from a node to its successors
can represent two or more alternative paths
to sub goals.
 One path or another path could lead to the
goal is called OR nodes because one branch,
OR anther, OR another could be the path to
the goal.
Search Tree
 In some problems, however, the successor,
nodes might represent problem states that
must all be achieved or traversed before the
goal is reached are referred to as AND
nodes.
 One sub goal AND another sub goal (AND
possible others) must be achieved to solve
the problem.
 A search tree is also known as problem
reduction representation because it can be
used to divide a problem into sub problems.
Blind Search Methods

Figure: Blind Search Methods

Blind Search Methods

 A blind search is collection of

procedures used to arbitrarily to
search a state space.
 Blind search methods can be
classified as :
1. Exhaustive search
2. Partial search
Exhaustive Search Method

 In an exhaustive search operators are used

to generate successor states.
 Beginning at the root node, the search
continues until a solution is found.
 The idea behind an exhaustive search is to
examine the entire tree in an orderly manner,
using all the operators and generating as
many successor nodes as possible to find the
desired solution.
 Starting with the root node, several procedures are
possible for proceeding through the tree; but the
approaches are usually inefficient.
Exhaustive Search Method

 In very large problems, a huge number of

new states are generated and many
alternatives are considered.
 As a result, it takes a considerable amount of
time and effort to find the solution.
 Very-high-speed computers make blind search
acceptable for some problems; however, others are
too large for an exhaustive search.
 Consider the possible number of moves in a chess
game-estimated to be 10 to the 120th power.
 For such cases, a heuristic search is more appropriate.
Partial Search Methods

There are two partial search

methods:

1. Breadth-First Search
2. Depth-First Search
Breadth-First Search

 A breadth-first search examines

all of the nodes (states) in a
search tree, beginning with the
root node.
 The nodes in each level are
examined completely before
moving on to the next level.
 A simple breadth search is illustrated
in Figure.
Breadth-First Search
Root Node
(start) level 0

level
1 2 3 4

5 6 7 8
9 10

Figure Breadth first search Goal (end) level

2
 2
Breadth-First Search

 The numbers inside the node

circles designate the sequence in
which the nodes are examined.
 The process usually starts at the
initial state node and works
downward in the tree from left to
right.
Breadth-First Search

 A terminal node is not necessarily a

goal node; it can be a dead-end node.
 Breadth-first procedures are good
when the number of paths emanating
from each goal is relatively small and
where the number of levels in each
branch is of a different depth (number
of levels).
Depth-First Search
 A depth-first search begins at the root
node and works downward to
successively deeper levels.
 An operator is applied to the node to
generate the next deeper node in
sequence.
 This process continues until a solution
is found or backtracking is forced by
reaching a dead end. A simple depth first
search is illustrated in Figure.
Depth-First Search

1 Root Node (start)

2 11
8

3 5 9
12 14

4 6 13 15 16
7
10
Figure depth first search
Depth-First Search
 Again, the number inside the nodes
designate the sequence of nodes
generated or searched.
 This process seeks the deepest
possible nodes.
 If a goal state is not reached in this
way, the search process backtracks to
the next highest level node where
addition paths are available to follow.
Depth-First Search
 This process continues downward and
in a left-to-right direction until the
state goal is discovered.
 Here, the search would actually end at
node 13.
 When a dead-end node is discovered,
such as node 4 in the above figure, the
search process backtracks so that any
additional branching alternative at the
next higher node level is attempted.
 The search backs up to node 3.
Depth-First Search
 It has no alternate paths, so the
search backtracks to node 2.
 Here, another path through node
5 is available.
 The path through node 6 is
explored until its depth is
exhausted.
 The backtracking continues until
the goal is reached.
Depth-First Search
 The depth-first search guarantees a solution,
but the search may be a long one.
 Many different branches will have to be
considered to a maximum depth before a
solution is reached.
 (By setting a "depth bound," it is frequently
possible to reduce the search.)
 The method is especially attractive in cases
where short paths exist and where there are
no lengthy sub-branches.
Heuristic Methods
 Heuristic search methods are designed to
reduce the amount of search for a solution.
 When a problem is presented as a search
tree, the heuristic approach attempts to
reduce the size of the tree by pruning
nonvital or nonpromising nodes.
 For example, when the Coast Guard searches
for a missing person at sea, they don’t check
the entire ocean.
BENEFITS OF HEURISTICS

1. It has an inherent flexibility- which allows them to be used on

ill-structured and complex problems.
2. Although an exact (Blind Search) solution procedure may
exist, it may be computationally prohibitive to use, or
unrealistic in its data. In such cases, heuristic methods are
better.
3. It is simpler for decision maker to understand by design
which is supported by qualitative analysis.
4. It may be used as part of an iterative procedure that
guarantees the finding of an optimal solution.
Disadvantages and limitations

 The inherent flexibility of heuristic methods can

lead to misleading or even fraudulent
manipulations and solutions.
 Certain heuristics may contradict others that are
applied to the same problem, which generates
confusion and lack of trust in heuristic methods.
 Optimal solutions are not identified.
 Heuristics are not as general as algorithms;
therefore, they can normally be used only for the
specific situations intended.
Disadvantages and limitations

 Enumeration heuristics that consider all

possible combinations in practical problems
can seldom be achieved.
 Sequential decision choices can fail to
anticipate future consequences of each
choice.
 Interdependencies in one part of the
system can sometimes have a profound
influence on the whole system.
Representative Approaches

 Generate and Test

 The basic idea is to generate (via
rules) possible candidate solutions and
devise a test to determine if the
solutions are indeed good.
 An example is opening a combination
lock without knowing the combination.
Generate and Test

One way to execute this search is to follow a five-step

procedure (suggested by Wolfgram et al.):

1. Add a specification criterion (e.g., a known symptom,

such as it is common to open a combination lock by
turning it first to the left).
2. Try a path that satisfies the specification.
3. Determine whether the path is plausible; "prune" that
path if it is not plausible.
4. Move to the next path.
5. Check to see if all specifications have been mentioned
6. The generate and test method is effective only if
pruning can be done early in the search so many paths
can be eliminated.
Hill Climbing
 The hill-combing method is
similar to a depth-first blind
search.
 However, paths are not selected
arbitrarily, but in relationship to
their proximity to the desired
goal.
 The search tree in the following
figure illustrates the concept:
 .
Hill Climbing

I A III

II
B D
C
8 13 11

4 3 7 5
E G H
F

2 I 1 J
Figure: Hill climbing
Hill Climbing
 Each production process I, II, and
III can continue for several
stages, in which several states
(nodes) exist.
 The numbers above the nodes
designate the potential number of
defects in a specific product at a
certain state in the process
Hill Climbing
 The more stages we go through, the fewer
defects we will find.
 The target is to find the method with the
minimum number of defects.
 A depth-first search goes to I, then to II, and
then to III.
 If the desired number of defects is two, then
the process is speedy.
 But, if the goal is one defect, then the entire
tree will have to be visited.
 In the hill-climbing method, nodes B, C, and
D are compared, and a search starts in
branch I (the lowest number of defects-8).
Hill Climbing
 Since one defect was not discovered,
backtracking is done to branch III.
 (III is done before II since there are fewer
defects on D than on C.)
 The search goes to D and H; backtracking is
then exercised and the D-G-J path leads to
the desired solution.
 Path A-C-F was not visited.
 Thus, hill climbing is faster than a depth-first
search.
 Variations of this method exist, and the
saving over a blind search can be substantial
if large trees are involved.
Induction
 Induction means to generalize from a smaller
(or simpler) version of the same problem.
 Two features of induction are essential.
 First, in the problem statement the problem
must be modeled in terms of the associated
(or predicted) data.
 Second, the induced result must be tested
against real examples to verify its
reasonableness.
Construction
o The input for methods based on the construction
strategy is the data that defines a specific instance of
the problem.
o A solution is built up one component at a time.
o A construction strategy begins by examining this data
and attempting to identify an element that is likely to
be a valuable part of a good final solution.
o Next, successive additional elements of a solution are
added.
o Once the final solution has been built up, it may be
obvious that improvement can be made.
o Therefore, the hill-climbing procedure is often applied
to the output of the construction method.
Component Analysis Strategy

o Some problems are so large or so

complicated that the only practical approach
is to break them into manageable portions.
o These portions are then dealt with
independently by heuristics and/or
algorithms.
o The solutions for the portions are then
combined.
o Of course, it may be extremely difficult to
piece the solutions to the different
components into an acceptable plan.
o This approach is also called the
decomposition method.
Best First
 In the best-first approach, which is based on
some heuristic evaluation function, you
select the next move by searching for the
best available solution that you can move to
in one step, no matter where it is located on
the tree.
 For example, you are about to cross a stream
and there are many rocks on your way.
 First you go the most promising one, then
you look around and make the best progress
you can.
Best First
 If you evaluate only one jump at a
time, you are using the best-first
approach.
 This procedure is often labeled
"greedy," since it is shortsighted.
 It searches only one step at a time
instead of looking at the entire set of
steps.
 An implementation of the best-first
approach is called the A* algorithm.
Other Approaches
 Several other heuristic approaches can be
useful, especially when numeric analysis is
involved.

Control Strategies
 Once a search method is selected it needs to be activated.
 That is, it is necessary to decide which operators to apply
and when to apply them.
 In a rule-based system, for example, we must decide which
rule to check next.
 Four basic control strategies are common: forward chaining,
backward chaining, means-end, and least commitment.
Control Strategies
 Forward Chaining
 Backward Chaining
 Means-End Analysis
 Least Commitment
Forward Chaining

 The forward chaining method

emulates human deductive
reasoning.
 It is a data-driven process that
starts when certain information is
provided by the user.
 Clues are collected as we move
toward a conclusion.
Backward Chaining
 Backward chaining is a goal-driven
search strategy.
 It begins with the goal and works
backward towards the initial
conditions.
 The process starts with a hypothesis; a
search is then launched to find and
verify the necessary supporting facts.
 The process ends with the acceptance
or rejection of the hypothesis.
Means-End Analysis
 Means-end analysis is an iterative process of
subdividing the difference between the
current state and the goal state until the
difference is eliminated.
 The solution shows the means by which to
traverse the knowledge base.
 The method applies a set of operators that
assist in moving us toward the goal.
 The method attempts to achieve the
difference reduction in the most efficient
manner; that is, by selecting the appropriate
operators, in the proper sequence.
Means-End Analysis
 For example, you live in a ranch near
Austin, Tex., and you would like to see
the Statue of Liberty in New York.
 You have a short vacation.
 The distance between Austin and New
York is about 1,800 miles (the
"difference").
 The operator that is most effective in
reducing this difference is to "fly."
Means-End Analysis
 To activate this operator you need to
reach an airport which is 23 miles
away.
 Now you need a means to satisfy a
subgoal of getting to the airport.
 For that you need another operator
(e.g., drive or be driven in a cab).
 In either case, you have to get a car;
similarly, several more operators will
be needed to help reach your goal.
Least Commitment
 According to the least commitment control
strategy, we assume that no decision should
be made until there is enough information.
 To activate this approach, we need to know
what is "enough information."
 Also, we need to know what to do when
there is not enough information (e.g., how to
get additional information, or when to make
a guess).
 This control strategy could be combined with
neural computing to increase the reliability of
the "guesses."
Least Commitment
 Control strategies are implemented by
a control program called the inference
engine.
 The program determines how and in
what sequence the state space, which
is the knowledge base, is searched.
 The inference engine and the
knowledge base are the two major
components of the most widely used
AI technology-expert systems.