Summary of Previous Lecture

• What is an Intelligent agent?

• Agents & Environments
• Performance measure
• Environment
• Actuators
• Sensors
• Features of intelligent agents
 Environments
• Actions are done by the agent on the environment.
• Environment provides percepts to the agent.
• Determine to a large degree the interaction between
the “outside world” and the agent
– the “outside world” is not necessarily the “real world” as
we perceive it
• it may be a real or virtual environment the agent lives in
• In many cases, environments are implemented within
computers
– They may or may not have a close correspondence to the
“real world”
 Properties of environments
• Fully observable vs. partially observable
• Or Accessible vs. inaccessible
– If an agent’s sensory equipment gives it access to the
complete state of the environment, then we say that
environment is fully observable to the agent.
– An environment is effectively fully observable if the
sensors detect all aspects that are relevant to the choice of
action.
– A fully observable environment is convenient because the
agent need not maintain any internal state to keep track of
the world.
 Properties of environments
• Deterministic vs. nondeterministic.
– If the next state of the environment is completely
determined by the current state and the actions
selected by the agents, then we say the
environment is deterministic.
– If the environment is inaccessible, then it may
appear to be nondeterministic (bunch of
uncertainties).
 Properties of task environments
• Episodic vs. sequential.
– Agent’s experience is divided into “episodes.”
• Each episode consists of the agent perceiving and
acting.
– Subsequent episodes do not depend on what
actions occur in previous episodes.
– In sequential environments current actions affect
all succeeding actions
 Properties of task environments
• Static vs. Dynamic
– If the environment can change while an agent is
performing action, then we say the environment is
dynamic.
– Else its static.
– Static environments are easy to deal with, because the
agent does not keep on looking at the environment
while it is deciding on an action.
– Semidynamic: if the environment does not change
with the passage of time but the agent performance
score does.
 Properties of environments
• Discrete vs. continuous
– If there are a limited number of distinct, clearly
defined percepts and actions, we say that the
environment is discrete.
• Chess, since there are a fixed number of possible
moves on each turn.
• Taxi driving is continuous.
 Properties of environments
• Single agent vs. Multiagent
– In the single agent environment there is only one
agent
• A computer software playing crossword puzzle
– In multiagent systems, there are more than one
active agents
• Video games
 Environment Examples
Environment Obser Determi Episodic Static Discrete Agents
vable nistic
Chess with a clock Fully Strategic Sequential Semi Discrete Multi

Chess without a clock Fully Strategic Sequential Static Discrete Multi

Poker Partial Strategic Sequential Static Discrete Multi

Backgammon Fully Stochast Sequential Static Discrete Multi

ic
Fully observable vs. Taxi driving Partial Stochast Sequential Dyna Continu Multi
ic mic ous
partially observable
Medical diagnosis Partial Stochast Episodic Static Continu Single
Deterministic vs. ic ous
stochastic / strategic Image analysis Fully Determi Episodic Semi Discrete Single
Episodic vs. sequential nistic
Static vs. dynamic Robot part picking Fully Determi Episodic Semi Discrete Single
nistic
Discrete vs. continuous
Interactive English Partial Stochast Sequential Dyna Discrete Multi
Single agent vs. tutor ic mic
multiagent
 Agent types

• Four basic types in order of increasing

generalization:
– Simple reflex agents
– Reflex agents with state/model
– Goal-based agents
– Utility-based agents
 Simple Reflex Agent
 Instead of specifying individual mappings in an explicit table,
common input-output associations are recorded
 Requires processing of percepts to achieve some
abstraction
 Frequent method of specification is through
condition-action rules
 if percept then action
 If car-in-front-is-braking then initiate-braking

 Similar to innate reflexes or learned responses in

humans
 Efficient implementation, but limited power
 Environment must be fully observable
 Easily runs into infinite loops
Simple reflex agents
 Simple Reflex Agent
• function SIMPLE-REFLEX-AGENT (percept)
returns action
– static: rules, a set of condition-action rules

– state ← INTERPRET-INPUT (percept)

– rule ← RULE-MATCH (state, rules)
– action ← RULE-ACTION [rule]
– return action
 A simple reflex agent..
• which works by finding a rule whose condition
matches the current situation and then doing
the action associated with that rule
 Reflex agents with state/model
• Evan a little bit of un observability can cause
serious trouble.
– The braking rule given earlier assumes that the
condition car-in-front-is-braking can be
determined from the current percept – the
current video image.
• More advanced techniques would require the
maintenance of some kind of internal state to
choose an action.
 Reflex agents with state/model
 Aninternal state maintains important information from
previous percepts
 Sensors only provide a partial picture of the environment
 Helps with some partially observable environments
 Theinternal states reflects the agent’s knowledge about
the world
 This knowledge is called a model
 May contain information about changes in the world
 Model-based reflex agents
• Required information:
– How the world evolves independently of the
agent?
• An overtaking car generally will be closer behind than it
was a moment ago.
• The current percept is combined with the old internal
state to generate the updated description of the
current state.
Model-based reflex agents
 Model-based reflex agents
• function REFLEX-AGENT-WITH-STATE (percept)
returns an action
– static: state, a description of the current world state
rules, a set of condition-action rules
action, the most recent action, initially none

– state ← UPDATE-STATE (state, action, percept)

– rule ← RULE-MATCH (state, rules)
– action ← RULE-ACTION [rule]
– state ← UPDATE-STATE (state, action)
– return action
 Goal-based agent
• Merely knowing about the current state of the
environment is not always enough to decide
what to do next.
• The right decision depends on where the taxi
is trying to get to.
• So the goal information is also needed.
 Goal-based agent
• Goal-based agents are far more flexible.
– If it starts to rain, the agent adjusts itself to the
changed circumstances, since it also looks at the
way its actions would affect its goals (remember
doing the right thing).
– For the reflex agent we would have to rewrite a
large number of condition-action rules.
Goal-based agents
 Utility-based agents
• Goals are not really enough to generate high-
quality behavior.
• There are many ways to reach the destination,
but some are qualitatively better than others.
– More safe
– Shorter
– Less expensive
 Utility-based agent
• We say that if one world state is preferred to
another, then it has higher utility for the
agent.
• Utility is a function that maps a state onto a
real number.
– state → R
• Any rational agent possesses a utility function.
Utility-based agents
 Problem-Solving Agent
• In which we look at how an agent can decide
what to do by systematically considering the
outcomes of various sequences of actions that
it might take.
- Stuart Russell & Peter Norvig
 Problem solving agent
• A kind of Goal-based agent.
• Decide what to do by searching sequences of
actions that lead to desirable states.

27
 Problem Definition
• Initial state : starting point
• Operator: description of an action
• State space: all states reachable from the initial state
by any sequence action
• Path: sequence of actions leading from one state to
another
• Goal test: which the agent can apply to a single state
description to determine if it is a goal state
• Path cost function: assign a cost to a path which the
sum of the costs of the individual actions along the
path.
 What is Search?
• Search is the systematic examination of states to find
path from the start/root state to the goal state.
• The set of possible states, together
with operators defining their connectivity the search
space.
• The output of a search algorithm is a solution, that is, a
path from the initial state to a state that satisfies the goal
test.
• In real life search usually results from a lack of
knowledge. In AI too search is merely a offensive
instrument with which to attack problems that we can't
seem to solve any better way.
 Search groups
Search techniques fall into three groups:

1. Methods which find any start - goal path,

2. Methods which find the best path,
3. Search methods in the face of opponent.
 Search
• An agent with several immediate options of
unknown value can decide what to do by first
examining different possible sequences of
actions that lead to states of known value, and
then choosing the best one. This process is
called search.
• A search-algorithm takes a problem as input
and returns a solution in the form of an action
sequence.
 Problem formulation
• What are the possible states of the world
relevant for solving the problem?
• What information is accessible to the agent?
• How can the agent progress from state to
state?
• Follows goal-formulation.
• Courtesy: Dr. Franz J. Kurfess
 Well-defined problems and solutions
• A problem is a collection of information that the agent will use
to decide what to do.
• Information needed to define a problem:
– The initial state that the agent knows itself to be in.
– The set of possible actions available to the agent.
• Operator denotes the description of an action in terms
of which state will be reached by carrying out the
action in a particular state.
• Also called Successor function S. Given a particular
state x, S (x) returns the set of states reachable from x by
any single action.
 State space and
a path
• State space is the set of all states reachable
from the initial state by any sequence of
actions.
• Path in the state space is simply any sequence
of actions leading from one state to another.
 Search Space Definitions
• Problem formulation
– Describe a general problem as a search problem
• Solution
– Sequence of actions that transitions the world from the initial
state to a goal state
• Solution cost (additive)
– Sum of the cost of operators
– Alternative: sum of distances, number of steps, etc.
• Search
– Process of looking for a solution
– Search algorithm takes problem as input and returns solution
– We are searching through a space of possible states
• Execution
– Process of executing sequence of actions (solution)
 Goal-formulation
• What is the goal state?
• What are important characteristics of the goal
state?
• How does the agent know that it has reached
the goal?
• Are there several possible goal states?
– Are they equal or are some more preferable?
• Courtesy: Dr. Franz J. Kurfess
 Goal
• We will consider a goal to be a set of world
states – just those states in which the goal is
satisfied.
• Actions can be viewed as causing transitions
between world states.
 Looking for Parking
• Going home; need to find street parking
• Formulate Goal:
Car is parked
• Formulate Problem:
States: street with parking and car at that street
Actions: drive between street segments
• Find solution:
Sequence of street segments, ending with a street
with parking
 Example Problem

Start Street

Street with
Parking
Search Example
Formulate goal: Be in
Bucharest.

Formulate problem: states

are cities, operators drive
between pairs of cities

Find solution: Find a

sequence of cities (e.g., Arad,
Sibiu, Fagaras, Bucharest)
that leads from the current
state to a state meeting the
goal condition
 Problem Formulation

A search problem is defined by the

1.Initial state (e.g., Arad)

2.Operators (e.g., Arad -> Zerind, Arad -> Sibiu, etc.)
3.Goal test (e.g., at Bucharest)
4.Solution cost (e.g., path cost)
Examples (2) Vacuum World

• 8 possible world states

• 3 possible actions:
Left/Right/ Suck
• Goal: clean up all the
dirt= state(7) or
state(8)
 Vacuum World
S1 S2

S3 S6 S5 S4

S7 S8

•States: S1 , S2 , S3 , S4 , S5 , S6 , S7 , S8
• Operators: Go Left , Go Right , Suck
• Goal test: no dirt left in both squares
• Path Cost: each action costs 1.
Example Problems – Eight Puzzle
States: tile locations

Initial state: one specific tile configuration

Operators: move blank tile left, right, up, or

down

Goal: tiles are numbered from one to eight

around the square

Path cost: cost of 1 per move (solution cost

same as number of most or path length)

Eight Puzzle
http://mypuzzle.org/sliding
 Single-State problem and
Multiple-States problem
• World is accessible  agent’s sensors give
enough information about which state it is in
(so, it knows what each of its action does),
then it calculate exactly which state it will be
after any sequence of actions. Single-State
problem
• world is inaccessible  agent has limited
access to the world state, so it may have no
sensors at all. It knows only that initial state is
one of the set {1,2,3,4,5,6,7,8}. Multiple-
States problem
Think of the graph defined as follows:
– the nodes denote descriptions of a state of the world, e.g.,
which blocks are on top of what in a blocks scene, and
where the links represent actions that change from one
state to the other.
– A path through such a graph (from a start node to a goal
node) is a "plan of action" to achieve some desired goal
state from some known starting state. It is this type of
graph that is of more general interest in AI.
 Searching for Solutions
Visualize Search Space as a Tree
• States are nodes
• Actions are edges
• Initial state is root
• Solution is path
from root to goal
node
• Edges sometimes
have associated
costs
• States resulting
from operator are
children
 Directed graphs
• A graph is also a set of nodes connected
by links but where loops are allowed and
a node can have multiple parents.
• We have two kinds of graphs to deal with:
directed graphs, where the links have
direction (one-way streets).
 Undirected graphs
• undirected graphs where the links go
both ways. You can think of an undirected
graph as shorthand for a graph with
directed links going each way between
connected nodes.
 Searching for solutions:
Graphs or trees
• The map of all paths within a state-space is
a graph of nodes which are connected by links.
• Now if we trace out all possible paths through the graph,
and terminate paths before they return to nodes already
visited on that path, we produce a search tree.
• Like graphs, trees have nodes, but they are linked
by branches.
• The start node is called the root and nodes at the other
ends are leaves.
• Nodes have generations of descendents.
• The aim of search is not to produce complete physical trees
in memory, but rather explore as little of the virtual tree
looking for root-goal paths.
Search Problem Example (as a tree)
(start: Arad, goal: Bucharest.)
 Summery of Today’s Lecture
• Problem solving by searching
• What is Search?
• Problem formulation
• Search Space Definitions
• Goal-formulation
• Examples
• Searching for Solutions Visualize Search Space as a
Graphs
• Directed graphs and Undirected graphs