Artificial Intelligence

Intelligent Agents
Outline
 Agents and environments
 Rationality
 PEAS (Performance measure,
Environment, Actuators, Sensors)
 Environment types
 Agent types
Agents
An agent is anything that can be viewed as
perceiving its environment through sensors and
acting upon that environment through actuators.
Human agent has, for example:
 eyes, ears, and other organs as sensors;
 hands, legs, mouth, and other body parts as
actuators.
Robotic agent has, for example:
 cameras and infrared range finders as sensors;
 various motors as actuators.
Agents
 A software agent receives keystrokes, file
contents, and network packets as sensory
inputs and acts on the environment by
displaying on the screen, writing files, and
sending network packets.
Agent and Environment
The Vacuum-Cleaner Mini-World

 Environment: square A and B

 Percepts: location and status, e.g., [A, Dirty]
 Actions: left, right, suck, and no-op
The Vacuum-Cleaner Mini-World

Percept Action
sequences
[A,Clean] Right
[A, Dirty] Suck
[B, Clean] Left
[B, Dirty] Suck
[A, Dirty], [A, Clean] Right
[A, Clean], [B, Dirty] Suck
[A, Clean], [B, Clean] No-op
... ...
Agent Function
 The agent function maps from percept histories to
actions:
[f: P*  A]
 An agent is completely specified by the agent
function mapping percept sequences to actions
 The agent program runs on the physical
architecture to produce f.
 agent = architecture + program
The Vacuum-Cleaner Mini-World

function REFLEX-VACUUM-AGENT ([location, status]) return an

action
if status == Dirty then return Suck
else if location == A then return Right
else if location == B then return Left
Does not work this way. Need full state space (table) or memory.
The Vacuum-Cleaner Mini-World

World State Action

[A, Clean] Right

[A, Dirty] Suck
[B, Clean] Left
[B, Dirty] Suck
[A, Dirty], [A, Clean] Right
[A, Clean], [B, Dirty] Suck
[B, Dirty], [B, Clean] Left
[B, Clean], [A, Dirty] Suck
[A, Clean], [B, Clean] No-op
[B, Clean], [A, Clean] No-op
Good Behavior: The Concept Of Rationality

 Rational Agent: A rational agent is said to

perform the right things.
 For each possible percept sequence, a
rational agent should select an action that is
expected to maximize its performance
measure, given the evidence provided by the
percept sequence and whatever built-in
knowledge the agent has.
Rationality
 Rationality  omniscience
 An omniscient agent knows the actual outcome
of its actions.
 Rationality  perfection
 Rationality maximizes expected performance,
while perfection maximizes actual performance.
 "Ideal Rational Agent": Always does "the
right thing".
Rationality
 The proposed definition requires:
 Information gathering/exploration
 To maximize future rewards
 Learn from percepts
 Extending prior knowledge
 Agent autonomy
 Compensate for incorrect prior knowledge
Rationality
 The rationality of an agent is measured by its
performance measure.
 Rationality can be judged on the basis of
following points:
 Performance measure,
 Prior environment knowledge,
 Actions,
 Percept sequence to date (sensors).
The Nature of Environments
Task Environment
 To design a rational agent we must first
specify its task environment.
 PEAS description of the task environment:
 Performance
 Environment
 Actuators
 Sensors
Task Environment - Example
For example, a fully automated taxi driver:
PEAS description of the environment:
 Performance

 Safety, destination, profits, legality, comfort

 Environment
 Streets/freeways, other traffic, pedestrians, weather,, …
 Actuators
 Steering, accelerating, brake, horn, speaker/display,…
 Sensors
 cameras, sonar, speedometer, engine sensors,
keyboard, GPS, …
Examples of Agents (Norvig)
Specifying the task environment
PEAS
Agent: Medical diagnosis system
 Performance measure: Healthy patient, minimize

costs, lawsuits
 Environment: Patient, hospital, staff

 Actuators: Display questions, tests, diagnoses,

treatments, referrals.
 Sensors: Keyboard entry of symptoms, findings,

patient's answers
PEAS
Agent: Part-picking robot
 Performance measure: Percentage of parts in

correct bins
 Environment: Conveyor belt with parts, bins

 Actuators: Jointed arm and hand

 Sensors: Camera, joint angle sensors

PEAS
Agent: Interactive English Tutor
 Performance measure: Maximize student's score

on test
 Environment: Set of students, Testing Agency

 Actuators: Display exercises, suggestions,

corrections
 Sensors: Keyboard Entry
Classification of Environment Types
 Fully observable (vs. partially observable): An agent's
sensors give it access to the complete state of the
environment at each point in time.
 for example, a vacuum agent with only a local dirt sensor cannot tell
whether there is dirt in other squares, and an automated taxi cannot see
what other drivers are thinking.
 Deterministic (vs. stochastic): The next state of the
environment is completely determined by the current state
and the action executed by the agent.
 Example-The vacuum world is deterministic
 Episodic (vs. sequential): The agent's experience is divided
into atomic "episodes" (each episode consists of the agent
perceiving and then performing a single action), and the
choice of action in each episode depends only on the
episode itself.
 Ex-Chess and taxi driving are sequential
 For example, an agent that has to spot defective parts
 Properties of task environments
Environment types
 Static (vs. dynamic): The environment is unchanged while
an agent is deliberating.
 For example Taxi driving is dynamic.
 Crossword puzzles are static.

 Discrete (vs. continuous): A limited number of distinct,

clearly defined percepts and actions.
 For example ,Chess also has a discrete set of percepts and actions
 Taxi driving is a continuous-state

 Single agent (vs. multiagent): An agent operating by itself

in an environment.
 For example, an agent solving a crossword puzzle by itself is clearly in
a single-agent environment,
 whereas an agent playing chess is in a two agent environment.
Task Environments (Norvig)
Agents design depends on task environment:
 deterministic vs. stochastic
 assembly line vs. weather
 episodic vs. Sequential
 assembly line vs. diagnostic repair robot, Flakey
 static vs. dynamic
 Crossword puzzles vs. Taxi driving
 discrete vs. continuous
 chess game vs. autonomous vehicle
 single vs. multi agent
 solitaire game vs. soccer, taxi driver
 fully observable vs. partially observable
 video camera vs. infrared camera, vacuum agent
Environment types
Solitaire Backgammon Internet shopping Taxi

Observable?? FULL FULL PARTIAL PARTIAL

Deterministic?? YES NO YES NO

Episodic?? NO NO NO NO

Static?? YES YES SEMI NO

Discrete?? YES YES YES NO

Single-agent?? YES NO NO NO
Examples of Environment Types

Chess with Chess w.o. Taxi

clock clock driving
Fully observable Yes Yes No
Deterministic Strategic Strategic No
Episodic No No No
Static Semi Yes No
Discrete Yes Yes No
Single agent No No No

The real world is (of course) partially observable,

stochastic, sequential, dynamic, continuous, multi-agent.
Environment types
 The simplest environment is
 Fully observable, deterministic, episodic, static,
discrete and single-agent.
 Most real situations are:
 Partially observable, stochastic, sequential,
dynamic, continuous and multi-agent.
 The Structure of Agents
Agent Types
 Agent function, Agent program
 Architecture- Agent program will run on some
sort of computing device with physical sensors
and actuators
 How does the inside of the agent work?
 Agent = architecture + program

 All agents have the same skeleton:

 Input = current percepts

 Output = action

 Program= manipulates input to produce output

 Note difference with agent function.

Agent Programs

Function TABLE-DRIVEN_AGENT(percept) returns an

action

static: percepts, a sequence initially empty

table, a table of actions, indexed by percept
sequence
append percept to the end of percepts
action  LOOKUP(percepts, table)
return
The action
TABLE-DRIVEN-AGENT program is invoked for each
new percept and returns an action each time. It retains the
complete percept sequence in memory.
Agent types
 Four basic kinds of agent programs will be
discussed:
 Simple reflex agents
 Model-based reflex agents
 Goal-based agents
 Utility-based agents
 All these can be turned into learning
agents.
Simple Reflex Agents
 Select action on the
basis of only the
current percept.
 E.g. the vacuum-agent
 These agents only
succeed in the fully
observable
environment.
 Implemented through
condition-action rules
 If dirty then suck
Simple Reflex Agents
 The Simple reflex agents are the simplest agents.
 These agents take decisions on the basis of the current
percepts and ignore the rest of the percept history.
 These agents only succeed in the fully observable
environment.
 The Simple reflex agent does not consider any part of
percepts history during their decision and action
process.
 The Simple reflex agent works on Condition-action rule,
which means it maps the current state to action.
 For Examplea Room Cleaner agent, it works only if
there is dirt in the room.
Simple Reflex Agents
function SIMPLE-REFLEX-AGENT(percept)
returns an action
static: rules, a set of condition-action rules
state  INTERPRET-INPUT(percept)
rule  RULE-MATCH(state, rules)
action  RULE-ACTION[rule]
return action

Will only work if the environment is fully observable.

Otherwise infinite loops may occur.
The Vacuum-Cleaner Mini-World

function REFLEX-VACUUM-AGENT ([location, status]) return an

action
if status == Dirty then return Suck
else if location == A then return Right
else if location == B then return Left
Model/State-based Agents
 To tackle partially
observable environments.
 Maintain internal state
 Over time update state
using world knowledge
 How does the world
change.
 How do actions affect
world.
 Model of World
Model/State-based Agents
 The Model-based agent can work in a partially observable
environment, and track the situation.
 A model-based agent has two important factors:
 Model: It is knowledge about "how things happen in the

world," so it is called a Model-based agent.

 Internal State: It is a representation of the current state

based on percept history.

 These agents have the model, "which is knowledge of the
world" and based on the model they perform actions.
 Updating the agent state requires information about:
 How the world evolves

 How the agent's action affects the world.

Model/State-based Agents

function REFLEX-AGENT-WITH-STATE(percept)
returns an action
static: rules, a set of condition-action rules
state, a description of the current world
state
(action, the most recent action)
state  UPDATE-STATE(state, (action,) percept)
rule  RULE-MATCH(state, rule)
action  RULE-ACTION[rule]
return action
Goal-based Agents
 The agent needs a goal to
know which situations are
desirable.
 Things become difficult when long
sequences of actions are required
to reach the goal.
 Typically investigated in search
and planning research.
 Major difference: future is taken
into account.
 Is more flexible since
knowledge is represented
explicitly - to a certain degree -
and can be manipulated.
Utility-based Agents
 Certain goals can be reached
in different ways.
 Some are better, have a
higher utility.
 Utility function maps a
(sequence of) state(s) onto a
real number.
 Improvement on goal setting:
 Selecting between conflicting
goals.
 Select appropriately between
several goals based on
likelihood of success.
Utility-based Agents
 Utility-based agent act based not only goals
but also the best way to achieve the goal.
 The Utility-based agent is useful when
there are multiple possible alternatives, and
an agent has to choose in order to perform
the best action.
 The utility function maps each state to a
real number to check how efficiently each
action achieves the goals.
Learning Agents
 A learning agent in AI is the
type of agent which can
learn from its past
experiences, or it has
learning capabilities.

 It starts to act with basic

knowledge and then able to
act and adapt automatically
through learning.
Learning Agents  Learning element: It is
responsible for making
improvements by learning from
environment.
 performance element: it is
responsible for selecting external
actions.
 Critic :The learning element uses
feedback from the critic on how
the agent is doing and determines
how the performance element
should be modified to do better in
the future.
 Problem generator: This
component is responsible for
suggesting actions that will lead to
new and informative experiences.
Robotic Sensors
 (digital) camera
 infrared sensor
 range finders, e.g. radar, sonar
 GPS
 tactile (whiskers, bump panels)
 proprioceptive sensors, e.g. shaft
decoders
 force sensors
 torque sensors
Robotic Effectors
 ‘limbs’ connected through joints;
 degrees of freedom = #directions in
which limb can move (incl. rotation axis)
 drives: wheels (land), propellers, turbines
(air, water)
 driven through electric motors, pneumatic
(gas), or hydraulic (fluids) actuation
 statically stable, dynamically stable