Professional Documents
Culture Documents
Lecture 2 1
Habeebah A. Kakudi Ph.D.
11/25/21
qIntelligent Agents (Lecture
2,3)
üAgents and environment
üGood behavior
ü The concept of rationality
ü The nature of environments
ü Structure of agents
11/25/21 2
CHAPTER 2
INTELLIGENT AGENTS
What is an agent ?
l An agent is anything that perceiving its environment
through sensors and acting upon that environment through
actuators
l Example:
l Human is an agent
l A robot is also an agent with cameras and motors
l A thermostat detecting room temperature.
11/25/21 3
What AI should fill 11/25/21 4
Percept
l Agent’s perceptual inputs at any given instant
Percept sequence
l Complete history of everything that the agent has
ever perceived.
11/25/21 5
AGENT FUNCTION & PROGRAM
Agent’s behavior is mathematically
described by
l Agent function
l A function mapping any given percept sequence
to an action
Practically it is described by
l An agent program
l The real implementation
11/25/21 6
VACUUM-CLEANER WORLD
Perception: Clean or Dirty? where it is in?
Actions: Move left, Move right, suck, do nothing
11/25/21 7
11/25/21 8
Function Reflex-Vacuum-
Agent([location,status]) return an action
If status = Dirty then return Suck
else if location = A then return Right
else if location = B then return left
11/25/21 9
CONCEPT OF RATIONALITY
Rational agent
l One that does the right thing
l = every entry in the table for the agent function
is correct (rational).
What is correct?
l The actions that cause the agent to be most
successful
l So we need ways to measure success.
11/25/21 10
PERFORMANCE MEASURE
Performance measure
l An objective function that determines
l How the agent does successfully
l E.g., 90% or 30% ?
11/25/21 11
PERFORMANCE MEASURE
A general rule:
l Design performance measures according to
l What one actually wants in the environment
l Rather than how one thinks the agent should behave
11/25/21 12
RATIONALITY
What is rational at any given time depends on
four things:
l The performance measure defining the criterion of
success
l The agent’s prior knowledge of the environment
l The actions that the agent can perform
l The agents’s percept sequence up to now
11/25/21 13
RATIONALITY
What is rational at any given time depends on
four things:
l The performance measure defining the criterion of
success
l The agent’s prior knowledge of the environment
l The actions that the agent can perform
l The agents’s percept sequence up to now
11/25/21 14
RATIONAL AGENT
For each possible percept sequence,
l an rational agent should select
l an action expected to maximize its performance measure, given
the evidence provided by the percept sequence and whatever
built-in knowledge the agent has
E.g., an exam
l Maximize marks, based on
the questions on the paper & your knowledge
11/25/21 15
EXAMPLE OF A RATIONAL AGENT
Performance measure
l Awards one point for each clean square
l at each time step, over 10000 time steps
11/25/21 16
EXAMPLE OF A RATIONAL AGENT
Actions that can perform
l Left, Right, Suck and NoOp
Percept sequences
l Where is the agent?
l Whether the location contains dirt?
An example
l crossing a street but died of the fallen
cargo door from 33,000ft à irrational?
11/25/21 18
OMNISCIENCE
Based on the circumstance, it is
rational.
As rationality maximizes
l Expected performance
Perfection maximizes
l Actual performance
Hence rational agents are not
omniscient.
11/25/21 19
LEARNING
Does a rational agent depend on only
current percept?
l No, the past percept sequence should also be
used
l This is called learning
l After experiencing an episode, the agent
l should adjust its behaviors to perform better for the same
job next time.
11/25/21 20
AUTONOMY
If an agent just relies on the prior knowledge
of its designer rather than its own percepts
then the agent lacks autonomy
A rational agent should be autonomous- it
should learn what it can to compensate for
partial or incorrect prior knowledge.
E.g., a clock
l No input (percepts)
l Run only but its own algorithm (prior knowledge)
l No learning, no experience, etc.
11/25/21 21
SOFTWARE AGENTS
Sometimes, the environment may not be
the real world
l E.g., flight simulator, video games, Internet
l They are all artificial but very complex
environments
l Those agents working in these
environments are called
l Softwareagent (softbots)
l Because all parts of the agent are software
11/25/21 22
TASK ENVIRONMENTS
Task environments are the problems
l While the rational agents are the solutions
Specifying the task environment
l PEAS description as fully as possible
l Performance
l Environment
l Actuators
l Sensors
11/25/21 24
TASK ENVIRONMENTS
Environment
lA taxi must deal with a variety of
roads
l Traffic lights, other vehicles,
pedestrians, stray animals, road
works, police cars, etc.
l Interact with the customer
11/25/21 25
TASK ENVIRONMENTS
Actuators (for outputs)
l Control over the accelerator, steering,
gear shifting and braking
l A display to communicate with the
customers
Sensors (for inputs)
l Detect other vehicles, road situations
l GPS (Global Positioning System) to know
where the taxi is
l Many more devices are necessary
11/25/21 26
TASK ENVIRONMENTS
A sketch of automated taxi driver
11/25/21 27
PROPERTIES OF TASK ENVIRONMENTS
Fully observable vs. Partially
observable
l If an agent’s sensors give it access to the
complete state of the environment at each
point in time then the environment is
effectively and fully observable
l if
the sensors detect all aspects
l That are relevant to the choice of action
11/25/21 28
Partially observable
§ An environment might be Partially
observable because of noisy and
inaccurate sensors or because parts of
the state are simply missing from the
sensor data.
§ Example:
l A local dirt sensor of the cleaner cannot
tell
l Whether other squares are clean or not
11/25/21 29
PROPERTIES OF TASK ENVIRONMENTS
Deterministic vs. stochastic
l next state of the environment Completely
determined by the current state and the
actions executed by the agent, then the
environment is deterministic, otherwise, it
is Stochastic.
l Strategic environment: deterministic
except for actions of other agents
§ -Cleaner and taxi driver are:
l Stochastic because of some unobservable
aspects à noise or unknown
11/25/21 30
PROPERTIES OF TASK ENVIRONMENTS
Episodic vs. sequential
l An episode = agent’s single pair of perception
& action
l The quality of the agent’s action does not
depend on other episodes
l Every episode is independent of each other
l Episodic environment is simpler
l The agent does not need to think ahead
Sequential
l Current action may affect all future decisions
§ -Ex. Taxi driving and chess. 11/25/21 31
PROPERTIES OF TASK ENVIRONMENTS
Static vs. dynamic
l A dynamic environment is always
changing over time
l E.g., the number of people in the street
l While static environment
l E.g., the destination
Semidynamic
l environment is not changed over time
l but the agent’s performance score does
11/25/21 32
PROPERTIES OF TASK ENVIRONMENTS
11/25/21 33
PROPERTIES OF TASK ENVIRONMENTS
Single agent VS. multiagent
l Playing a crossword puzzle – single
agent
l Chess playing – two agents
l Competitive multiagent
environment
l Chess playing
l Cooperative multiagent
environment
l Automated taxi driver
11/25/21 34
l Avoiding collision
PROPERTIES OF TASK ENVIRONMENTS
Known vs. unknown
This distinction refers not to the
environment itslef but to the agent’s
(or designer’s) state of knowledge
about the environment.
-In known environment, the outcomes
for all actions are
given. ( example: solitaire card games).
- If the environment is unknown, the
agent will have to learn how it works
in order to make good decisions.(
example: new video game). 11/25/21 35
11/25/21 36
11/25/21 37
STRUCTURE OF AGENTS
Agent = architecture + program
l Architecture = some sort of computing
device (sensors + actuators)
l (Agent) Program = some function that
implements the agent mapping = “?”
l Agent Program = Job of AI
11/25/21 38
AGENT PROGRAMS
Input for Agent Program
l Only the current percept
Input for Agent Function
l The entire percept sequence
l The agent must remember all of them
11/25/21 39
Skeleton design of an agent program
11/25/21 40
AGENT PROGRAMS
P = the set of possible percepts
T= lifetime of the agent
l The total number of percepts it receives
Size of the look up table
å
T t
t =1
P
Consider playing chess
l P =10, T=150
l Will require a table of at least 10150 entries
11/25/21 41
AGENT PROGRAMS
Despite of huge size, look up table does what
we want.
The key challenge of AI
l Find out how to write programs that, to the extent
possible, produce rational behavior
l From a small amount of code
l Rather than a large amount of table entries
11/25/21 42
TYPES OF AGENT PROGRAMS
Four types
l Simple reflex agents
l Model-based reflex agents
l Goal-based agents
l Utility-based agents
11/25/21 43
SIMPLE REFLEX AGENTS
It uses just condition-action rules
l The rules are like the form “if … then …”
l efficient but have narrow range of applicability
l Because knowledge sometimes cannot be stated
explicitly
l Work only
l if the environment is fully observable
11/25/21 44
11/25/21 45
11/25/21 46
A Simple Reflex Agent in Nature
percepts
(size, motion)
RULES:
(1) If small moving object,
then activate SNAP
(2) If large moving object,
then activate AVOID and inhibit SNAP
ELSE (not moving) then NOOP
needed for
completeness Action: SNAP or AVOID or NOOP 11/25/21 47
MODEL-BASED REFLEX AGENTS
For the world that is partially observable
l the agent has to keep track of an internal state
l That depends on the percept history
l Reflecting some of the unobserved aspects
l E.g., driving a car and changing lane
11/25/21 48
Example Table Agent
With Internal State
IF THEN
Saw an object ahead, Go straight
and turned right, and
it’s now clear ahead
Saw an object Ahead, Halt
turned right, and object
ahead again
See no objects ahead Go straight
start
11/25/21 53
GOAL-BASED AGENTS
Conclusion
l Goal-based agents are less efficient
l but more flexible
l Agent ß Different goals ß different tasks
l Search and planning
l two other sub-fields in AI
l to find out the action sequences to achieve its goal
11/25/21 54
11/25/21 55
UTILITY-BASED AGENTS
Goals alone are not enough
l to generate high-quality behavior
l E.g. meals in Canteen, good or not ?
11/25/21 56
11/25/21 57
UTILITY-BASED AGENTS
it is said state A has higher utility
l If state A is more preferred than others
Utility is therefore a function
l that maps a state onto a real number
l the degree of success
11/25/21 58
UTILITY-BASED AGENTS (3)
Utility has several advantages:
l When there are conflicting goals,
l Only some of the goals but not all can be
achieved
l utility describes the appropriate trade-off
11/25/21 59
LEARNING AGENTS
After an agent is programmed, can it work
immediately?
l No, it still need teaching
In AI,
l Once an agent is done
l We teach it by giving it a set of examples
l Test it by using another set of examples
We then say the agent learns
l A learning agent
11/25/21 60
LEARNING AGENTS
Four conceptual components
l Learning element
l Making improvement
l Performance element
l Selecting external actions
l Critic
l Tells the Learning element how well the agent is doing with
respect to fixed performance standard.
(Feedback from user or examples, good or not?)
l Problem generator
l Suggest actions that will lead to new and informative
experiences.
11/25/21 61
11/25/21 62