Artificial Intelligence and Machine Learning

Dr.Venugopal P
SENSE
• Artificial
– Produced by human art or effort, rather than originating
naturally.
• Intelligence
• is the ability to acquire knowledge and use it" [Pigford and
Baur]
• So AI was defined as:
– AI is the study of ideas that enable computers to be
intelligent.
– AI is the part of computer science concerned with design of
computer systems that exhibit human intelligence(From the
Concise Oxford Dictionary)
Four definitions of AI, laid out along two dimensions

Ex:
Ex:

Ex: Ex:
Eight definitions of AI, laid out along two dimensions
 Turing Test in AI

The questions and answers can be like:

Interrogator: Are you a computer?

PlayerA (Computer): No

Interrogator: Multiply two large numbers such as (256896489*456725896)

Player A: Long pause and give the wrong answer.

Is computer passes the test?
In this game, if an interrogator would not be able to identify which is a machine and which is
human, then the computer passes the test successfully, and the machine is said to be intelligent
and can think like a human.
 The foundations of AI
In this we are going to discuss various disciplines such as philosophy mathematics,
economics, neuroscience, psychology computer engineering, control theory and cybernetics
that contribute various ideas, view points and techniques to AI.
 1. Philosophy

Example:
Statement-1: If you have my home key then you can unlock my home. P→Q
Statement-2: If you can unlock my home then you can take my money. Q→R
Conclusion: If you have my home key then you can take my money. P→R
2.Mathematics
3. Economics
4.Neuroscience
5.Psychology
6.Computer Engineering
6.Computer Engineering…
7. Control theory
8. Linguistics
 Applications
1.Robotics Vehicles:
A driverless robotic car named STANLEY sped through
the rough terrain of the Mojave desert at 22mph,
finishing the 132 mile course first to win DARPA
Grand Challenge.
2.Speech Recognition:
A traveller call a United Airlines to book a flight can
have the entire conversation guided by an automated
speech recognition and dialog management system
3.Autonomus planning and schedule:
A hundred million miles from Earth, NASA’s
remote agent program become the first on-
board autonomous planning program to control
the scheduling of operations for a spacecraft.
REMOTE AGENT generated plans from high-level
Goals specified from ground and monitored the
execution of those plans- detecting, diagnosing
and recovering from problems as they occurred.
3.Game playing:
IBM’s DEEP BLUE became the first computer
program to defeat the world champion in a
chess match when it bested Garry Kasparov
4.Spam fighting:
Each da, learning algorithms classify over a
billion messages as spam, saving the recipient
from having to waste time to delete.
Learning algorithms work best.
5.Logistic planning:
DART(Dynamic Analysis Replanning Tool) to do
automated logistics planning and scheduling for
transportation
6.Robotics:
The iRobot Corporation has sold over 2 million
Roomba robotic vacuum cleaners for home use.
7.Machine Translation:
A computer program automatically translates from
Arabic to English
The program uses a statistical model built from
examples of English text two trillion words
 Agents and Environments
An agent is anything that can be viewed as perceiving its
environment through sensors and acting upon that environment
through actuators.
Figure 1 Agents interact with environments through sensors and actuators.
Agent Terminology
•Performance Measure of Agent − It is the criteria, which determines how successful
an agent is.

•Behavior of Agent − It is the action that agent performs after any given sequence of
percepts.

•Percept − It is agent’s perceptual inputs at a given instance.

•Percept Sequence − It is the complete history of all that an agent has perceived till
date.

•Agent Function − It is a map from the precept sequence to an action.

•Agent Program - an implementation of an agent function

Note: The agent function is an abstract mathematical description; the agent program
is a concrete implementation, running within some physical system.
 We can imagine tabulating the agent function that describes any
given agent; for most agents, this would be a very large table—
infinite, in fact, unless we place a bound on the length of percept
sequences we want to consider.

 Given an agent to experiment with, we can, in principle,

construct this table by trying out all possible percept sequences
and recording which actions the agent does in response.
 The table is, of course, an external characterization of the agent.
Internally, the agent function for an artificial agent will be
implemented by an agent program

 To illustrate these ideas, we use a very simple example—the

vacuum-cleaner world shown in Figure 2.
Figure 2 A vacuum-cleaner world with just two locations.

Figure 3 Partial tabulation of a simple agent function for the vacuum-cleaner world
shown in Figure 2.
GOOD BEHAVIOR: THE CONCEPT OF RATIONALITY
• A rational agent is one that does the right thing—conceptually
speaking, every entry in the table for the agent function is filled out
correctly. Obviously, doing the right thing is better than doing the
wrong thing, but what does it mean to do the right thing?
• We answer this age-old question in an age-old way: By considering the
consequences of the agent’s behavior.
• When an agent is plunked down in an environment, it generates a
sequence of actions according to the percepts it receives.

• This sequence of actions causes the environment to go through a

sequence of states.

• If the sequence is desirable, then the agent has performed well.

• This notion of desirability is captured by a performance measure that

evaluates any given sequence of environment states.
Consider, for example, the vacuum-cleaner agent.

We might propose to measure performance by the amount of dirt cleaned up in a

single eight-hour shift.

With a rational agent, of course, what you ask for is what you get.

A rational agent can maximize this performance measure by cleaning up the dirt,
then dumping it all on the floor, then cleaning it up again, and so on.

A more suitable performance measure would reward the agent for having a clean
floor. For example, one point could be awarded for each clean square at each time
step (perhaps with a penalty for electricity consumed and noise generated).

As a general rule, it is better to design performance measures according to what

one actually wants in the environment, rather than according to how one thinks the
agent should behave.
 Rational Agent
What is rational at any given time depends on four things
1.The performance measure.
2.The agent’s prior knowledge of the environment.
3.The actions that the agent can perform
4.The agent’s percept sequence to date.

This leads to a definition of a rational agent

For each possible percept sequence, a rational agent should select

an action that is expected to maximize its performance measure,
given the evidence provided by the percept sequence and
whatever built-in knowledge the agent has.
 AI Environment
Task environment:
•In our discussion of the rationality of the simple vacuum-cleaner agent, we had
to specify the performance measure, the environment, and the agent’s actuators
and sensors. We group all these under the heading of the task environment. For
the acronymically minded, we call this the PEAS (Performance, Environment,
Actuators, Sensors) description.

•In designing an agent, the first step must always be to specify the task
environment as fully as possible.

•An environment is everything in the world which surrounds the agent, but it is
not a part of an agent itself. An environment can be described as a situation in
which an agent is present.

•The environment is where agent lives, operate and provide the agent with
something to sense and act upon it.
Let us consider a more complex problem: an automated taxi driver.

Figure 4 PEAS description of the task environment for an automated taxi.

Figure 5 Examples of agent types and their PEAS descriptions.
 1-Fully Observable vs. Partially Observable
• If an agent sensor can sense or access the complete state of
an environment at each point of time then it is a fully
observable environment, else it is partially observable.

• A fully observable environment is easy as there is no need to

maintain the internal state to keep track history of the world.

Fully Observable Partially Observable

An agent with no sensors in all environments then such an

environment is called as unobservable.
 2-Deterministic vs. Stochastic
• If an agent's current state and selected action can completely
determine the next state of the environment, then such
environment is called a deterministic environment.

• Real-life Example: The traffic signal is a deterministic

environment where the next signal is known for a pedestrian
(Agent).

• The Stochastic environment is the opposite of a deterministic

environment. The next state is totally unpredictable for the agent.
So randomness exists in the environment.

• Real-life Example: The radio station is a stochastic environment

where the listener is not aware about the next song or playing a
soccer is stochastic environment.
2-Deterministic vs. Stochastic ….
 3.Episodic vs. sequential
• In an Episodic task environment, each of the agent’s actions is divided
into atomic incidents or episodes. There is no dependency between
current and previous incidents. In each incident, an agent receives
input from the environment and then performs the corresponding
action.
• Example: Consider an example of Pick and Place robot, which is used
to detect defective parts from the conveyor belts. Here, every time
robot(agent) will make the decision on the current part i.e. there is no
dependency between current and previous decisions.
• In a Sequential environment, the previous decisions can affect all
future decisions. The next action of the agent depends on what action
he has taken previously and what action he is supposed to take in the
future.
• Example:
– Playing tennis is a perfect example where a player observes
the opponent’s shot and takes action.
3.Episodic vs. sequential
 4-Static vs. Dynamic
• If the environment does not change while an agent is acting,
then it is static;
• Example: Cleaning a room (Environment) by a dry-cleaner
reboot (Agent ) is an example of a static environment where
the room is static while cleaning.

• Dynamic Environment could be changed while an agent is

precepting the environment. So agents keep looking at the
environment while taking action.
• Example: Playing soccer is a dynamic environment where
players’ positions keep changing throughout the game. So a
player hit the ball by observing the opposite team.
 5-Discrete vs. Continuous
• Discrete Environment consists of a finite number of states
and agents have a finite number of actions.
• Example: Choices of a move (action) in a tic-tac game are
finite on a finite number of boxes on the board
(Environment).
• While in a Continuous environment, the environment can
have an infinite number of states. So the possibilities of
taking an action are also infinite.
Example: In a basketball game, the position of players
(Environment) keeps changing continuously and hitting
(Action) the ball towards the basket can have different
angles and speed so infinite possibilities.
 6 - Known vs. Unknown
• Known and unknown are not actually a feature of an
environment, but it is an agent's state of knowledge to
perform an action.
• In a known environment, the results for all actions are
known to the agent.
• While in unknown environment, agent needs to learn how
it works in order to perform an action.

Unknown
Known
 7- Single-agent vs Multi-agent
• If only one agent is involved in an
environment, and operating by itself then
such an environment is called single agent
environment.
• However, if multiple agents are operating in
an environment, then such an environment
is called a multi-agent environment.
• The agent design problems in the multi-
agent environment are different from single
agent environment.
7- Single-agent vs Multi-agent
 8- Competitive vs Collaborative
• An agent is said to be in a competitive environment when it
competes against another agent to optimize the output.
• Example: The game of chess is competitive as the agents
compete with each other to win the game which is the
output.
• An agent is said to be in a collaborative environment when
multiple agents cooperate to produce the desired output.
• Example: When multiple self-driving cars are found on the
roads, they cooperate with each other to avoid collisions
and reach their destination which is the output desired.
There are mainly six groups
Fully vs ofDetermini
environment and an environment can
Episodi be in multiple groups.
Discrete
Below are 10 morePartially
real-life examples Single vs
stic vs and categories
c vs into environment
Static vs vs groups
Multi
Observa Stochasti Sequen Dynamic Continuo
Agents
ble c tial us

Brushing
Your
Teeth

Playing
Chess

Playing
Cards

Playing

Autonomo
us
Vehicles

Order in
Restauran
t
 Fully vs Episodic Discrete
Determini Single vs
Partially vs Static vs vs
stic vs Multi
Observa Sequent Dynamic Continuo
Stochastic Agents
ble ial us

Brushing Sequenti Continuou

Fully Stochastic Static Single
Your Teeth al s

Playing Sequenti Continuou

Partially Stochastic Dynamic Multi-Agent
Chess al s

Playing Sequenti Continuou

Partially Stochastic Dynamic Multi-Agent
Cards al s

Sequenti
Playing Partially Stochastic Dynamic Continuou Multi Agent
al

Autonomo Sequenti Continuou

Fully Stochastic Dynamic Multi-Agent
us Vehicles al s

Order in Determinis
Fully Episodic Static Discrete Single Agent
Restaurant tic
Task environment and their
characteristics
 The structure of Agents
• Agents in Artificial Intelligence follow this simple structural formula:

Architecture + Agent Program = Agent

• These are the terms most associated with agent structure:

• Architecture: This is the machinery or platform that executes the agent.

• Agent Function: The agent function maps a precept to the Action,

represented by the following formula: f:P* - A

• Agent Program: The agent program is an implementation of the agent

function. The agent program produces function f by executing on the
physical architecture.
 Types of Agent/Agent programs
• Simple reflex agents
• Model-based reflex agents
• Goal-based agents
• Utility-based agents
• Learning agent
 1.Simple reflex agents
• This is a simple type of agent which works on the basis of
current percept and not based on the rest of the percepts
history.
• The agent function, in this case, is based on condition-action
rule where the condition or the state is mapped to the action
such that action is taken only when condition is true or else it
is not.
• If the environment associated with this agent is fully
observable, only then is the agent function successful, if it is
partially observable, in that case the agent function enters
into infinite loops that can be escaped only on
randomization of its actions.
Example:
The vacuum agent is a simple reflex agent because the decision is based only on the
current location, and whether the place contains dirt.
Figure Schematic diagram of a simple reflex agent.
Example:
•Problems for the simple reflex agent design approach:

• They have very limited intelligence

• They do not have knowledge of non-perceptual parts
of the current state
• Mostly rules too big to generate and to store.
• Not adaptive to changes in the environment.
 2.Model-based reflex agents
•The Model-based agent can work in a partially observable environment, and track
the situation.

•A model-based agent has two important factors:

• Model: It is knowledge about "how things happen in the world," so it is

called a Model-based agent.
• Internal State: It is a representation of the current state based on percept
history.

•These agents have the model, "which is knowledge of the world" and based on the
model they perform actions.

•Updating the agent state requires information about:

• How the world evolves

• How the agent's action affects the world.
Figure: Model-based reflex agent
Pseudo code

Example:
With Waymo, for example, the model-based agent uses GPS to understand its
location and predict upcoming drivers. You and I take for granted that, when the
brake lights of the car ahead of us come on, the driver has hit the brakes and so
the car in front of us is going to slow down. But there's no reason to associate a
red light with the deceleration of a vehicle, unless you are used to seeing those
two things happen at the same time. So the Waymo can learn that it needs to hit
the brakes by drawing on its perceptual history. Waymo can learn to associate red
brake lights just ahead with the need to slow itself down. Another common task is
when the Waymo car decides it needs to change lanes. Should it just shimmy over
as though it's the only car in the world? No, it uses the same processes to estimate
whether any other cars might be in the path of its intended lane change, to avoid
3.Goal-based agents
Figure : Schematic diagram of Goal-based agent
• Sometimes goal-based agent is straightforward and
sometimes it will be more tricky.
• Goal-based agent involves consideration of the
future: “What will happen if I do such and such?” and
“Will that make me happy”
• Eg: Automated Taxi:
Reflex agent -Apply brakes when it sees brake
lights
Goal-based agent-It will slow down and given the way
the world usually evolves, the only action that will
achieve the goal of not hitting others cars is to brake.
• Goal-based agent: Less efficient, it is more
flexible because the knowledge that support
its decisions is represented explicitly and can
be modified.
• Goal-based agent: Behaviour can easily be
changed to go to a destination, simply by
specifying that destination as the goal.
• The reflex agent’s: Rules for when to turn and
when to go straight will work only for a single
destination.
 4.Utility-based agents
1.Utility agent have their end uses as their building blocks and is used when best action
and decision needs to be taken from multiple alternatives.

2.It is an improvement over goal based agent as it not only involves the goal but also the
way the goal can be achieved such that the goal can be achieved in a quicker, safer,
cheaper way.

3.The extra component of utility or method to achieve a goal provides a measure of

success at a particular state that makes the utility agent different.

4.It takes the agent happiness into account and gives an idea of how happy the agent is
because of the utility and hence, the action with maximum utility is considered. This
associated degree of happiness can be calculated by mapping a state onto a real number.

5.Mapping of a state onto a real number with the help of utility function gives the
efficiency of an action to achieve the goal.
Figure : Schematic diagram of Utility-based agent
 5.Learning Agent
1.Learning agent, as the name suggests, has the capability to learn from past experiences
and takes actions or decisions based on learning capabilities.

2.It gains basic knowledge from past and uses that learning to act and adapt automatically.

3.It comprises of four conceptual components, which are given as follows:

•Learning element: It makes improvements by learning from the environment.

•Critic: Critic provides feedback to the learning agent giving the performance
measure of the agent with respect to the fixed performance standard.

•Performance element: It selects the external action.

•Problem generator: This suggests actions that lead to new and informative
experiences.
Figure : Schematic diagram of Learning agent
 Reference
• Artificial Intelligence, A Modern Approach,
Third Edition
By
Stuart J. Ressell
Peter Norvig