Extra reading – Bratmann 1988

In this article they represent an architecture that (partly) solves the problems doing reasoning (both
means-end analysis and weighing competing alternatives) but within resource boundedness

Ovals are info stores, rectangles are processes.

Four main processes:
 Means-End reasoner
 Opportunity analyser
 Filtering process
 Deliberation (=overweging) process
* together nog only produce action, but also in constraining practical reasoning

Roles of agent’s plans in Means-End reasoner:

 Drive means-end reasoning (what to do to reach my goal)
 Provide constraints on what options need to be seriously considered
 Influence beliefs on which further reasoning will be based

Plans:
 Should be able to successfully execute in a world in which agent’s beliefs are true (if not->
filtered out).
 Shouldn’t be to influenced by reconsideration or abandonment (=stability)
 Should be partial: (not too detailed) done by:
o Temporally partial: only accounting in some periods of time
o Structurally partial: deciding on ends but leaving means still open to later deliberate
and fill in (also makes probability of inconsistency with other plans less)
 Must be means-end coherent: as time goes by, plans must be filled in with subplans that
(eventually) lead to executing the main plan successfully

Opportunity analyser:
Proposes options in response to perceived changes in the environment

Options from either means-end reasoner or opportunity analyser go to Filter process: (parallel)
 Compatibility filter: checks options to check agreement with agent’s existing plans (if
enough-> option = surviving option -> go to deliberation process)
 Filter Override mechanism: encodes agent’s sensitivities to problems and opportunities in
her environment (= conditions under which some portion of the existing plan is to be
suspended and weighed against other options). If an option fails compatibility filter but
survives Filter Override mechanisms -> option is up for reconsideration
* if too sensitive-> inefficient, if not sensitive enough-> will fail to react to situations that
differ a lot from its expectations

Abandoning a plan / belief (in filter override mechanism):

 If it is a means-> threatened with means-end incoherence-> deliberation process
 If not a means and desire for plan/belief is still there-> further deliberation if appropriate
opportunity arises
* still possible that an agent deliberates on something that is not worth the cost. But is unavoidable
* If an incompatible option triggers a filter override -> is reconsidered-> agent is cautious
* if incompatible option doesn’t trigger filter override-> agent is bold. (both are sometimes good)
* The more fine-tuned, the better

So procedures are needed for:

 Detecting threats to means-end coherence
 Proposing new options when environment has changed
 Monitoring prior plans if beliefs change
 Checking compatibility with prior plans
 Overriding the compatibility filter
Extra reading: Georgeff – BDI model of agency
This article will be about the BDI system (in rational agents) combining the Bratman model, IRMA
architecture and various PRS-systems. And compare BDI to other related models of agencies.

Limits of BDI agents:

 Cannot learn and adapt
 Don’t have social abilities (no multi-agent system)
 Have local view of the world (limited access to info)
 Are resource bounded (information processing limits)

Terms in BDI:
 Beliefs: computational representation of the world
 Desires (sometimes called Goals): represents some desired end state
* Bold and cautious agents:
 Conventional software is task-oriented (each task is executed without knowing why)
(bold)
 Classical decision theory is goal-oriented and always replans. (cautious)
 System needs to commit to plans but also reconsider plan at appropriate moments->
intentions
 Intentions: set of execution threads that can be interrupted by feedback from world
 Plans

BDI models employ folk psychology of belief, desire and intention. (don’t have to be bratman’s)
IRMA models: models that embody the Bratman claim: rational agents will reason on adopted
intentions, but bypass full consideration of options that conflict with the intentions.
PRS: a programming environment for developing complex applications that execute in dynamic
environments.

To learn and adapt: IRMA models should be implemented such that they can commit to a partial
plan, but then also know when to add detail to those plans (not solved yet)

Multi-agent system:
Soar is a compatible model, based on operators (which are similar to reactive plans) and states (incl
its highest-level goals and beliefs). Operators are preconditions that help select operators for an
action in the current state. Selecting high-level operators for execution leads to sub goals ->
hierarchical expansion or operators.
BDI parts in Soar:
 Intentions = selected operators
 Beliefs are included in current state
 Desires are goals (incl sub goals)
 Commitment strategies are for defining operator termination terms

Differences:
 Soar takes and empirical approach, where systems are first constructed and some
underlying principles are understood via constructed systems (Soar includes modules as
 chunking (form of explanation-based learning) and a truth maintenance system for
maintaining state consistency)
 BDI systems first clearly understand logical or philosophical underpinnings and then build
systems
 BDI and Soar should learn from each other
Extra reading: Brooks_1990
Nouvelle AI = Fundamentalist AI = situated activity is methodology based on ‘the physical
grounding hypothesis’, so decomposition of intelligence into individual behaviour generating
modules, whose coexistence and co-operation let more complex behaviours emerge.

Symbol System Hypothesis: intelligence operates on a system of symbols. But how to represent
world->
 Default assumption: description of world in terms of typed, named individuals and their
relationship (i.e. monkey, banana)
 But psychophysical evidence claims: perception is an active and task dependent operation
(just knowing objects might be enough, but sometimes you might need to know more about
the object, i.e. is banana rotten?)
The more chaotic the world-> the more complex the perception and symbols-> the harder the
computations (frame problem: impossible to assume anything that is not explicitly stated). All kinds
of evaluation functions have been tried to guide the search, but without much luck.

Physical grounding hypothesis: to build a system that is intelligent, it is necessary to have its
representations grounded in the physical world.
 Earth-based biological evolution suggests that every intelligent behaviour is rather simple
once the essence of being and reacting are available. This behaviour is the physically
grounded part of animal systems

Subsumption architecture: connects perception to action, embedding robots concretely in the world.
Containing subsumption compiler that takes a source file as input and produces and assembly
language program as output. Behaviour language (=new subsumption) = networks of augmented
finite state machines (AFSM) with timing elements. The AFSMs are grouped in units that can be
selectively activated or de-activated. so rule sets or real-time rules which compile into AFSMs in a
one-to-one manner

AFSMs consist of registers and a set of timers connect to a finite state machine. FSMs can either wait
on some event or compute a combinational function of the registers directing the result either back
to one of the registers or to an output of the augmented state machine.
Ways to connect new machines: (used when multiple layers are activated)
 Inhibition: New inputs can be connected to existing registers (might have contained a
constant before)
 Suppression: New machines can inhibit existing outputs, or supress existing inputs by being
attached as side-taps (message goes through side-tap wire, a continuous input is required to
maintain control of side-tap wire))

Characteristic frequency: rate of sending messages through a wire (max = 1 message per clock tick,
side-taps must have higher characteristic frequency than original wire)) (using alarm clocks)

In robot Allen: (first layer is lowest, third is top)

First layer made Allen run (in opposite direction) from obstacles if they were nearby
Second layer made Allen randomly wander around (every 10 s a random direction was generated) +
an obstacle avoidance
Third layer made Allen look for distant places (with sonar) and head towards them. (suppressed
wandering of second layer

In Tom and Jerry (two similar robots): (layers)

1. For obstacle avoidance (using vector sum of repulsive forces from obstacles)
2. To wander about
3. To detect moving objects and create a follow behaviour (wandering suppressed when
chasing)
* demonstrate notion of independent behaviours combing without knowing about each other

Herbert:
Programmed to wander around in office areas and steal empty soda cans from office desks, showed
obstacle avoidance, wall following, recognition of soda-like objects (with a laser) and behaviours like
picking it up.
* there was no internal communication (a next module started by seeing the change in the
environment (like a hand grabbing the can)-> can could also be given) -> showed that control system
can see opportunities if circumstances present itself and easily respond to changing circumstances.

Genghis:
Robot could stand up, walk without sensing (even through rough terrain), could step over obstacles
and could follow a moving source. All were independent actions (each leg another control system)

Squirt:
Acts as a bug (hiding in dark corners and is attracted by noises, after noise finds dark again by
spiralling), noise supresses the desire to stay in the dark

Toto:
Could move around without running into obstacles and tried to keep track of where it is on a map by
seeing certain landmarks in the environment. It could also have a goal destination (local pieces of
info combined to direct the robot through its dynamics of interaction with the world to the desired
place)

Seymour:?

All robots are task specific and don’t learn over time. Generality is wanted, but is way more complex
and it messes up the robustness of a system. Also professionals (i.e. a doctor) cannot do other
expertise, so why should these robots.

So:
 Traditional AI demonstrates sophisticated reasoning. Hope is for more robust behaviour in
complex domains
 Nouvelle AI showed robust behaviour in noisy complex domains. Hope is that it will become
more sophisticated
Extra reading: Faragher 2012
Kalman filter assumes that the state of a system at a time t evolved from prior state t-1 in equation:

 Xt = state vector containing terms of interest for system at time t

 Ut = vector containing control inputs
 Ft = state transition matrix, applying effect of state at t-1 on t
 Bt = control input matrix, applying effect of each control input parameter in u t of xt
 Wt = process noise terms vector for each parameter in x t (drawn form a zero mean
multivariate normal distribution with covariance given by covariance matrix Q t)

Measurements can also be performed according to the model:

 Zt = vector of measurements
 Ht = transformation matrix (mapping state vector parameters into measurement domain)
 Vt = vector containing measurement noise terms for each observation in z t. like process noise
(wt) measurement noise is assumed to be zero mean Gaussian white noise with covariance
Rt.

Estimation of xt = ^xt in a pdf (probability density function) (Gaussian)

Two stages:
 Prediction
With standard equations:
Where Qt = process noise covariance matrix associated
with noisy control inputs.

 Measurement update: (noisy)

 Product of these 2 Gaussians is another Gaussian-> new Gaussian can be used in next
estimation (endless) (no increase in complexity or number of terms)

* Kalman filter can be taught using a simple derivation involving scalar mathematics, basic algebraic
manipulations and an easy-to-follow thought experiment