Module CS3MAS/CS4760

Multi-agent Systems

Lectures 12 and 13:

Working together
Maria Chli

Based on “An Introduction to MultiAgent Systems” by Michael

Wooldridge, John Wiley & Sons, 2009.
 Learning Outcomes
By the end of these lectures you should be able to:
• Recognise the main issues that multi-agent problem
solving entails.
• Explain the cooperative distributed problem solving
approach and contrast it to multi-agent approaches.
Explain similarities
– Coherence and Coordination
– Task decomposition/sharing, Subproblem solution,
Solution synthesis/sharing
• Explain the working of the ContractNet protocol
• Discuss different approaches to coordination
– Partial global planning
– Coordination through joint interactions
– Coordination through mutual modelling
– Social norms
 Working Together
• Main issue of MAS to be treated in more
depth: Collaboration
• Distinctions between “traditional”
Distributed Systems and MAS:
– Agents are self-interested: interactions
resemble games
– Agents: Decision making and coordination
and collaboration dynamically at run-time
(vs. at design time in traditional
Distributed Systems)
 Cooperative Distributed
Problem Solving
• CDPS studies how a loosely-coupled network of
problem solvers can work together to solve problems
that are beyond their individual capabilities.
• Important to make a distinction between:
– Benevolent agents: If we “own” the whole system, we can
design agents to help each other whenever asked.
– Benevolence simplifies the system design task
enormously!
• Self-interested agents: If agents represent individuals or
organizations, we cannot make the benevolence
assumption.
• Agents will be assumed to act to further their own
interests, possibly at expense of others: Potential for
conflict.
• May complicate the design task enormously.
 Coherence and Coordination
• How to ensure “success” of MAS? (Emergence is hard
to engineer)
• Criteria to consider: Coherence & Coordination
• Coherence: Refers to how well the MAS behaves as a
unit along some dimension of evaluation. May be
measured in terms of:
– Solution quality, resource usage, conceptual clarity of
operation, performance degradation if unexpected failure
occurs
• Coordination: The degree to which the agents can
avoid ‘extraneous’ activity such as synchronizing and
aligning their activities” . Poor coordination if
– Agents clobber each other’s sub-goals
– Lots of communication (no mutual predictability (e.g. by
expressive models of each other))
– Destructive interference if conflict
 CPDS Stages
(also applicable to MAS)
1. Problem decomposition: how can a problem be
divided into smaller tasks for distribution among
agents?
2. Sub-problem solution
3. Solution synthesis: how can a problem solution be
effectively synthesized from sub-problem results?
Complex Task

Task
Decomposition

Sub-sub-sub tasks performed

 Stage 1: Problem
decomposition
• Iteratively hierarchically decompose overall
problem into smaller sub-problems until agent can
solve them
• Different decomposition levels → different levels of
abstraction
• Important: Decomposition granularity.
– E.g.: ACTOR system decomposed problem until sub-
problems are at the level of programming language
commands → too fine grained → problems with
synthesis, management overhead outweigh
decomposition advantages
• Single decomposition instance vs. collaborative
decomposition
– Competence of agents must be known
 Stages 2 and 3
2. Sub-problem solution: Sharing of
information during sub-problem solution
3. Solution synthesis: may also be hierarchical
(respecting different levels of abstraction)

In order to apply CDPS stages we basically

need:
– task sharing/distribution/allocation (for problem
decomposition) and ←ContractNet deals with this problem
– result sharing (for solution computation and
synthesis).
 Task Sharing and Result
Sharing
Two main modes of cooperative problem
solving:
• Task sharing: Decompose problem and allocate sub-
problems to agents
– Homogeneous agents: Allocation is trivial: Any agent can
do any sub-problem task
– Inhomogeneous or self-interested agents: Allocation
through reaching agreements
• Result sharing: Agents share results: communicate
information relevant to sub-problem-solution;
either proactively or reactively (on request)
 The Contract Net Protocol
• A well known task-sharing protocol for task
allocation is the Contract Net (CNET).
• Task distribution viewed as a kind of contract
negotiation.
• “Protocol” specifies content of communication, not
just form.
• Primary concerns: distributed control, achieving
reliability, avoiding bottlenecks
• Stages in CNET:
1.Recognition
2.Announcement
3.Bidding
4.Awarding
5.Expediting
 The Contract Net Protocol
Manager Contractor
agent agent

Task announcement
Task evaluation
Bid message
Bid evaluation
Award message Contract established
Task performed
Reports
Report assimilation
Termination message
 The Contract Net Protocol
• Task generating node (task manager) advertises task
with task announcement
– No knowledge about agent capabilities → general broadcast
– Subset of agents known to be capable → limited broadcast
– Single agents known to be capable → point-to-point
announcement
• Agents evaluate task announcements; if suitable →
agent issues bid.
• Manager evaluates bids; selects most appropriate agent;
sends award message to winner (contractor).
• Contractor expedites task; may generate further sub-
tasks.
• After task completion, contractor sends report.
 Node Issues Task
Announcement
Task Announcement

Manager
 Idle Node Listening to Task
Announcements

Manager

Potential
Contractor Manager

Manager
 Node Submitting a Bid

Bid

Manager

Potential
Contractor
 Manager listening to bids

Bids

Potential
Manager
Contractor

Potential
Contractor
 Manager Making an Award

Award

Manager

Contractor
 Contract Established

Contract

Manager

Contractor
 Contract Net
• The collection of nodes is the “contract net”
• Each node on the network can, at different times or
for different tasks, be a manager or a contractor
• When a node gets a composite task (or for any
reason can’t solve its present task),
– it breaks it into subtasks (if possible) and
– announces them (acting as a manager),
– receives bids from potential contractors, then
– awards the job (example domain: network resource
management, printers, …)
 ContractNet processing

• In the original ContractNet implementation each

node (agent) comprised of
– Task announcement processing: Agent decides if it is
capable (and eligible) to carry out task by evaluating
eligibility specification of task announcement. If
eligible → details of task are stored.
– Bid processing: After issuing announcement, manager
accepts bids for limited amount of time.
– Award processing: Losers delete task details. Winner
starts task execution → may need to decompose task
into subtasks
– Request/Inform processing: If info is not immediately
available, agent informs requestor that info is
unknown; inform msg. → add info to database
 Features of Protocol
• Two-way transfer of information
• Local Evaluation
• Mutual selection (bidders select from among task
announcements, managers select from among bids)
• Relation to other mechanisms for transfer of
control
– The contract net views transfer of control as a runtime,
symmetric process that involves the transfer of complex
information in order to be effective
– Other mechanisms (procedure invocation, production
rules, pattern directed invocation, blackboards) are
unidirectional, minimally run-time sensitive, and have
restricted communication.
 ContractNet - Advantages
1. Simple to implement, and constitutes the
basis for other protocols
2. Fully distributed
3. Graceful performance degradation
• Applications suitable for the ContractNet
protocol are those that feature:
– A hierarchy of tasks or levels of data abstraction
– Relatively large subtasks which justify
considerable effort being put into agent selection
– Primary concerns are distributed control,
achieving reliability, avoiding bottlenecks
– Examples: manufacturing control and scheduling
 ContractNet - Limitations

• Before sub-problems can be distributed (or

announcements can be made),
– problem decomposition needs to be performed
(highly non-trivial).
– recognition stage (agent realizes that it needs help
with a problem) is not explicitly covered.
• Communication produces overhead, is slow.
• Problems must have right granularity (rather
coarse).
 Result Sharing
• Result sharing: Agents cooperatively exchange
result information as solution is developed. In the
course of communication, simple results may be
combined into more abstract results.
• Problem solvers can improve group performance in
result sharing by:
– Confidence increases through result cross checking →
increased correctness (errors can be more easily
detected)
– Completeness increases through sharing local
information, giving better overall global view
– Precision of results increases (less errors, more details
etc.)
– Timeliness: Result sharing may improve speed of
problem solution through parallelization, distinct
competences / knowledge etc.
 Handling Inconsistency
• Inconsistencies between agents:
– belief-inconsistency (e.g. due to fact that no
agent has complete picture, or faulty sensors
etc.)
– goal/intention-inconsistency (typical of self-
interested agents)
• How to deal with incosistencies:
– Do not allow inconsistencies or at least ignore
them
• E.g. in contract net, task sharing is initiated and governed by
single manager agent → only this agent’s view matters)
– Resolve inconsistencies through negotiation
• expensive, large overhead, slow
– Build systems that degrade gracefully in the
presence of inconsistencies
 Functional Accuracy /
Cooperation
• Systems that degrade gracefully in the presence of
inconsistencies are called functionally accurate /
cooperative (FA/C)
• Lesser & Corkill(1981) suggest characteristics of FA/C
systems:
– Collaborative problem solving progresses opportunistically
(not in a strict predetermined order, but taking advantage of
whatever opportunities arise) and incrementally (gradually
piecing together solutions)
– Agents should exchange high level intermediate results and
not raw data
– Uncertainties and inconsistencies should be resolved in as
problem solving progresses not in the beginning or at the end
– Solution finding not constrained to single route; if one fails,
alternative routes should be there.
 Coordination
• Coordination: Managing inter-dependencies between
the activities of agents
• Examples of interdependencies:
– Two people want to go through the same door
– I cannot proceed with my work until you have given your ok
– I make you a copy of an interesting paper without being
asked to do so
• Interdependencies can be positive or negative
• Positive relationships (benefits for at least one of
the agents while leaving others at least as happy
(pareto-optimality) may be requested or non
requested
 consumable
resource
resource
non-consumable
negative resource

inter-
incompatibility
dependencies

requested
positive (explicit)

non-
requested
(implicit)
van Martial’s coordination task
interdependencies
• Three types of non-requested
interdependencies:
– Action-equality-interdependence: Both agents
need to have action α done
• one of them can do it and save the other effort
– Consequence-interdependence: Actions of one
agent’s plan have side effect of achieving other
agent’s goal.
– Favour-interdependence: Actions of one agent’s
plan have side effect of partially achieving other
agent‘s goal (positively contributing to it)
Coordination through partial
global planning (PGP)
• PGP: Agents exchange info to reach conclusions
about problem-solving plan
• Partial planning because MAS is not required to
deliver complete solution, global because agents
exchange local plans to achieve a global solution
(plan)
• Three iterated stages:
1. each agent decides about its goals, creates local plan
2. agents exchange plans to determine interdependencies
3. agents alter local plans to achieve better coordination
 The partial global plan
• Agent’s plans are incorporated into data-
structure: partial global plan. Contains
– Objective: The overall goal the MAS works
towards
– Activity maps: What agents are currently
doing, what the result of their current action is
– Solution construction graph: Representation of
when and how agents should interact; what
info should be exchanged, when overall
solution is assembled
 Generalised Partial Global
Planning (GPGP)
• Decker formulated improvement-aspects:
– Update non-local viewpoints
• global viewpoint may be reached
– Communicate results
– Resolve simple redundancy (several agents to
the same, then select one at random)
– Resolve hard (negative) interdependencies
through rescheduling partial plans
– Use soft (positive) interdependencies by
possibly rescheduling partial plans
 Coordination through joint
intentions
• Intentions (committed goals) were important for
practical reasoning. Intentions also important for
coordination in team scenaria.
– E.g. knowing that I want to lose weight anybody can rule out
interdependencies wrt. to offering me cakes.
• Important: distinguish individual intentions (that may
be coordinated) from intentions to cooperatively and
coordinatedly achieve a goal as a team
• Commitment associated with an intention: Future
directed, persistent, should not be dropped for no
reason → conventions exist that regulate when it’s
appropriate to drop an intention.
– E.g.: when lifting a heavy object together.
 Joint Persistent Goal
• In a joint persistent goal (JPG),a group of agents have a
collective commitment to bringing about some goal ϕ;
• The motivation for this goal, i.e. the reason that the group has
the commitment, is represented by ψ.
• ϕ might be ‘move the heavy object’,
• ψ might be ‘Maria wants the heavy object moved’.
• The mental state of the team of agents with this JPG might be
described as follows:
• Initially each agent: believes φ has not been satisfied and
believes possibleToDo(φ)
• Until termination condition is met, each agent has goal φ
• Termination condition: It is mutually believed that either
• φ is satisfied OR φ is impossible OR ψ is no longer valid
• Until termination condition is met:
• If an agent believes that either
• the goal is met OR the goal is impossible OR
• the motivation no longer holds,
it has the goal to make this mutually believed (the goal to
convince all others about that)
 Teamwork-based model of
CDPS
Recall stages of CDPS from beginning. Now: CDPS-model
focused on team-work-based coordination. Four stages:
1. Recognition: Agent has goal → realizes potential for
cooperative action wrt. goal. Cases:
1. Agent is not able to make it alone
• insufficient capabilities or information , e.g.: lifting heavy object
2. Agent can make it alone but cooperative solution is
preferable
• more accurate or execution alone would interfere with other goals
• E.g.: lifting possible but with back injury
2. Team formation: If successful, team is established and
has commitment to collaborative goal.
– Commitment to ends, not to means. This stage corresponds to
collective deliberation.
3. Plan formation: Established through negotiation or
argumentation (commitment to means)
4. Team action: Joint plan is executed (e.g. as in JPG)
 Coordination by mutual
modelling
• Having model of other agents (beliefs, intentions,
goals) and of cooperative conventions enables
coordination without communication
• Example:
– A man and a woman approach a door → resource conflict
– Possible solution: both wait → waste of resource.
– Another solution: each agent builds a model of each other
(beliefs, intentions)
– Knowing usual convention and having a model of each
other (“other agent is conservatively polite”) → woman
will go first
• Model could be induced by game-theoretic payoff-
matrix → agents will know rational action
• Example system: MACE
 Coordination by norms and
social laws
• A norm is an established, expected pattern of
behaviour.
– The term social law means the same, but it is usually implied
that social laws carry with them some authority.
• Example of a norm in human society:
In the UK, it is a norm to form a queue when waiting for a bus.
This norm is not enforced in any way – usually no penalty if we
do not follow it.
• In software MAS, conventions can be implemented
– At design time (offline design) (easier, more straightforward,
more direct control)
– As an emergent behaviour (more flexible in unforseeable
environments, producing possibly better coherence)
 Emergent norms and laws
• Difficulty for each agent: Establish global
convention on the basis of local information
• Example:
– T-Shirt game: MultiAgent System, where each agent
can wear red or blue t-shirt.
– Goal: all agents wear same colour;
– In each iteration: agent sees only one other agent.
– After a round: each agent is allowed to either stay
wearing the same coloured tee shirt, or to swap to
the other colour.
 Strategy update functions
• Strategy update functions (strategy = t-shirt colour)
– Simple majority: change from strategy s1 to s2 if more
agents with s2 were seen.
– Simple majority with agent types (wrt. strategy) and
communication (exchange of complete memory with
agents of same type) (→ broadening their statistical basis
for majority decision)
– Simple majority with communication on success : When
agent has reached certain success-level with current
strategy → communicate memory related to successful
strategies to other agents (→ only successful strategies are
communicated)
– Highest cumulative reward: Record payoff of current
strategy; choose strategy with highest cumulative payoff
• Efficiency of strategy update function measured
e.g. by convergence time in t-shirt game.
Experiments: all strategies were successful
 Learning Outcomes
By the end of these lectures you should be able to:
• Recognise the main issues that multi-agent problem
solving entails.
• Explain the cooperative distributed problem solving
approach and contrast it to multi-agent approaches.
Explain similarities
– Coherence and Coordination
– Task decomposition/sharing, Subproblem solution, Solution
synthesis/sharing
• Explain the working of the ContractNet protocol
• Discuss different approaches to coordination
– Partial global planning
– Coordination through joint interactions
– Coordination through mutual modelling
– Social norms