Professional Documents
Culture Documents
Abstract—Computational tools for modeling multi-agent work, Some AA&R metrics proposed in [3] for evaluating the
including taskwork and teamwork, are needed to support the taskwork and teamwork induced by a given function alloca-
design of complex operational concepts involving human op-
tion include taskload/workload, mismatches between authority
erators and automated systems. This paper presents a human
performance model of task sharing with delegation where an and responsibility, and coherency of a function allocation.
agent can execute, delay, or delegate assigned taskwork accord- Taskload/workload describes the demands on the agent from
ing to its multi-tasking capacity and other attributes. Using a its own assigned taskwork and from coordinating with other
simulation framework for modeling multi-agent work, several agents. Mismatches between authority and responsibility arise
case studies apply the task sharing with delegation model to
when the performance of taskwork is assigned to a different
the evaluation of an air transportation operational concept. The
cases demonstrate how multi-tasking capacity, the delegation of agent from the agent deemed responsible for the work’s
taskwork, and the coordination of delegated taskwork impact outcome. The coherency of a function allocation highlights
the amount and time requirements of taskwork (on-time and the level to which work functions are allocated to one agent
delayed) and teamwork performed by agents. Thus, integrating compared to multiple agents, thus resulting in situations where
human performance models in the simulation framework can
multiple agents access and modify the same information
inform operational concept development for multi-agent systems.
resources in the work environment and must coordinate to
confirm their current values, when values changed, and which
I. I NTRODUCTION agent last updated the values. In [1], a future concept of
The design and evaluation of multi-agent concepts of oper- operation in which in-trail spacing functions may be allocated
ation require models and methods which can characterize and to either an air traffic controller (ATC) or to a flight crew
measure effective allocations of work across teams of humans is evaluated with different function allocations. The study
and automated agents, especially in work environments where illustrates how allocations which reduce the taskwork on the
the autonomy, authority, and responsibility (AA&R) of agents ATC are traded off against an increase in the taskwork of the
[1] are rapidly evolving. Here, autonomy describes whether an flight crews as well as the teamwork between the ATC and the
agent can perform work independently, highlighting agent ca- flight crews in exchanging information required for performing
pability. Authority describes the work that an agent is assigned spacing functions. The exchange of information is also affected
to perform. Responsibility describes the work whose outcomes by the coherency of a function allocation; when the ATC and
an agent is accountable for even if the agent does not perform flight crews share the taskwork which composes the same work
the work. function, the coordination requirements between the agents are
Function allocation is one mechanism for assigning work higher than when the taskwork within each work function is
for agents to perform (taskwork), while also defining agent allocated to the ATC or flight crews on their own.
roles and the work required by agents to coordinate their Evaluation of function allocation in an AA&R context can
taskwork (teamwork). [2] and [3] propose formal requirements further be conducted using models of delegation, in which the
of an effective function allocation and metrics for evaluating agents initially assigned authority to perform some taskwork
a function allocation, especially in terms of the distribution of need to re-assign the work to another agent, but still have
autonomy, authority, and responsibility within a multi-agent responsibility to coordinate with the delegee and monitor that
team. These requirements highlight AA&R by specifying that the work is successfully completed. A theory and model of
function allocations should match the capabilities of the agents delegation from [4] describe delegation as a bilateral agreement
involved and be realizable with sufficient teamwork, as when between a delegating agent, which delegates when it has a goal
an agent with responsibility for some taskwork monitors the to achieve requiring action from another agent, and an adopting
agent assigned to perform it, or when different agents have au- agent, who performs the action as part of adoption of the
thority to perform related taskwork and need to communicate delegating agent’s goal. [5] describes adjustments to this agree-
information about the work environment with one another.
963
TABLE I
ATTRIBUTES FOR THE H UMAN P ERFORMANCE AGENT R ELEVANT FOR TASK S HARING
Attribute Description
Proactive delegation Defines the number of actions being executed that prompts the agent to delegate; this attribute allows
threshold the agent to maintain some spare capacity
Delegation status Defines whether the agent delegates never, always, or at/above the proactive delegation threshold
Multi-tasking capac- Defines how many actions that the agent can execute at the same time; if the agent never delegates,
ity actions are delayed when the number of actions the agent is executing equals the capacity
Time to delegate Defines the length of time it takes to delegate an action.
Time to monitor Defines the length of time it takes to monitor that the adopting agent is taking action.
Actions that cannot Defines actions that the agent cannot delegate (generally used for perceptual actions and actions for
be delegated which there is no infrastructure for another agent to execute)
Actions that do not
Defines actions that are not considered as taking up multi-tasking capacity
impact capacity
If the agent is below its multi-tasking capacity and the model, the attributes of the agent (as described in Table I)
proactive delegation threshold, the agent immediately executes are also defined and modified in a script. For each individual
and adds an assigned action to the list of executing actions. run of a given scenario, an analyst can set inline the values
Upon addition of this new executing action, the agent calcu- of the proactive delegation threshold, multitasking capacity,
lates the earliest completion time of any action(s) in the list of time to delegate, time to monitor, and delegation status of each
executing actions and schedules the removal of those action(s) instantiated human performance agent model. Similarly, for an
at the calculated time. When this removal occurs, the agent individual run, the analyst can specify which actions can be
removes all actions from the list of executing actions whose delegated and/or ignored for capacity.
completion time is before or at the current time. 4) Logging: WMC provides capability for logging all activ-
An action assigned to and processed by the agent can be ity of instantiated agent models. For each human performance
identified as one that can be delegated or not. If the agent is agent model, an action trace logs each action executed by the
at or below its capacity, is at or above the proactive delegation agent, the time the agent begins execution of each action, the
threshold, and an incoming action cannot be delegated, then duration of each executed action, the number of times each
the agent executes the action. If the incoming action can be action has been delayed (if delayed), and the initial time that
delegated, the agent delegates by searching a list of available each action was delayed (if delayed). Additionally, each human
agents, selecting an adopting agent, and changing the agent performance agent model is associated with a log of delegated
executing the action from itself to the adopting agent. actions, which records the time at which an agent delegates
If the action can be delegated, it may be further identified an action, the action that is delegated, and the adopting agent
by the agent as one whose delegation requires communication who is delegated execution of the action.
of the delegation or not, as well as one whose delegation
requires monitoring or not. If the delegated action requires B. Case Studies
a communication of the delegation, then the agent adds a A set of case studies demonstrate how the human perfor-
communication action to its list of executing actions and mance agent model for WMC, described in Section III-A2,
begins executing the communication (along with the adopting can be used to describe how the performance of taskwork and
agent) which completes before the adopting agent begins teamwork amongst teams of agents is impacted by different
executing the delegated action. If the delegated action requires settings of the attributes of the human performance agent
monitoring, then the agent executes a monitoring action (which model from Table I. For all cases, we fix the time to delegate,
does not count as a unit of capacity in the agent’s list of while varying the multitasking capacity, proactive delegation
executing actions) which begins according to the agent’s time threshold, delegation status, and time to monitor.
to monitor (Table I) before completion of the delegated action 1) Scenario: The case studies concern a scenario used in
by the adopting agent. [1] to represent an air transportation concept of operation, in
3) Scripts: A script in WMC configures agents, actions, which work functions for navigation and spacing (by interval
and resources for specific simulation runs, corresponding to management) of aircraft in terminal airspace of Amsterdam
different cases of a given scenario. Within a script, workmodels Schiphol Airport are shared between flight crews and air traffic
are instantiated to define the actions and resources describing controllers. Three aircraft on different arrival routes are merged
the work for a given concept of operation, and agents are into a single stream for descent and final approach. One aircraft
instantiated and assigned actions to execute from a workmodel. is designated as the lead aircraft and follows an optimized
If an agent is instantiated as a human performance agent profile descent (OPD). The remaining two then follow the lead
964
aircraft in sequence, maintaining a 60-second interval behind taskwork actions thus proceeds by “capability,” such that the
the aircraft in front. See [1] for more details. ATC agent delegates taskwork concerning a particular aircraft
2) Actions: The following 15 actions describe the taskwork to its respective flight crew.
associated with the case study scenario, categorized into the 4) Independent Variables: For all case studies, the ATC
functions describing the work. These actions are a subset of agent is initially assigned all taskwork actions in Sec-
actions described in [1]; actions not included here are treated as tion III-B2. The cases are then configured according to the
actions that do not impact multi-tasking capacity (from Table I) following attributes in Table I for the ATC agent:
of a human performance agent model. Aircraft are modeled • Multi-tasking capacity, set to infinite, or set to a fixed
with a non-linear 6DOF model and the properties of a Boeing number of tasks
747, and are individually associated with all listed actions. • Proactive delegation threshold, set to a level one task
• Vertical Path Management below the multi-tasking capacity
– Clear for descent • Delegation status, set to never delegate, or delegate at or
– Start descent above the proactive delegation threshold
– Set flaps and speedbrakes • Time to monitor, set to never monitor or a fixed time
965
on ATC when delegating with communication only and with
communication and monitoring.
6) Cases: The case studies focus on a period of peak
activity for the ATC agent (100 to 102-second interval across
all simulation runs) and are differentiated across the attributes
of the ATC agent from Table I as described in Section III-B4.
In cases where ATC delegates taskwork (delegation status is
“delegate at or above the proactive delegation threshold”) , all
taskwork actions described in Section III-B2 are eligible to be
both delegated and accompanied by a communication action
for the delegation assignment. In cases where monitoring of
delegated taskwork is specified (time to monitor is a fixed
Fig. 1. Histogram of ATC Taskwork, Baseline Case
time), all taskwork actions from Section III-B2 again are
eligible to be monitored.
a) Baseline Case: For this case, the multi-tasking ca- the third, fourth, sixth and seventh columns in Figure 2). This
pacity of the ATC agent is set to infinite, or equivalently, to a reduced taskwork for ATC shifts to the flight crews (Figure 3).
sufficiently high level so that the agent is able to execute all However, since the ATC agent must communicate the dele-
assigned taskwork actions instantaneously and without need gation of this taskwork, the cost of delegation is realized in
for delay or delegation. an identical number of total actions executed by ATC (see
b) Delay Case: The multi-tasking capacity on the ATC Delegate/Comm 7 and Delegate/Comm 4 columns in Figure 2).
agent is set to a fixed number of actions (4 or 7 simultaneous When the ATC monitors delegated taskwork in addition to
actions) such that the agent will not always be able to execute communicating delegations, the reduced taskwork for the ATC
all assigned taskwork actions instantaneously. The agent’s agent is coupled with additional teamwork, yielding the largest
delegation status is set to “never,” so the ATC agent only delays numbers of total actions executed and lowest percentages of
the execution of taskwork. idle time across all cases (Table II).
c) Delegation with Communication: As in the delay B. Delayed Actions
case, the ATC agent operates at two levels of multi-tasking
capacity (4 or 7 simultaneous actions), but the agent’s dele- Table III provides descriptive statistics for individual de-
gation status is now set to “delegate at or above the proactive layed actions by the ATC agent for each case where actions
delegation threshold” where the threshold (delegate at 3 or 6 are delayed. Both the number of times an individual action is
simultaneous actions) is set to accommodate the communica- delayed, and the time the action is delayed, will be expectedly
tion action. The ATC agent can also still delay the execution greater when the ATC multi-tasking capacity decreases. When
of taskwork actions, when the agent is at capacity. the ATC agent can delegate taskwork to the flight crews instead
d) Delegation with Communication and Monitoring: of executing on its own, both the number of times and overall
This case is identical to the delegation with communication time an individual action is delayed also appear to decrease.
case, only that the agent’s time to monitor is a fixed time, However, these gains need to be weighed against the additional
instead of to never monitor. work from communication and monitoring.
966
TABLE II
ATC AGENT ACTION EXECUTION AND UTILIZATION DURING THE PEAK ACTIVITY PERIOD , ALL CASES
No.ofActions Percentageoftime
Case Total Taskwork Coordination Delegated Delayed Idle AtCapacity
Baseline 10 10 0 0 0 88.0
DelayCap7 10 10 0 0 2 88.0 1.0
DelayCap4 10 10 0 0 5 87.0 4.5
Delegate/CommCap7 10 8 2 2 1 87.0 1.0
Delegate/CommCap4 10 6 4 4 2 86.5 3.0
Delegate/Comm/MonitorCap7 12 8 4 2 1 86.0 1.0
Delegate/Comm/MonitorCap4 14 6 8 4 2 85.5 3.0
TABLE III
D ESCRIPTIVE STATSTICS OF INDIVIDUAL DELAYED ACTIONS BY THE ATC AGENT, FOR CASES WITH DELAY
No.oftimesdelayed Delaytime(s)
Case Maximum Median Maximum Mean
DelayCap7 1 1 0.010 0.010
DelayCap4 3 2 0.040 0.024
Delegate/CommCap7 1 1 0.010 0.010
Delegate/CommCap4 1 1 0.020 0.015
Delegate/Comm/MonitorCap7 1 1 0.010 0.010
Delegate/Comm/MonitorCap4 1 1 0.020 0.015
TABLE IV
D ISTRIBUTION OF TOTAL WORK REQUIRED ( IN TOTAL TIME ) ACROSS ALL AGENTS , FOR CASES WITH DELEGATION
ATCTotalRequiredWorkTime(s) FlightCrewsTotalRequiredWorkTime(s)
Case Taskwork Coordination Taskwork Coordination
Delegate/CommCap7 0.45 0.04 0.17 0.04
Delegate/CommCap4 0.41 0.08 0.21 0.08
Delegate/Comm/MonitorCap7 0.45 0.08 0.17 0.04
Delegate/Comm/MonitorCap4 0.41 0.16 0.21 0.08
967
Fig. 3. Distribution of ATC and Flight Crew Taskwork and Teamwork
can either execute taskwork immediately, or otherwise de- tization of taskwork [8], and also specify decision-making
lays execution or delegates to others. Using a computational mechanisms for the agent with respect to taskwork the agent
framework for modeling and simulating multi-agent work, we executes, delays, delegates, and interrupts so the model can
have demonstrated the impacts of this agent’s behaviors on the be applied to define strategies for effectively managing issues
distribution of taskwork and teamwork for a given air traffic with autonomy, authority and responsibility across a team of
control scenario. agents.
An analyst can easily generalize the application of this
ACKNOWLEDGMENTS
model to a wider range of air traffic control scenarios due to
the modular architecture of the WMC simulation framework This work is sponsored by the NASA Aviation Safety
[7], [8]. WMC has been constructed such that any workmodel Program with Dr. Guillaume Brat serving as Technical Mon-
representing a given operational concept is independent of any itor under grant number NNX13AB71A S04 (Amy Pritchett,
agent model, and the two components are only linked at run- Principal Investigator). The authors also thank the other WMC
time when the simulation passes each action to its assigned developers for their ongoing mutual support.
agent for execution. By default, an agent model only specifies R EFERENCES
the generic processing and execution of assigned actions, and
[1] M. IJtsma, J. Hoekstra, R. P. Bhattacharyya, and A. Pritchett, “Computa-
not any representation of specific contexts or taskwork. The tional assessment of different air-ground function allocations,” in Eleventh
human performance behaviors described here are added to USA/Europe Air Traffic Management Research and Development Seminar,
these default behaviors with the same intention of generic 2015.
[2] A. R. Pritchett, S. Y. Kim, and K. M. Feigh, “Modeling human-automation
application and compatibility with any workmodel, so that only function allocation,” Journal of Cognitive Engineering and Decision
a different specification of actions and resources is needed to Making, vol. 8, no. 1, pp. 33–51, 2014.
apply the model of task sharing with delegation to different [3] ——, “Measuring human-automation function allocation,” Journal of
Cognitive Engineering and Decision Making, vol. 8, no. 1, pp. 52–77,
scenarios, as have been previously modeled using WMC [2], 2014.
[3]. [4] C. Castelfranchi and R. Falcone, “Towards a theory of delegation for
The agent model presented here has only modeled and agent-based systems,” Robotics and Autonomous Systems, vol. 24, pp.
141–157, 1998.
tested the delegation of taskwork from the side of the delegat- [5] R. Falcone and C. Castelfranchi, “The human in the loop of a delegated
ing agent, without considering how a potential adopting agent agent: The theory of adjustable social autonomy,” IEEE Transactions on
might respond to a delegation assignment. Extensions of the Systems, Man, and Cybernetics, Part A: Systems and Humans, vol. 31,
no. 5, pp. 406–418, 2001.
model should describe how a potential adopting agent makes [6] D. W. Lee and E. J. Bass, “Delegation for authority and autonomy:
decisions to accept or reject a delegation, as well as additional An assignment and coordination model,” in 2014 IEEE International
forms of monitoring and reporting between delegating and Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2014,
pp. 1744–1751.
adopting agents for a given delegation assignment. [7] A. R. Pritchett, K. M. Feigh, S. Y. Kim, and S. Kannan, “Work models
The current implementation of the model also only illus- that compute to support the design of multi-agent concepts of operation,”
trates some of the trade-offs in delaying taskwork against Journal of Aerospace Information Systems, vol. 11, no. 10, pp. 610–622,
2014.
delegating to other agents, as well as only a subset of potential [8] A. R. Pritchett, K. M. Feigh, S. Mamessier, and G. Gelman, “Generic
agent behaviors for managing taskwork. Future work should agent models for simulations of concepts of operation,” Journal of
incorporate behaviors, such as the interruption and priori- Aerospace Information Systems, vol. 11, no. 10, pp. 623–631, 2014.
968