DESIGNING MULTIPLE UAV DISPLAYS BASED ON THE HUMAN CAPACITY TO PROCESS INFORMATION

PRESENTED BY: Dr. William Marshak Jeff S. Collier SYTRONICS, Incorporated Dayton OH PRESENTED FOR: Cognitive Engineering Research Institutes Fourth Annual Human Factors of UAVs Workshop Manning the Unmanned 21-24 May 2007

Contents
Project Background Human Cognitive Operating Characteristic Multiple UCAV/UAV Cognitive Task Demands Design Strategies
Voice Input/Output Using Context Agent Mediated Shallow Information Access Temporal Mission Event Management Distinguishing Plan from Hypothetical Replanning Recognition Primed Decision-making (RPD) Display Visual Thinking / External Cognition (VT/EC)

Summary
SYTRONICS, Incorporated 2

Multiple UAV AGEncy (MAGE)

Tactical Situation System Display

Decision Support System Display

Sandbox System Display

SYTRONICS, Incorporated

Purpose: Develop means for integrating net-centric information to facilitate multiple UCAV/UAV control Employs intelligent agent software and theory-based design strategy Transition to current systems and systems in development Sponsor: Air Force Research Lab / Human Effectiveness Directorate (Dr. Mark Draper & Ms. Gloria Calhoun)
3

Human Cognitive Operating Characteristic

Perception bandwidth
Vision-high, Audition-medium, Touch/Taste/Position-low

Attention driven sampling from sensory stream Limited Short-Term Memory (STM) Capacity (Miller's 72) Large Long-Term Memory (LTM) Capacity (if consolidated) Communication occurs within common cognitive framework Impedance matching between user and interface facilitates information flow
Matching includes perception and cognitive (way we think)
SYTRONICS, Incorporated 4

Multiple UCAV/UAV Cognitive Task Demands Vehicle states

Vehicle environment

Internal status of propulsion, sensors, communications, etc. Consumables status: fuel, power Flight characteristics, traffic deconfliction Weather Threats Ground engagement (surveillance, search, weapons strike) Coordination (C3, Ground/JTAC, Intel, Air, ATC) Lost Link SYTRONICS, Incorporated

Mission demands

Voice Input/Output with Context

Voice input using Nuance; speaker independent, vocabulary limited system
Switch actuated recognition made at any time

Voice output using Microsoft Speech SDK

Text-To-Speech (TTS) speaks answers to queries

Context manager maintains elements of previous conversation, filling in omissions in subsequent queries
Speech suitable responses delivered by TTS Non-verbal responses available in Decision Support System

Context correspondence maintained by selective echoing

Ubiquitous way people maintain context
SYTRONICS, Incorporated 6

Agent Mediated Shallow Info Access

Information requirements driven by both mission event and operator MAGE queries internal and netcentric sources to gather needed information Most important information loaded into workspace in expandable window Other information organized under tasks and accessible with one click Avoids multi-layered menus Reminds operator about relevance Operator queries supplement agent generated information
SYTRONICS, Incorporated 7

Temporal Mission Event Management

One strategy for workload management is avoiding events from happening at the same time Identifying time conflict among mission events requires a temporal display Display includes time line for each vehicle, now indicator, typed event strategy Automatic or manual scrolling through mission time

SYTRONICS, Incorporated

Distinguishing Mission from Hypothetical Replanning

Tactical Display

Sandbox Display

How to replan without losing awareness of the current executing plan Separate Tactical Situation and hypothetical Sandbox displays Separates reality from hypothetical, but allows side-byside comparison Includes spatial and time line renditions of the mission plans

SYTRONICS, Incorporated

Recognition Primed Decision-Making Display

Based on Klein (1989) observation experts recognize what to do MAGE agency detects need for intervention Assembles a Decision Support Template to aid and accelerate decision process Instigation and decision alternatives presented Workspace allows recognition of the solution Shallow menu and voice query system make more information available MAGE operator picks the most desirable solution Enables better and faster decisions in high workload environment
SYTRONICS, Incorporated 10

Decision Support Template

SYTRONICS, Incorporated

11

Visual Thinking / External Cognition (VT/EC)

Arnheim (1969) studies artists creative thinking and postulates higher cognitive function evolved from vision

Examples include nomographs and DSTs

Visual representations consistent with visual processing are easier to understand, mimic higher order reasoning Blink and RPD may be manifestations of Visual Thinking Not only provide solutions, but reveal process of solution

McKim (1972) proposes concrete principals governing VT McEachren (1995) applies concept to maps Card, S. K., Mackinlay, J. D., and Shneiderman, B. (1999) rediscover VT/EC Expand visual thinking to other senses broader term of External Cognition
SYTRONICS, Incorporated 12

Complex Judgements by Visual Thinking

SYTRONICS, Incorporated

13

Evaluation of the VT Display

Brief study comparing bar graph representation of alternative mission metrics with VT Widget (VTW) Observers taught criterion for judging 3 alternatives on 7 dimensions, including minimum acceptability Then judged sets of 12 trios of mission alternatives either in a block of tables, bar graphs or in a block single VTWs Choices were to rank three alternatives, detect if all three violate required minimums, and decision time recorded Deviations from mathematical ideals, correct rejections of minimum violations, and decision times were compared between blocks in a dependent t-test

SYTRONICS, Incorporated

14

Summary
Cognitive Systems Engineering of human cognitive operating characteristic and system displays reduces impedance and increases bandwidth This is accomplished by Voice input/output with Context Agent facilitated shallow information access Temporal mission event management Recognition primed decision-making aiding display Visual Thinking / External Cognition displays MAGE integrates net-centric feeds and multiple vehicle control, overcoming much of the workload overload
SYTRONICS, Incorporated 15