Collaborative Time-Sensitive Targeting Mary Missy Cummings Stacey Scott

Collaborative Time-Sensitive Targeting Mary (Missy) Cummings, Stacey Scott Humans and Automation Laboratory http: //halab. mit. edu {missyc, sdscott}@mit. edu

Current HAL Research Projects Research in the Humans and Automation Lab (HAL) focuses on the multifaceted interactions of human and computer decision-making in complex sociotechnical systems. • Time-Sensitive Operations for Distributed Teams • Measurement of Display Complexity • Collaborative Human Computer Decision Making • Integrated Sensor Decision Support • Reduced Manning for Unmanned Vehicle Teams • Single Operator Control of Multiple Unmanned Aerial Vehicles

Human Supervisory Control in NCO

Supervisory Control & Teams Supporting Supervisors of Teams of HSC Operators Supporting the Control of Teams of Heterogeneous Unmanned Vehicles

Inverting Current UAV Control Schemes • How do operators manage multiple supervisory control processes? • What effects do human performance limitations have on the overall system? • Reduced Manning • Combining Shadow positions • Multiple UV control for a single operator • Distributed control Multi-Modal Watch Station

Collaborative Decision Making • Tomahawk Strike Planning: missions and missiles • Determine how humans and computer optimization algorithms can complement each other • Cognitive strategies

Aiding Human-Computer Path Planning • Decision Support for Planetary Exploration • Astronaut traversal replanning task • Path replanning with increasingly complex constraints • Affect of level of automated decision support

Goal of this Research Project To identify or develop technologies to facilitate collaborative decision-making in the time-sensitive targeting task domain

Time-Sensitive Targeting is Highly Collaborative

Teamwork in Network-Centric Operations • Highly connected operators, often working remotely • Heavy reliance on commercially available collaboration technologies - email - instant messaging (‘chat’) - video & desktop conferencing

Benefits of Collaboration Technologies • Facilitate collaboration with remote colleagues • Facilitate sharing of and access to digital media during group work (in both co-located and distributed settings) • Provide opportunity and support for casual interactions • Promote ad hoc collaboration & communication • Enhance team cohesion

Disadvantages of Collaboration Technologies • Require explicit effort to access / share information - implicit or more subtle communication / interaction is often missing - lack of non-verbal cues • Typically convey information about completed activities, ongoing progress/status information is also important: - Has anyone seen my request for help? - Is someone working on the problem? What’s their status? • Associated costs with using collaboration technologies: - distraction - time & mental effort of asking for or sharing information

Enhancing Teamwork in Non-Military Domains • Research in the CSCW domain has been focused on providing technological support for teamwork in many collaborative settings: - corporate, educational, research • Reviews of the CSCW literature has revealed common threads across many new approaches to CTs: - Minimizing overhead costs of sharing information about remote collaborators - Increased use of automation to help provide this information • Many approaches are aimed at supporting the shared activity process

Supporting the Shared Activity Process • Focused on providing team member’s activity awareness (Carroll et al. , 2003): - awareness of project work that supports group performance in complex tasks - implies an awareness of other people’s plans and understandings Project View IM (Scupelli et al. , 2005) Virtual School Project Timeline (Carroll et al. , 2003) IBM’s Activity Explorer (Millen et al. , 2005)

Supporting the Shared Activity Process (cont’d) Approaches for supporting activity awareness: • Feedthrough of action • Visibility of action Project View IM (Scupelli et al. , 2005) Scupelli, P. , Kiesler, S. , Fussell, S. R. , & Chen, C. (2005). Project View IM: A Tool for Juggling Multiple Projects and Teams. In Extended Abstracts of CHI 2005: ACM Conference on Human Factors in Computing Systems, pp. 1773 -1776.

Supporting the Shared Activity Process (cont’d) Approaches for supporting activity awareness: • Feedthrough of action • Visibility of action Virtual School Project Timeline (Carroll et al. , 2003) • Important to situate this information within the context of the shared activity process Carroll, J. M. , Neale, D. C. , Isenhour, P. L. , Rosson, M. B. & Mc. Crickard, D. S. (2003). Notification and Awareness: Synchronizing task-oriented collaborative activity. International Journal of Human-Computer Studies, 58(5), pp. 605 -632.

Challenges Introduced by the TST Domain • Determining what constitutes relevant activities when ‘shared objects’ are not necessarily ‘files’ • Reducing the cost of providing activity-related information to team members • Balancing awareness and information overload • Balancing individual performance and team performance

Methodology for Developing Activity Awareness Support for Collaborative TST Operations • Develop representative collaborative TST task scenario • Conduct task analysis on scenario to develop collaborative technology requirements, focusing on the shared activity process • Develop initial team display designs for collaborative TST scenario incorporating CT requirements • Implement team display designs • Develop evaluation protocol and teamwork metrics for evaluation of team displays • Design and build team testing environment • Evaluate team displays • Iterate team display designs based on study results

Representative Task Scenario: GTT Task Generalized Team TST (GTT) Task • A convoy is scheduled to pass through a hostile region • Team of UAV operators must protect convoy until convoy exits region • Stakeholders: - Convoy UAV Operations Team + UAVs Strike Team Intelligence Enemy

Activity Awareness Information Analysis for GTT Task Scenario: Inter-team Interaction report to upper command TST alert intel requests BDA strike team engages controls TST BDA TST ISR orders engagement ISR BDA detects location UAV Operations Team TST ISR TST BDA convoy location target pinpoint Enemy (TSTs) 1 A 2 A 3 A 1 B 2 B 3 B 1 C 2 C 3 C 1 D 2 D 3 D UAVs ISR detects convoy

Activity Awareness Information Analysis for GTT Task Scenario: Intra-team Interaction TST alerts (convoy) TST ISR (intel) TST BDA (intel) convoy location (intel) BDA (strike team) TST ISR (UAVs) TST BDA (UAVs) UAV Operations Team ta BD A a rg ssig et al nme lo ca nts tio n COM UAV Ops request engagement authorize engagement situation update m. CDR ISR – STR – BDA report – control report to command controls orders engagement requests BDA

Activity Awareness Requirements for the GTT Task Scenario • Types of intra-team shared activity information - relevance to current tasks / team member - available task resources - priority of current task activity - operators’ availability (progress, workload, tasking availability) - operators’ interruptibility (status, cognitive availability) • This information helps team members address the questions: - when / how can I help somebody else? - when / how can I be helped? - how do my actions impact others’ activities?

Relevant Activity Awareness Information is Role Dependent • COM and m. CDR need status and workload information of the UAV Ops to ensure individuals and team are not overwhelmed - m. CDR can help UAV Ops prioritize task activities or request support from external resources - helps COM distributed TST and BDA assignments among UAV Ops • UAV Ops need operator & resource availability and interruptibility information - helps minimize disruptions when someone is involved in critical tasking - helps operators know who can feasibly help them, or who might need their help

Supporting Team Supervision: Status and Workload Information on a Situation Display for Battlefield Context

Supporting Team Supervision: Status and Workload Information on a Team Status Display CONVOY 5% Z 1 OK Z 1 Z 2 Z 3 UAV OP 1 OP 2 OP 3 total 1 A 1 B ISR route: 10% total: 20% 25% 50% 10% 90% 15% 53% 1 C targets 1 D ISR 5 STR 3 BDA: 2 UAV OP 2 2 A 2 B 2 C Comms Connectivity Status 2 D SATELLITE AWACS UAV OP 3 3 A 3 B UAV TEAM 3 C 3 D STRIKE TEAM

Ongoing Work: Display Design & Implementation • Iterating on the status and workload information displays • Developing activity awareness displays to integrate into our existing UAV Op simulation environment, MAUVE, that provides operator & resource availability and interruptibility information MAUVE Tactical Display MAUVE Decision Support Display

Ongoing: Developing Team Testing Lab HAL Complex Operation Center • 3 Large-screen displays (A) • 4 Reconfigurable operator stations (B) • 2 Mobile operator stations (tablet. PCs & handhelds) (C) • Wireless headsets with microphones for ‘always on’ audio via Internet software

Project Summary • Enhancing activity awareness in collaborative TST environments seems to help promise for helping teams coordinate their actions and help reduce the costs of collaborating via technology • Overarching goal of this project is to understand the tradeoffs between supporting individual operator performance and team performance to provide overall mission effectiveness during TST operations

Thanks! This research is funded by: Boeing Phantom Works & Air Force Research Lab For more information: http: //halab. mit. edu or sdscott@mit. edu