13th International Conference on Naturalistic Decision Making 2017, Bath, UK
Quenching the Thirst for Human-Machine Teaming
Guidance: Helping Military Systems Acquisition Leverage
Cognitive Engineering Research
Patricia L. McDermott
a
, Katherine E. Walker
a
, Cynthia O. Dominguez, Ph.D.
a
, Alex Nelson
b
and Nicholas Kasdaglis, Ph.D.
a
a
The MITRE Corporation
b
Air Force Research Laboratory
ABSTRACT
There is a need in systems acquisition that is not currently being met. The insights from cognitive
engineering research in human-automation interaction are not being systematically applied to
acquisition processes associated with operational military systems. To address this gap, we
synthesized guidance from the literature and translated it into a set of general cognitive interface
requirements for human-machine teaming. By presenting the guidance as requirements, we are
attempting to remove barriers from effective insights being used in implementation. This paper
describes ten themes of human-machine teaming that need to be supported: Observability,
Predictability, Directing Attention, Exploring the Solution Space, Directability, Adaptability,
Common Ground, Calibrated Trust, Design Process, and Information Presentation. Example
requirements are provided for Exploring the Solution Space. The general set of requirements can be
tailored to specific systems as needed. To support this tailoring, we are developing and piloting
cognitive task analysis techniques focused on human-machine teaming.
KEYWORDS
Coordination; military; cognitive engineering; common ground; systems development; human automation
interaction
INTRODUCTION
The cognitive engineering (CE) communitity has produced useful research findings and related insights for
designing effective collaboration between automation and humans. These findings and insights are not making
their way into the design of operational systems. We are finding a general lack of familiarity and application of
these findings among program managers, systems engineers and developers. There is no systematic process for
applying this human-automation research when engineering complex human-machine systems. Users and
designers talk about concepts like “transparency” but it is not clear what this means in terms of design requirements
for a system. The systems acquisition community may not be familiar with the CE literature or they may not have
the necessary background to apply the findings to envisioned or upgraded systems. Regardless, a gap exists
between the CE research and the system accquistion process for fielding new systems.
The lack of CE insight in highly autonomous systems is apparent. So-called “clumsy” automation has many
negative consequences, such as brittle performance, miscalibrated trust in the system, and lack of user acceptance
(Wiener, 1989; Sarter, Woods, & Billings, 1997; Lee & See, 2004; Parasuraman & Riley, 1997). Although the
research community has answered the call to provide guidance on how to improve interactions between humans
and advanced technological systems (Lyons, 2013), surprisingly little work has been done to translate the literature
into materials that non-researchers can use as guidance for design of highly automated and autonomous systems.
Our analysis has aggregated and packaged the human-machine teaming (HMT) literature into several
interconnected themes for the design of human-autonomy interaction.
METHODS
Researchers completed three main tasks: literature review, analysis, and creation of general cognitive
interface requirements. First, to leverage and package existing research for system acquision personnel and
program managers, we reviewed the large applied literature on human-automation interaction, human-machine
trust, human-robot teaming, flight deck automation surprises, and other related work. Seventy-six papers were
triaged to identify the papers most likely to yield citable guidance for HMT. We summarized 39 of these papers,
extracting key information on HMT philosophy, principles, key terms, and requirements and considerations (The
authors can be contacted for a list and summary of papers reviewed). The research findings were analyzed for
common themes and elements of cognitive support. The analysis yielded ten themes that capture unique aspects
