APL PROPRIETARY
APL PROPRIETARY
Background: The Chief Digital and Artificial Intelligence Officer, in partnership
with the Johns Hopkins Applied Physics Lab, Air Force Research Lab and Air Force
Lifecycle Management Center, is rapidly and iteratively modifying existing
unmanned airborne platforms to develop an AI-enabled platform that can
perceive its environment, execute surveillance and reconnaissance missions, and
communicate targeting data to other platforms – autonomously
The goal of Smart Sensor is to rapidly develop and deploy an AI-enabled
autonomous “brain” (software and computer hardware) that can be put on any
unmanned platform based on available size, weight, and power
User Engagement:
Notional design iterations driven via user feedback given statistical
significance:
Critical Challenges:
(1) Achieve ISR mission resilience in a denied,
degraded, or communications limited state
through robust AI/Autonomy
(2) Create effective AI/Human shared
situational awareness both “in-mission”
and “post-mission”
(3) Increase speed and accuracy of multi-
sensor analysis despite increased system
complexity
(4) Decrease processing time while increasing
analytical accuracy/precision
(5) Achieve explainability and trust of
AI/Autonomy (regardless of Human-In-
The-Loop, Human-On-The-Loop, or
Human-Out-Of-The-Loop)
Approach: We engaged with users early and often in order to drive Human-
Centered UI Design, identify requirements, and understand mission goals for the
relevant stakeholders
Visit End-User, Sample Size Objective
March AFB Sensor Operators, Pilots, Mission Intelligence
Coordinator, Squadron Intelligence
[n=17]
- Identify potential users, associated
tasking and informational needs
- Validate goal-directed task analysis and
associate task/need with User roles
- Obtain user feedback on design concept
COCOM PED
Organizations
PED Organizations
[n=11]
- Identify reporting informational needs
MCB Camp
Lejeune
PED Cell (All Source, Geospatial Intelligence,
Targeting)
[n=22]
- Identify specific user workflow from start
to end of mission; Explore level of trust
associated with automation level on tasks
- Validate task analysis and drafted
workflow
- Obtain user feedback on wireframe
Langley AFB &
Holloman AFB
PED Cell (Screener, Geospatial Analyst, Imagery
Mission Supervisor), Pilot and Sensor Operator
[n=33]
- Formulate comprehensive understanding
of workflow between varying user types
- Obtain user feedback on prototype
1. Operator intel
report output
Nov 2021
2. Design concept
wireframe
Feb 2022
3. Interactive
prototype mockup
Jun 2022
Goal-Directed Task Analysis
(GDTA) provide a common
understanding state of user tasks,
decisions points, and information
need to complete decisions
Interviews & Qualitative surveys to
collect knowledge, skills, abilities (KSAs),
training and demographic information to gain
common understanding of current end-user
roles and to envision how future end-user role
will change over time with the integration of
Smart Sensor
Design Thinking Sessions
informed us to how end-users
currently carry out operations
and what information is needed
to optimally perform their tasks
CDAO
Rapid Prototyping driven by
User feedback
Iterative and rapid prototyping to
provide users with UI visual context
necessary for constructive design
feedback
LCDR Joseph Geeseman,
PhD, Smart Sensor Program Manager
Heather James,
Senior Human Systems Engineer
General Findings & Key Concepts:
Determined perceived trust and comfort level associated with level of automation on
operational tasks throughout a mission
• Identified which operational tasks users preferred to maintain manual input
and the preferred level of automation for other tasks
• Explored explainability of machine decisions required to promote operator
trust
Determined mission-critical tasks and associated informational need for relevant user groups
• Role-based access control (RBAC) with information layout/segregation
differentiation between roles to support SA and decision making
• Determined actionable intelligence items and ways to display grouped
information to support decision making
Human Centered Design concepts driven by iterative user engagements
• Determined design features to enhance likelihood of machine error detections
and provided means for human intervention to achieve mission success
• Rapid retraining AI/ML algorithms via edge-processing with privileged RBAC
Introduction of competency measure (i.e., how competent is machine on the given task) based
on traditional confidence associated with probability while accounting for machine training on
specified given task
Other Techniques includes
• User Profiling
• Workflow Diagrams
• MBSE OV-5b Diagrams
• User stories
Controlled by: Chief Digital & Artificial Intelligence Office
Controlled by: LCDR Joseph Geeseman
DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited (13 June 2022).
POC: LCDR Joseph Geeseman, joseph.w.geeseman.mil@us.navy.mil
Mei Y. Lau,
PhD, Senior Human Systems Engineer
Autonomous System Platform > Smart Sensor
Brain Evidence Autonomy Management Interface Design Approach
