SYSAI for System Health Management - a Statistical Framework for
the Analysis of Diagnosis Systems
Yuning He
1
and Johann Schumann
2
1
NASA Ames Research Center, Moffett Field 94035 CA, USA
yuning.he@nasa.gov
2
KBR LLC, NASA Ames Research Center, Moffett Field 94035 CA, USA
johann.schumann@nasa.gov
ABSTRACT
On-board failure diagnosis and health management systems
(HMS) are crucial for the operation of complex autonomous
aerospace systems. False alarms (false positives, FPs) or false
negatives (FNs) can lead to lower system performance or
even loss of mission or the autonomous vehicle. Therefore, a
careful verification and validation (V&V) is important. Due to
the high dimensionality of the system’s state space, however,
exhaustive testing of the HMS is usually not possible.
In this paper, we present how our SYSAI (System Analy-
sis for Systems with AI components) framework can support
intelligent analysis and testing of HMS on the system level.
SYSAI’s capabilities to efficiently explore high-dimensional
state and parameter spaces and to identify diagnosability re-
gions and their boundaries, makes a comprehensive analysis of
the diagnosis system possible and can provide feedback to the
designer. We will illustrate our approach using the ADAPT
(Advanced Diagnostics and Prognostics Testbed) redundant
power storage and distribution system.
1. INTRODU CTION
A Health Management Systems (HMS) on board an autonomous
vehicle has to continuously monitor the system’s components
and behavior to detect anomalies and to identify and diagnose
faults.
For systems with a high degree of autonomy, the on-board
estimation of system health is extremely important. Only then,
the autonomous system obtains knowledge about its current
health status and capabilities. HMS therefore are key to sup-
port autonomous decision-making and contingency planning
to ensure that the mission can be executed safely and success-
fully even in the presence of adverse events.
Yuning He et al. This is an open-access article distributed under the terms of
the Creative Commons Attribution 3.0 United States License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the
original author and source are credited.
Numerous different approaches for fault detection, diagnosis,
and system health management have been developed
(
Abid,
Khan, & Iqbal, 2021; Gertler, 2021). They use vastly different
techniques and algorithms but share one commonality: un-
detected or misdetected faults can lead mission failure and
potential loss of the autonomous system. Unnecessary alarms
(false positives) can hamper mission success but might have
more severe consequences as well. In many cases, such sit-
uations comprise a safety risk, which even could jeopardize
human life.
Therefore, the Health Management System of an autonomous
vehicle need to be considered a safety-critical component,
requiring careful design, verification and validation (V&V)
and possibly certification.
Formal-methods-based approaches, like model-checking can
be used for the verification of the discrete fault detection and
diagnosis components
(
Cimatti, Pecheur, & Cavada, 2003).
Realistic testing of the entire HMS in conjunction with the
vehicle itself, as it is done in scenario-based testing faces large
and high-dimensional search spaces that need to be explored
during testing; exhaustive testing is not possible.
In this paper, we present, how our SYSAI (System Analysis
using Statistical AI) framework can support the V&V of an
on-board health management system. SYSAI
(
He, 2015; He
& Schumann, 2020; He, Yu, Brat, & Davies, 2022) has been
designed for the concise analysis of complex systems with
AI components. It executes the system under test (SuT), the
entire autonomous system with its environment or just a single
component in a parametric way. The use of advanced surro-
gate models and active learning allows SYSAI to efficiently
explore high-dimensional state and scenario spaces while au-
tomatically focusing on relevant regions like failure regions
and their boundaries.
In this paper, we describe, how SYSAI can support automatic
scenario testing of a HMS within its autonomous vehicle and
operational environment, i.e., on the system level. This in-
1
