PRESIDENTIAL TRANSITION: PRIORITY TOPIC MEMO
JULY 2024
AI RED TEAMING:
ADVANCING SAFE AND SECURE AI SYSTEMS
AI systems are uniquely vulnerable to novel threats, which can
severely impact critical national systems and public services.
These include hallucination, mimicry, and extraction of sensitive
information. Bad actors can exploit these vectors, leading to
identity theft, fraud, loss of life and property, and the erosion of
public trust. To counter adversarial attacks on AI systems, we
must institute recurring AI red teaming, which employs adversarial
thinking to both identify exploitable AI systems’ vulnerabilities and
allow the AI community to counter those threats before they occur.
The Case for Action
AI’s potential as a powerful dual-use technology means its adoption will be
swift and pervasive throughout industry and government. AI’s rapid and
widespread adoption will be paced by the speed of AI’s development and
advancement, which is unprecedented. These factors together will create an
environment where AI systems in use by government and industry possess a
broad “attack surface,” vulnerable to the malign actions of our nation’s
adversaries at the state and non-state levels. The exploitation of these
vulnerabilities affects critical public services and public trust in institutions.
While current industry practices
1,2,3,4
and prior executive action
5,6
have
underscored the importance of AI red teaming, more work must be done to
institutionalize AI red teaming as an indispensable part of AI assurance.
7
The incoming administration should prioritize continuous AI red teaming
efforts during the development of AI applications, during their deployment,
and in their ongoing use, to increase public trust in AI, mitigate the
significant and proven risk of adversarial exploitation, and safeguard the
United States’ critical infrastructure, both civilian and military.
AI’s potential as a
powerful dual-use
technology means
its adoption will be
swift and pervasive
throughout industry
and government.
MITRE’s mission-driven teams are
dedicated to solving problems for a
safer world. Through our public-private
partnerships and federally funded R&D
centers, we work across government and
in partnership with industry to tackle
challenges to the safety, stability, and
well-being of our nation.
mitre.org
