CHRISTOPHER A. MOUTON, CALEB LUCAS, ELLA GUEST
The Operational Risks
of AI in Large-Scale
Biological Attacks
A Red-Team Approach
T
he rapid advancement of artificial intelligence (AI) has far-reaching implications across
multiple domains, including its potential to be applied in the development of advanced
biological weapons. This application raises particular concerns because it is accessible to
nonstate entities and individuals. The speed at which AI technologies are evolving often
surpasses the capacity of government regulatory oversight, leading to a notable gap in existing poli-
cies and regulations.
The coronavirus disease 2019 (COVID-19) pandemic serves as a pertinent example of the dev-
astating impact that even a moderate pandemic can have on global systems.
1
Further exacerbating
this issue is the economic
imbalance between offense
and defense in biotechnol-
ogy. For instance, the mar-
ginal cost to resurrect a
dangerous virus similar to
smallpox can be as little as
$100,000,
2
while develop-
ing a complex vaccine can
be over $1 billion.
3
Previ-
ous attempts to weaponize
biological agents, such as
Aum Shinrikyo’s endeavor
with botulinum toxin,
failed because of a lack
of understanding of the
bacterium.
4
However, the
existing advancements in
C O R P O R A T I O N
KEY FINDINGS
■ In our experiments to date, large language models (LLMs) have not generated
explicit instructions for creating biological weapons. However, LLMs did offer
guidance that could assist in the planning and execution of a biological attack.
■ In a fictional plague pandemic scenario, the LLM discussed, for example,
biological weapon–induced pandemics, identifying potential agents, and
considering budget and success factors. The LLM assessed the practical
aspects of obtaining and distributing Yersinia pestis–infected specimens
while identifying the variables that could affect the projected death toll.
■ In another fictional scenario, the LLM discussed foodborne and aerosol
delivery methods of botulinum toxin, noting risks and expertise requirements.
The LLM suggested aerosol devices as a method and proposed a cover
story for acquiring Clostridium botulinum while appearing to conduct legiti-
mate research.
■ These initial findings do not yet provide a full understanding of the real-
world operational impact of LLMs on biological weapon attack planning. Our
ongoing research aims to assess what these outputs mean operationally for
enabling nonstate actors. The final report on this research will clarify whether
LLM-generated text enhances the effectiveness and likelihood of a malicious
actor causing widespread harm or is similar to the existing level of risk posed
by harmful information already accessible on the internet.
Research Report
