1
Malware Target Recognition of Unknown Threats
Thomas E. Dube, Member, IEEE, Richard A. Raines, Senior Member, IEEE, Michael R. Grimaila, Senior
Member, IEEE, Kenneth W. Bauer, Member, IEEE, and Steven K. Rogers, Fellow, IEEE
Abstract—Organizations traditionally use signature-based
commercial antivirus products as a frontline defense against
malware, but advanced persistent threats craft custom malicious
tools to achieve their objectives. Organizations safeguarding sen-
sitive information have difficulty identifying new malware threats
among millions of benign executables using only signature-
based antivirus systems. This paper extends a performance-
based malware target recognition architecture that currently
uses only static heuristic features. Experimental results show
this architectural component achieves an overall test accuracy
of 98.5% against a malware set collected from operational
environments, while three commercial antivirus products com-
bine for a detection accuracy of only 60% with their most
sensitive settings. Implementations of this architecture will enable
organizations to self-discover new malware threats providing
enhanced situation awareness for cyberspace operators in hostile
threat environments.
Index Terms—malware detection, intrusion detection, an-
tivirus, situation awareness, advanced persistent threat.
I. INTRODUCTION
S
ECURING computer networks of large organizations is
difficult, primarily due to their scale, scope and com-
plexity. Analysts must examine a potentially overwhelming
set of data to discover new malware threats. Attacker tools
and malicious network traffic successfully hide in plain sight
among millions of executable programs and billions of net-
work connections, because organizations cannot effectively
reduce these datasets to manageable levels.
Signature-based commercial antivirus and intrusion de-
tection systems are effective for identifying known threats,
but relatively ineffective against new unknown threats. To
attackers, these systems serve as design constraints when
designing new tools to avoid detection. Research demonstrates
that commercial products are relatively easy to avoid [1].
Advanced Persistent Threats (APTs) as described by Bejtlich
[2] certainly have this capability. Furthermore, these systems
do not appreciably perform useful data reduction to reduce
analyst workload as their detections are typically coarse binary
“yes” or “no” outputs.
Malware detection research provides a potentially viable
method of data reduction of analyst workload. Heuristic analy-
sis techniques generally fall into two distinct categories: static
and dynamic [3]. Static heuristics generally use non-runtime
indicators [3], such as structural anomalies, program disas-
sembly [4] and n-grams [5], [6], [7], [8], [9]. Alternatively,
dynamic heuristics employ runtime indicators [3] normally
obtained in virtual environments, such as commercial sandbox
applications [10], [11], [12].
Patent pending.
Despite the success that static heuristics enjoyed during
the 1990s [3], today’s research heavily favors dynamic (or
behavioral) heuristics [13], [14], [15], [16], [17]. Szor, a well-
known industry expert, speaks to the current decline of static
analysis techniques in his popular book [3]. Moser et al.
presents limitations of static analysis techniques when gen-
erating a disassembly of program instructions [17]. Dynamic
analysis methods suffer from limited operational utility due to
slower runtime speeds than their static counterparts and incom-
pleteness [18]. Their performance makes them operationally
infeasible to test tens of thousands of unique programs on a
single system, unless first using another method to prioritize
workload. Dynamic heuristic analysis is also incomplete, be-
cause no guarantee of observing malicious activity exists.
While neither static or dynamic analysis alone is sufficient,
the two can complement one another (and even commercial
antivirus products) to provide a “full spectrum” defense against
malware with reduced effective scan and detection time. Con-
ceptually, sensitive and relatively fast static analysis methods
can serve as prefilters for slower dynamic analysis methods
reducing the effective runtime scan performance of the overall
system while producing a lower number of false positives
than either method alone. This research makes the following
contributions:
1) Extends the Malware Target Recognition (MaTR) ar-
chitecture initially proposed in [19] to an operational
model for organization self-discovery of malware with
low effective scan times and low false positive rates
through successive data reduction and analysis,
2) Demonstrates generalization of the two static analysis
models trained from [19] on a more current dataset of
new “unknown” malware, and
3) Assesses the performance of three major commercial
antivirus products against this same “unknown” malware
set at different heuristic sensitivity levels.
These applications are the most challenging for research and
commercial antivirus solutions as it represents new emerging
threats that defensive systems did not consider prior to de-
ployment. Furthermore, the test environment considers various
levels of sensitivity simulating organizational execution in
normal to hostile cyberspace environments. The MaTR static
component prototype demonstrates the ability to find such ma-
licious tools beyond the capabilities of another non-instruction
based static heuristic (n-gram) model and the three commercial
antivirus products tested at all sensitivity levels. Against a
malware set collected from operational environments, the two
static analysis methods far exceed the combined effective
detection accuracies of the three major antivirus products
tested.
