DEFINE “EXPERT”: CHARACTERIZING PROFICIENCY FOR
PHYSIOLOGICAL MEASURES OF COGNITIVE WORKLOAD
Amy Dideriksen
1
, Christopher Reuter
2
, Thomas Patry
2
, Thomas Schnell
2
, Jaclyn Hoke
1
,
Jocelyn Faubert
3,4
1
Collins Aerospace, Inc., Cedar Rapids, IA
2
University of Iowa Operator Performance Lab, Iowa City, IA
3
CogniSens Applied Research Center, Montreal, Qc, CA
4
University of Montreal, School of Optometry, Faubert Lab, Montreal Qc, CA
amy.dideriksen@collins.com, christopher-m-reuter@uiowa.edu, thomas-patry@uiowa.edu,
thomas-schnell@uiowa.edu, jaclyn.hoke@collins.com, jocelyn.faubert@umontreal.ca
ABSTRACT
Training providers continue to be challenged in accurately measuring the effectiveness of performance-based
training solutions. Studies have shown interest in measuring cognitive state to improve human performance
(Schmorrow & Kruse, 2002), yet the training industry still lacks a non-invasive, near real-time deployable method
to objectively measure the trainee’s cognitive state. Our collaborative research team has developed and
documented a valid methodology for quantitatively assessing training effectiveness, using physiological measures
of cognitive state coupled with task-specific performance metrics. To successfully employ this method and design
personalized training, we must develop standard definitions of proficiency levels in terms of the physiological
signature of cognitive workload.
During an initial study performed in 2017, we measured the total cognitive load, spare cognitive capacity and task-
specific performance metrics (i.e., flight technical performance) of novice pilots performing standardized hand-
flown tasks in a simulator and in live flight. We extended this evaluation in 2018 to include competent and expert
pilots. The purpose of this follow-on study was two-fold: to further validate the approach for measuring training
effectiveness, and to characterize the effect of pilot education and experience on cognitive workload, spare
cognitive capacity, and task-specific performance. Through this research, we have defined an initial set of
standards for the interplay between cognitive workload and performance associated with various learner
proficiency levels.
This paper summarizes the key results of the follow-on study and describes the standards of cognitive workload
developed as a result of the two-year research effort. It also illustrates how cognitive workload trends can assist in
developing personalized, performance-based learning for trainees with varying degrees of proficiency. It
concludes with a discussion of how this methodology can be applied to improve training outcomes and future
studies that would further extend its value in the simulation and training industry.
