Citation: Du, B.; Cheng, X.; Duan, Y.;
Ning, H. fMRI Brain Decoding and
Its Applications in Brain–Computer
Interface: A Survey. Brain Sci. 2022,
12, 228. https://doi.org/10.3390/
brainsci12020228
Academic Editors: Enrico Vezzetti,
Andrea Luigi Guerra, Gabriele
Baronio, Domenico Speranza and
Luca Ulrich
Received: 20 December 2021
Accepted: 30 January 2022
Published: 7 February 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
fMRI Brain Decoding and Its Applications in Brain–Computer
Interface: A Survey
Bing Du
1
, Xiaomu Cheng
1
, Yiping Duan
2
and Huansheng Ning
1,
*
1
School of Computer and Communication Engineering, University of Science and Technology Beijing,
Beijing 100083, China; dubing@ustb.edu.cn (B.D.); s20190669@xs.ustb.edu.cn (X.C.)
2
Department of Electronic Engineering, Tsinghua University, Beijing 100084, China;
Yipingduan@mail.tsinghua.edu.cn
* Correspondence: ninghuansheng@ustb.edu.cn
Abstract:
Brain neural activity decoding is an important branch of neuroscience research and a key
technology for the brain–computer interface (BCI). Researchers initially developed simple linear
models and machine learning algorithms to classify and recognize brain activities. With the great
success of deep learning on image recognition and generation, deep neural networks (DNN) have
been engaged in reconstructing visual stimuli from human brain activity via functional magnetic
resonance imaging (fMRI). In this paper, we reviewed the brain activity decoding models based on
machine learning and deep learning algorithms. Specifically, we focused on current brain activity
decoding models with high attention: variational auto-encoder (VAE), generative confrontation
network (GAN), and the graph convolutional network (GCN). Furthermore, brain neural-activity-
decoding-enabled fMRI-based BCI applications in mental and psychological disease treatment are
presented to illustrate the positive correlation between brain decoding and BCI. Finally, existing
challenges and future research directions are addressed.
Keywords:
brain decoding; variational autoencoder (VAE); generative adversarial network (GAN);
graph convolutional networks (GCN); functional magnetic resonance imaging (fMRI); brain–computer
interface (BCI)
1. Introduction
In recent years, the concept of the brain–computer interface has gradually entered
the public’s field of vision and has become a hot topic in the field of brain research. Brain
neural activity decoding is a key technology for the brain–computer interface. Therefore,
this paper first surveys research in the field of visually decoding brain neuronal activity, ex-
plains the strengths and weaknesses of these studies, and updates recent research progress
and potential clinical applications of brain–computer interfaces in psychotherapy. Finally,
potential solutions are proposed for the problems existing in the current brain activity
decoding models. Functional magnetic resonance imaging (fMRI) is a new neuroimaging
method. Its principle is to use magnetic resonance imaging to measure the changes in
hemodynamics caused by neuronal activity. From the perspective of neuroscience and neu-
roimaging, functional magnetic resonance imaging can be used to decode the perception
and semantic information of the cerebral cortex in a non-invasive manner [
1
]. The spatial
resolution of fMRI is high enough, while the noise contained in the measurement of the
brain activity is relatively small [
2
]. The real-time fMRI monitors the activity state of the
cerebral cortex through the blood-oxygen-level-dependent (BOLD) variation induced by
brain neural activity and simultaneously collects and analyzes the BOLD signals of the
brain [3,4]. In recent years, more and more researchers have adopted real-time fMRI to in-
vestigate the brain’s self-regulation and the connection among different brain
regions [5–7]
.
Compared with Electroencephalogram (EEG) [
2
,
8
–
11
], fMRI has higher spatial resolution
and suffers from lower noise. Therefore, decoding brain neural activity with fMRI data
Brain Sci. 2022, 12, 228. https://doi.org/10.3390/brainsci12020228 https://www.mdpi.com/journal/brainsci
