Citation: Cui, J.; Li, Z. Prediction of
Upper Limb Action Intention Based
on Long Short-Term Memory Neural
Network. Electronics 2022, 11, 1320.
https://doi.org/10.3390/
electronics11091320
Academic Editor: Gemma Piella
Received: 1 April 2022
Accepted: 19 April 2022
Published: 21 April 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Prediction of Upper Limb Action Intention Based on Long
Short-Term Memory Neural Network
Jianwei Cui * and Zhigang Li
School of Instrument Science and Engineering, Southeast University, Nanjing 210000, China;
220193290@seu.edu.cn
* Correspondence: cjw@seu.edu.cn; Tel.: +86-138-1392-3258
Abstract:
The use of an inertial measurement unit (IMU) to measure the motion data of the upper limb
is a mature method, and the IMU has gradually become an important device for obtaining information
sources to control assistive prosthetic hands. However, the control method of the assistive prosthetic
hand based on the IMU often has problems with high delay. Therefore, this paper proposes a method
for predicting the action intentions of upper limbs based on a long short-term memory (LSTM) neural
network. First, the degree of correlation between palm movement and arm movement is compared,
and the Pearson correlation coefficient is calculated. The correlation coefficients are all greater than
0.6, indicating that there is a strong correlation between palm movement and arm movement. Then,
the motion state of the upper limb is divided into the acceleration state, deceleration state and rest
state. The rest state of the upper limb is used as a sign to control the assistive prosthetic hand. Using
the LSTM to identify the motion state of the upper limb, the accuracy rate is 99%. When predicting
the action intention of the upper limb based on the angular velocity of the shoulder and forearm,
the LSTM is used to predict the angular velocity of the palm, and the average prediction error of
palm motion is 1.5 rad/s. Finally, the feasibility of the method is verified through experiments, in the
form of holding an assistive prosthetic hand to imitate a disabled person wearing a prosthesis. The
assistive prosthetic hand is used to reproduce foot actions, and the average delay time of foot action
was 0.65 s, which was measured by using the method based on the LSTM neural network. However,
the average delay time of the manipulator control method based on threshold analysis is 1.35 s. Our
experiments show that the prediction method based on the LSTM can achieve low prediction error
and delay.
Keywords:
action recognition of upper limbs; inertial measurement unit; motion intention prediction;
long short-term memory neural network; control of the assistive prosthetic hand
1. Introduction
The disabled are a special group in contemporary society. Physical defects have
brought many inconveniences to their lives. In order to make up for the missing upper
limbs of the handicapped and improve their self-care ability, upper limb prostheses are
used to replace part of the functions of the lost limbs [
1
,
2
]. For the control of upper limb
prostheses, predicting the user’s movement intention is as important as identifying the
type of action of the upper limb. Action recognition usually focuses on the complete action
performed by the upper limb. It is the result of doing the action, such as drinking water,
putting on shoes, and brushing teeth [
3
]. In contrast, intent prediction not only identifies
the types of actions performed by users, but also focuses on how to perform these actions.
It is the process from “what to do” to “how to do it” [
4
]. Intention prediction is not only
applied in the field of disability; it also plays an important role in the field of rehabilitation
and healthcare [5–7].
In the human body, information sources that can be used to control the upper limb
prosthesis mainly include electrophysiological signals and mechanical signals [
8
]. The IMU
Electronics 2022, 11, 1320. https://doi.org/10.3390/electronics11091320 https://www.mdpi.com/journal/electronics
