Citation: Chen, K.-Y.; Chou, L.-W.;
Lee, H.-M.; Young, S.-T.; Lin, C.-H.;
Zhou, Y.-S.; Tang, S.-T.; Lai, Y.-H.
Human Motion Tracking Using 3D
Image Features with a Long
Short-Term Memory Mechanism
Model—An Example of Forward
Reaching. Sensors 2022, 22, 292.
https://doi.org/10.3390/s22010292
Academic Editors: Nunzio Cennamo,
Yangquan Chen,
Subhas Mukhopadhyay, M.
Jamal Deen, Junseop Lee and
Simone Morais
Received: 20 October 2021
Accepted: 14 December 2021
Published: 31 December 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Human Motion Tracking Using 3D Image Features with a Long
Short-Term Memory Mechanism Model—An Example of
Forward Reaching
Kai-Yu Chen
1
, Li-Wei Chou
2
, Hui-Min Lee
3
, Shuenn-Tsong Young
4
, Cheng-Hung Lin
5
, Yi-Shu Zhou
1
,
Shih-Tsang Tang
6
and Ying-Hui Lai
1,7,
*
1
Department of Biomedical Engineering, National Yang Ming Chiao Tung University, Taipei 112, Taiwan;
s930470.be09@nycu.edu.tw (K.-Y.C.); leo641001@gmail.com (Y.-S.Z.)
2
Department of Physical Therapy and Assistive Technology, National Yang Ming Chiao Tung University,
Taipei 112, Taiwan; lwchou@nycu.edu.tw
3
The Research Center on ICF and Assistive Technology, National Yang Ming Chiao Tung University,
Taipei 112, Taiwan; noralee0724@gmail.com
4
Institute of Geriatric Welfare Technology & Science, MacKay Medical College, New Taipei City 252, Taiwan;
styoung@mmc.edu.tw
5
Department of Electrical Engineering, National Taiwan Normal University, Taipei 106, Taiwan;
brucelin@ntnu.edu.tw
6
Department of Biomedical Engineering, Ming Chuan University, Taoyuan 333, Taiwan;
sttang@mail.mcu.edu.tw
7
Medical Device Innovation & Translation Center, National Yang Ming Chiao Tung University,
Taipei 112, Taiwan
* Correspondence: yh.lai@nycu.edu.tw
Abstract:
Human motion tracking is widely applied to rehabilitation tasks, and inertial measurement
unit (IMU) sensors are a well-known approach for recording motion behavior. IMU sensors can
provide accurate information regarding three-dimensional (3D) human motion. However, IMU
sensors must be attached to the body, which can be inconvenient or uncomfortable for users. To
alleviate this issue, a visual-based tracking system from two-dimensional (2D) RGB images has been
studied extensively in recent years and proven to have a suitable performance for human motion
tracking. However, the 2D image system has its limitations. Specifically, human motion consists of
spatial changes, and the 3D motion features predicted from the 2D images have limitations. In this
study, we propose a deep learning (DL) human motion tracking technology using 3D image features
with a deep bidirectional long short-term memory (DBLSTM) mechanism model. The experimental
results show that, compared with the traditional 2D image system, the proposed system provides
improved human motion tracking ability with RMSE in acceleration less than 0.5 (m/s
2
) X, Y, and Z
directions. These findings suggest that the proposed model is a viable approach for future human
motion tracking applications.
Keywords:
depth image; time-of-flight camera; deep learning; human motion tracking;
rehabilitation application
1. Introduction
Rehabilitation is becoming an increasingly important issue owing to the rise in elderly
population. According to the World Population Ageing 2020 report [
1
], 727 million people
were aged 65 years or older around the world, and the number of the elderly was projected
to rise to 1.5 billion by 2050. In addition, the World Health Organization (WHO) indicates
that 15 million people suffer from stroke each year [
2
], with 75% of those being elderly [
3
].
This means that healthcare, such as motion rehabilitation to regain motor function, must be
valued and improved.
Sensors 2022, 22, 292. https://doi.org/10.3390/s22010292 https://www.mdpi.com/journal/sensors
