Article
Deep Learning-Based Optimal Smart Shoes Sensor Selection
for Energy Expenditure and Heart Rate Estimation
Heesang Eom
1,†
, Jongryun Roh
2,†
, Yuli Sun Hariyani
1,3
, Suwhan Baek
1
, Sukho Lee
4
, Sayup Kim
2,
*
and Cheolsoo Park
1,
*
Citation: Eom, H.; Roh, J.; Hariyani,
Y.S.; Baek, S.; Lee, S.; Kim, S.; Park, C.
Deep Learning-Based Optimal Smart
Shoes Sensor Selection for Energy
Expenditure and Heart Rate
Estimation. Sensors 2021, 21, 7058.
https://doi.org/10.3390/s21217058
Academic Editor: Ki H. Chon
Received: 24 September 2021
Accepted: 21 October 2021
Published: 25 October 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
1
Department of Computer Engineering, Kwangwoon University, Seoul 01897, Korea;
9200heesang@gmail.com (H.E.); yulisun@telkomuniversity.ac.id (Y.S.H.); zhsjzhsj@gmail.com (S.B.)
2
Digital Transformation R&D Department, Korea Institute of Industrial Technology (KITECH),
143 Hanggaulro, Ansan 15588, Korea; ssaccn@kitech.re.kr
3
School of Applied Science, Telkom University, Bandung 40257, Indonesia
4
Department of Leisure Sports, College of Ecological Environment, Kyungpook National University,
Sangju-si 37224, Korea; ehduq132@gmail.com
* Correspondence: sayub@kitech.re.kr (S.K.); parkcheolsoo@kw.ac.kr (C.P.)
† These authors contributed equally to this work.
Abstract:
Wearable technologies are known to improve our quality of life. Among the various
wearable devices, shoes are non-intrusive, lightweight, and can be used for outdoor activities. In this
study, we estimated the energy consumption and heart rate in an environment (i.e., running on a
treadmill) using smart shoes equipped with triaxial acceleration, triaxial gyroscope, and four-point
pressure sensors. The proposed model uses the latest deep learning architecture which does not
require any separate preprocessing. Moreover, it is possible to select the optimal sensor using a
channel-wise attention mechanism to weigh the sensors depending on their contributions to the
estimation of energy expenditure (EE) and heart rate (HR). The performance of the proposed model
was evaluated using the root mean squared error (RMSE), mean absolute error (MAE), and coefficient
of determination (
R
2
). Moreover, the RMSE was 1.05
±
0.15, MAE 0.83
±
0.12 and
R
2
0.922
±
0.005 in
EE estimation. On the other hand, and RMSE was 7.87
±
1.12, MAE 6.21
±
0.86, and
R
2
0.897
±
0.017
in HR estimation. In both estimations, the most effective sensor was the z axis of the accelerometer
and gyroscope sensors. Through these results, it is demonstrated that the proposed model could
contribute to the improvement of the performance of both EE and HR estimations by effectively
selecting the optimal sensors during the active movements of participants.
Keywords:
smart shoe; energy expenditure; heart rate; channel wise attention; DenseNet;
accelerometer; gyroscope; pressure sensor; deep learning
1. Introduction
Wearable technologies have been continuously developed to improve the quality of
human life and facilitate mobility and connectivity among users due to the rapid devel-
opment of the Internet of Things (IoT). Its global demand is increasing every
year [1–3]
.
Recently, several wearable devices, including wrist bands, watches, glasses, and shoes,
have started enabling the continuous monitoring of an individual’s health, wellness, and
fitness [
4
]. In particular, the coronavirus disease (COVID-19) pandemic highlighted the
importance of remote healthcare delivery, resulting in further expansion of the wearable
technology market [
3
,
5
]. This is because wearable devices could continuously collect and
analyze the movement and physiological data of a user and provide appropriate feedback
in function of users’ exercise information and health status.
The shoe is a useful wearable device that is easy to use, unobtrusive, lightweight, and
easily available when doing outdoor activities [
6
–
9
]. Previous studies on shoes include gait
type classification [
9
–
11
], step count [
8
,
12
,
13
], and energy expenditure (EE) estimation [
14
].
Sensors 2021, 21, 7058. https://doi.org/10.3390/s21217058 https://www.mdpi.com/journal/sensors
