Article
Sensor-Based Human Activity Recognition Using Adaptive
Class Hierarchy
Kazuma Kondo * and Tatsuhito Hasegawa
Citation: Kondo, K.; Hasegawa, T.
Sensor-Based Human Activity
Recognition Using Adaptive Class
Hierarchy. Sensors 2021, 21, 7743.
https://doi.org/10.3390/s21227743
Academic Editors: Subhas
Mukhopadhyay, Yangquan Chen,
Nunzio Cennamo, Mohamed Jamal
Deen, Junseop Lee and
Simone Morais
Received: 18 October 2021
Accepted: 17 November 2021
Published: 21 November 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Graduate School of Engineering, University of Fukui, Fukui 910-8507, Japan; t-hase@u-fukui.ac.jp
* Correspondence: mf200497@u-fukui.ac.jp
Abstract:
In sensor-based human activity recognition, many methods based on convolutional neural
networks (CNNs) have been proposed. In the typical CNN-based activity recognition model, each
class is treated independently of others. However, actual activity classes often have hierarchical
relationships. It is important to consider an activity recognition model that uses the hierarchical
relationship among classes to improve recognition performance. In image recognition, branch CNNs
(B-CNNs) have been proposed for classification using class hierarchies. B-CNNs can easily perform
classification using hand-crafted class hierarchies, but it is difficult to manually design an appropriate
class hierarchy when the number of classes is large or there is little prior knowledge. Therefore,
in our study, we propose a class hierarchy-adaptive B-CNN, which adds a method to the B-CNN
for automatically constructing class hierarchies. Our method constructs the class hierarchy from
training data automatically to effectively train the B-CNN without prior knowledge. We evaluated our
method on several benchmark datasets for activity recognition. As a result, our method outperformed
standard CNN models without considering the hierarchical relationship among classes. In addition,
we confirmed that our method has performance comparable to a B-CNN model with a class hierarchy
based on human prior knowledge.
Keywords: human activity recognition; class hierarchy; deep learning
1. Introduction
Human activity recognition is expected to be used in a wide range of fields [
1
]. Sensor-
based human activity recognition is the task of automatically predicting a user’s activity
and states using sensors. The prediction results can be used to support user actions or
decision-making in organizations.
In recent years, deep learning (DL) has been used in various fields and many DL
methods have been proposed for human activity recognition. Many activity recognition
models with DL are based on convolutional neural networks (CNNs) [
2
]. DL is a powerful
method for various fields and has been rapidly developed in computer vision and neural
language processing especially.
The activity recognition models based on CNNs use activity labels encoded to one-hot
vectors. Typical activity recognition models are trained ignoring the relationships among
activities, because the one-hot encoding treats each class as independent of each other.
However, there are hierarchical relationships among actual activities, which are based on
similarity of sensor data [
3
]. For example, considering four classes of stationary, walking,
going up the stairs and going down the stairs, the three classes other than stationary can
be regarded as an abstract class, non-stationary. This indicates that there is a hierarchical
structure among the activity classes. Hierarchical relationships among classes are known to
affect classification patterns of standard CNNs [
4
]. Low similar classes, such as stationary
and walking, can hardly be misclassified mutually. On the other hand, high similar classes,
such as walking and going up the stairs, are frequently misclassified mutually. Therefore,
the recognition model is expected to improve its performance using relationships among
classes known in advance.
Sensors 2021, 21, 7743. https://doi.org/10.3390/s21227743 https://www.mdpi.com/journal/sensors
