Citation: Yang, F.; Wang, G.; Li, D.;
Liu, N.; Min, F. Research on
Repetition Counting Method Based
on Complex Action Label String.
Machines 2022, 10, 419. https://
doi.org/10.3390/machines10060419
Academic Editors: Kelvin K.L. Wong,
Dhanjoo N. Ghista, Andrew W.H. Ip
and Wenjun (Chris) Zhang
Received: 2 May 2022
Accepted: 20 May 2022
Published: 26 May 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Research on Repetition Counting Method Based on Complex
Action Label String
Fanghong Yang
1
, Gao Wang
1
, Deping Li
2
, Ning Liu
1,2
and Feiyan Min
1,
*
1
Department of Electronic Engineering, College of Information Science and Technology, Jinan University,
Guangzhou 510632, China; fanghongyang@stu2019.jnu.edu.cn (F.Y.); twangg@jnu.edu.cn (G.W.);
tliuning@jnu.edu.cn (N.L.)
2
School of Intelligent Systems Science and Engineering, Jinan University, Zhuhai 519070, China;
lideping@jnu.edu.cn
* Correspondence: feiyanmin@jnu.edu.cn; Tel.: +86-020-8522-3063
Abstract:
Smart factories have real-time demands for the statistics of productivity to meet the needs
of quick reaction capabilities. To solve this problem, a counting method based on our decomposition
strategy of actions was proposed for complex actions. Our method needs to decompose complex
actions into several essential actions and define a label string for each complex action according to
the sequence of the essential actions. While counting, we firstly employ an online action recognition
algorithm to transform video frames into label numbers, which will be stored in a result queue. Then,
the label strings are searched for their results in queue. If the search succeeds, a complex action will be
considered to have occurred. Meanwhile, the corresponding counter should be updated to accomplish
counting. The comparison test results in a video dataset of workers’ repetitive movements in package
printing production lines and illustrate that our method has a lower counting errors, MAE (mean
absolute error) less than 5% as well as an OBOA (off-by-one accuracy) more than 90%. Moreover, to
enhance the adaptability of the action recognition model to deal with the change of action duration,
we propose an adaptive parameter module based on the Kalman filter, which improves counting
performances to a certain extent. The conclusions are that our method can achieve high counting
performance, and the adaptive parameter module can further improve performances.
Keywords:
action counting; action decomposition; complex action label string; template matching;
Kalman filtering
1. Introduction
1.1. Repetition Counting
The construction of smart factories gives rise to many requirements related to the
monitoring of production status, among which the productivity of workers comprises
key data. Calculating production efficiency in tradition means it will be performed after
workers finish production, which has a lag and cannot meet the needs of rapid response.
In order to satisfy the demand above, we came up with a repetitive action counting method
based on a decomposition strategy for the real-time statistics of workers’ productivity in
the packaging industry.
To complete the counting task of human repetitive movements, the academic commu-
nity researchers mainly focus on Spatio-temporal information of sequence data, trying to
restore the periodic information synchronized with repeated actions from the original data.
Finally, post-processing of the periodic information is performed to realize the counting
task. The existing methods are classified according to the mode of input, which can be
classified as computer-vision or non-computer-vision counting algorithms. Non-machine-
vision counting algorithms mainly obtain the number of repetitions by analyzing the data
collected from wearable devices [
1
–
3
]. In addition, the modes of data vary with data acqui-
sition equipment, such as acceleration sequences, angular momentum sequences, and so
Machines 2022, 10, 419. https://doi.org/10.3390/machines10060419 https://www.mdpi.com/journal/machines
