Article
Boosting Intelligent Data Analysis in Smart Sensors by
Integrating Knowledge and Machine Learning
Piotr Łuczak , Przemysław Kucharski , Tomasz Jaworski , Izabela Perenc and Krzysztof
´
Slot
and Jacek Kucharski *
Citation: Łuczak, P.; Kucharski, P.;
Jaworski, T.; Perenc, I.;
´
Slot, K.;
Kucharski, J. Boosting Intelligent
Data Analysis in Smart Sensors by
Integrating Knowledge and Machine
Learning. Sensors 2021, 21, 6168.
https://doi.org/10.3390/s21186168
Academic Editors: Panagiotis E.
Pintelas, Sotiris Kotsiantis,
Ioannis E. Livieris
Received: 20 August 2021
Accepted: 12 September 2021
Published: 14 September 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Institute of Applied Computer Science, Lodz University of Technology, Stefanowskiego 18/22,
90-537 Łód´z, Poland; pluczak@iis.p.lodz.pl (P.Ł.); pkuchars@iis.p.lodz.pl (P.K.); tjaworski@iis.p.lodz.pl (T.J.);
iperenc@iis.p.lodz.pl (I.P.); kslot@p.lodz.pl (K.
´
S.)
* Correspondence: jkuchars@iis.p.lodz.pl
Abstract:
The presented paper proposes a hybrid neural architecture that enables intelligent data
analysis efficacy to be boosted in smart sensor devices, which are typically resource-constrained
and application-specific. The postulated concept integrates prior knowledge with learning from
examples, thus allowing sensor devices to be used for the successful execution of machine learning
even when the volume of training data is highly limited, using compact underlying hardware. The
proposed architecture comprises two interacting functional modules arranged in a homogeneous,
multiple-layer architecture. The first module, referred to as the knowledge sub-network, implements
knowledge in the Conjunctive Normal Form through a three-layer structure composed of novel
types of learnable units, called L-neurons. In contrast, the second module is a fully-connected
conventional three-layer, feed-forward neural network, and it is referred to as a conventional neural
sub-network. We show that the proposed hybrid structure successfully combines knowledge and
learning, providing high recognition performance even for very limited training datasets, while also
benefiting from an abundance of data, as it occurs for purely neural structures. In addition, since the
proposed L-neurons can learn (through classical backpropagation), we show that the architecture is
also capable of repairing its knowledge.
Keywords: AI-enabled sensors; hybrid systems; feedforward neural networks; knowledge embedding
1. Introduction
In recent years, remarkable improvement has been shown in both the capabilities and
efficiency of intelligent systems [
1
], yet the state-of-the-art models continue to grow in size.
Not only are intelligent systems now capable of achieving state-of-the-art performance on
multiple complex games, as shown by AlphaZero [
2
], but they are also capable of solving
extremely complex real-world problems such as protein folding. The most recent release
of AlphaFold [
3
] proved to be capable of solving the 14th Critical Assessment of protein
Structure Prediction (CASP) challenge [
4
], thus providing an invaluable tool for modern
bioinformatics research. These performance improvements are achieved at the expense of
increases in model size, such as in the case of the GPT (Generative Pre-trained Transformer)
family of models that went from 1.5 billion parameters in 2019 [
5
] to 175 billion parameters
in 2020 [
6
]. These large models, while still feasible to train thanks to algorithmic and
technological advances, require ever-increasing amounts of input examples, which may
be unavailable, especially when application-specific tasks, typical for smart sensors, are
considered. In addition, implementing large neural networks on resource-limited devices
is infeasible, so if machine learning is to be considered as a problem-solving strategy for
smart sensors, one needs to look for network complexity reduction concepts that preserve
a sufficient capacity for handling real-world problems.
Since large neural models learn everything from scratch, a significant part of training
time is spent learning relations that are inherently obvious to a human expert. A possi-
Sensors 2021, 21, 6168. https://doi.org/10.3390/s21186168 https://www.mdpi.com/journal/sensors
