Communication
Decision Confidence Assessment in Multi-Class Classification
Michał Bukowski
1
, Jarosław Kurek
1,
* , Izabella Antoniuk
1
and Albina Jegorowa
2
Citation: Bukowski, M.; Kurek, J.;
Antoniuk, I.; Jegorowa, A. Decision
Confidence Assessment in Multi-
Class Classification. Sensors 2021, 21,
3834. https://doi.org/10.3390/
s21113834
Academic Editors: Panagiotis E.
Pintelas, Sotiris Kotsiantis and
Ioannis E. Livieris
Received: 31 March 2021
Accepted: 19 May 2021
Published: 1 June 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
1
Institute of Information Technology, Warsaw University of Life Sciences, Nowoursynowska 159,
02-776 Warsaw, Poland; michal.bukowski@buksoft.pl (M.B.); izabella_antoniuk@sggw.edu.pl (I.A.)
2
Institute of Wood Sciences and Furniture, Warsaw University of Life Sciences, Nowoursynowska 159,
02-776 Warsaw, Poland; albina_jegorowa@sggw.edu.pl
* Correspondence: jaroslaw_kurek@sggw.edu.pl; Tel.: +48-505-482-708
Abstract:
This paper presents a novel approach to the assessment of decision confidence when
multi-class recognition is concerned. When many classification problems are considered, while
eliminating human interaction with the system might be one goal, it is not the only possible option—
lessening the workload of human experts can also bring huge improvement to the production process.
The presented approach focuses on providing a tool that will significantly decrease the amount of
work that the human expert needs to conduct while evaluating different samples. Instead of hard
classification, which assigns a single label to each class, the described solution focuses on evaluating
each case in terms of decision confidence—checking how sure the classifier is in the case of the
currently processed example, and deciding if the final classification should be performed, or if the
sample should instead be manually evaluated by a human expert. The method can be easily adjusted
to any number of classes. It can also focus either on the classification accuracy or coverage of the
used dataset, depending on user preferences. Different confidence functions are evaluated in that
aspect. The results obtained during experiments meet the initial criteria, providing an acceptable
quality for the final solution.
Keywords:
confidence classification; confidence functions; multi-class classification; tool condition
monitoring; laminated chipboard
1. Introduction
Ensuring an efficient and smooth flow of production processes can be challenging,
time-consuming, and, at times, also problematic. For example, in the wood industry, from
the many tasks that need to be monitored, some of them will require specialized knowledge
and precision, while others will use up a significant amount of time, and there are quite a
lot of activities that combine all of those features. One of such tasks concerns evaluating the
state of drills in the manufacturing process, which is a subset of problems widely known as
tool condition monitoring. Usually, when manually performed, this task requires stopping
the production process in order to evaluate individual tools. At the same time, a human
expert is required to check used elements, without any indication to its actual state. Due
to that, unnecessary downtime may occur, when it could have been avoided if the entire
process had been, at least partially, automated.
When it comes to tool evaluation, many different approaches have been considered
to either speed up the process, or avoid human intervention in general. For example, the
main focus may include evaluating the state of elements without interrupting the actual
manufacturing process, as presented in [
1
]. Most basic and commonly used approaches,
such as the one presented in [
2
], measure different signals, such as vibration, noise, acoustic
emission, cutting torque, feed force, and others, in order to evaluate the tool state. Similar
approaches were used in [
3
], where data were extracted both from signal and frequency
domains, along with wavelet coefficients, all in order to evaluate the obtained elements
automatically, checking how relevant each item was to the selected problem. Further,
Sensors 2021, 21, 3834. https://doi.org/10.3390/s21113834 https://www.mdpi.com/journal/sensors
