Citation: Zhang, X.; Zhang, D.; Leye,
A.; Scott, A.; Visser, L.; Ge, Z.;
Bonnington, P. Autonomous Incident
Detection on Spectrometers Using
Deep Convolutional Models. Sensors
2022, 22, 160. https://doi.org/
10.3390/s22010160
Academic Editor: Kelvin K. L. Wong,
Dhanjoo N. Ghista, Andrew W. H. Ip
and Wenjun (Chris) Zhang
Received: 30 November 2021
Accepted: 23 December 2021
Published: 27 December 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Autonomous Incident Detection on Spectrometers Using Deep
Convolutional Models
Xuelin Zhang
1,†
, Donghao Zhang
1,†
, Alexander Leye
1,†
, Adrian Scott
2
, Luke Visser
2
, Zongyuan Ge
1,
*
and Paul Bonnington
1,
*
1
Monash eResearch Centre, 15 Innovation Walk, Monash University, Clayton Campus Victoria,
Clayton, VIC 3800, Australia; xuelin.zhang@monash.edu (X.Z.); donghao.zhang@monash.edu (D.Z.);
anley1@student.monash.edu (A.L.)
2
Agilent Technologies, 679 Springvale Rd, Mulgrave, VIC 3170, Australia; adrian.scott@agilent.com (A.S.);
luke.visser@agilent.com (L.V.)
* Correspondence: zongyuan.ge@monash.edu (Z.G.); Paul.Bonnington@monash.edu (P.B.)
† These authors contributed equally to this work.
Abstract:
This paper focuses on improving the performance of scientific instrumentation that uses
glass spray chambers for sample introduction, such as spectrometers, which are widely used in
analytical chemistry, by detecting incidents using deep convolutional models. The performance of
these instruments can be affected by the quality of the introduction of the sample into the spray
chamber. Among the indicators of poor quality sample introduction are two primary incidents: The
formation of liquid beads on the surface of the spray chamber, and flooding at the bottom of the
spray chamber. Detecting such events autonomously as they occur can assist with improving the
overall operational accuracy and efficacy of the chemical analysis, and avoid severe incidents such as
malfunction and instrument damage. In contrast to objects commonly seen in the real world, beading
and flooding detection are more challenging since they are of significantly small size and transparent.
Furthermore, the non-rigid property increases the difficulty of the detection of these incidents, as
such that existing deep-learning-based object detection frameworks are prone to fail for this task.
There is no former work that uses computer vision to detect these incidents in the chemistry industry.
In this work, we propose two frameworks for the detection task of these two incidents, which not
only leverage the modern deep learning architectures but also integrate with expert knowledge of the
problems. Specifically, the proposed networks first localize the regions of interest where the incidents
are most likely generated and then refine these incident outputs. The use of data augmentation and
synthesis, and choice of negative sampling in training, allows for a large increase in accuracy while
remaining a real-time system for inference. In the data collected from our laboratory, our method
surpasses widely used object detection baselines and can correctly detect 95% of the beads and 98%
of the flooding. At the same time, out method can process four frames per second and is able to be
implemented in real time.
Keywords: machine vision; deep learning; object detection
1. Introduction
Machine vision, also known as industrial computer vision, is the key component of In-
dustry 4.0 for the industrial automation revolution. It joins machine learning and computer
vision in a set of tools that are able to grant industry-level instruments unprecedented abili-
ties to observe and interpret their environment [
1
]. Industrial activities involving machine
vision systems such as incident detection, product quality monitoring, and manufacturing
automation, have demonstrated tremendous potential for bolstering productivity, reducing
waste, refining product quality, enhancing manufacturing flexibility, and decreasing oper-
ating costs. Machine vision plays an essential role in bolstering productivity for modern
precise agriculture, which is challenging due to wind disturbance, changing illumination,
Sensors 2022, 22, 160. https://doi.org/10.3390/s22010160 https://www.mdpi.com/journal/sensors
