Article
Implementation of a DPU-Based Intelligent Thermal Imaging
Hardware Accelerator on FPGA
Abdelrahman S. Hussein
1,
* , Ahmed Anwar
1
, Yasmine Fahmy
1
, Hassan Mostafa
1,2
,
Khaled Nabil Salama
3
and Mai Kafafy
1
Citation: Hussein, A.S.; Anwar, A.;
Fahmy, Y.; Mostafa, H.; Salama, K.N.;
Kafafy, M. Implementation of a
DPU-Based Intelligent Thermal
Imaging Hardware Accelerator on
FPGA. Electronics 2022, 11, 105.
https://doi.org/10.3390/
electronics11010105
Academic Editor: Nunzio Cennamo
Received: 31 October 2021
Accepted: 13 December 2021
Published: 29 December 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
1
Faculty of Engineering, Cairo University, Giza 12613, Egypt; ahmed.anwar.saeed@gmail.com (A.A.);
yasfahmy@eng.cu.edu.eg (Y.F.); hmostafa@staff.cu.edu.eg (H.M.); mai.b.s.ali@eng.cu.edu.eg (M.K.)
2
Nanotechnology and Nanoelectronics Program, Zewail City of Science and Technology,
University of Science and Technology, Giza 12578, Egypt
3
Electrical Engineering Program, King Abdullah University of Science and Technology,
Thuwal 23955-6900, Saudi Arabia; khaled.salama@kaust.edu.sa
* Correspondence: abdelrahman_hussein@ieee.org
Abstract:
Thermal imaging has many applications that all leverage from the heat map that can be
constructed using this type of imaging. It can be used in Internet of Things (IoT) applications to detect
the features of surroundings. In such a case, Deep Neural Networks (DNNs) can be used to carry
out many visual analysis tasks which can provide the system with the capacity to make decisions.
However, due to their huge computational cost, such networks are recommended to exploit custom
hardware platforms to accelerate their inference as well as reduce the overall energy consumption of
the system. In this work, an energy adaptive system is proposed, which can intelligently configure
itself based on the battery energy level. Besides achieving a maximum speed increase that equals
6.38
×
, the proposed system achieves significant energy that is reduced by 97.81% compared to a
conventional general-purpose CPU.
Keywords:
Convolutional Neural Network (CNN); Deep Learning Processing Unit (DPU); Field
Programmable Gate Array (FPGA)
1. Introduction
Thermal imaging, also known as thermography, is constructing a heat map of the
surrounding. This is done by detecting the heat signature of the different objects using
thermal cameras that operate in the infra-red range (9–14
µ
m). Although thermography
started as a military application to detect enemy forces, it has recently found its way to
many more applications thanks to its numerous advantages. For example, the dependence
on heat allows thermography to operate well in a non-lit environment, and consequently,
it is suitable for surveillance applications [
1
] and wildlife monitoring [
2
]. Thermal imaging
can be used to monitor the welfare of the elderly as it only captures the human body
temperature and, therefore, does not violate their privacy [
3
]. Thermography helps to
detect early diseases in humans and plants besides early detection of thermal discomfort
among farm animals.
Thermography applications, especially monitoring, produce long sequences of thermal
images (or video frames). Different frames are expected to have different significance. For
example, while some frames might be safely ignored, others might be essential. An
intelligent edge device should not only decide the significance of the frame but also adjust
its behavior accordingly. As suggested in [
4
], edge devices should adjust their compression
rate and activity rate according to the image importance, the battery status, and the
remaining operation time.
In this paper, an edge device with a thermal camera, a transmitter, and an Intelligent
Thermal Image Processing Unit (ITIPU) is proposed. Detailed design of the ITIPU is
Electronics 2022, 11, 105. https://doi.org/10.3390/electronics11010105 https://www.mdpi.com/journal/electronics
