Citation: Ryu, J. A Visual
Saliency-Based Neural Network
Architecture for No-Reference Image
Quality Assessment. Appl. Sci. 2022,
12, 9567. https://doi.org/10.3390/
app12199567
Academic Editors: M. Jamal Deen,
Subhas Mukhopadhyay, Yangquan
Chen, Simone Morais, Nunzio
Cennamo and Junseop Lee
Received: 4 August 2022
Accepted: 15 September 2022
Published: 23 September 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the author.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
A Visual Saliency-Based Neural Network Architecture for
No-Reference Image Quality Assessment
Jihyoung Ryu
Electronics and Telecommunications Research Institute (ETRI), Gwangju 61012, Korea; jihyoung@etri.re.kr
Abstract:
Deep learning has recently been used to study blind image quality assessment (BIQA) in
great detail. Yet, the scarcity of high-quality algorithms prevents from developing them further and
being used in a real-time scenario. Patch-based techniques have been used to forecast the quality of
an image, but they typically award the picture quality score to an individual patch of the image. As a
result, there would be a lot of misleading scores coming from patches. Some regions of the image
are important and can contribute highly toward the right prediction of its quality. To prevent outlier
regions, we suggest a technique with a visual saliency module which allows the only important region
to bypass to the neural network and allows the network to only learn the important information
required to predict the quality. The neural network architecture used in this study is Inception-
ResNet-v2. We assess the proposed strategy using a benchmark database (KADID-10k) to show its
efficacy. The outcome demonstrates better performance compared with certain popular no-reference
IQA (NR-IQA) and full-reference IQA (FR-IQA) approaches. This technique is intended to be utilized
to estimate the quality of an image being acquired in real time from drone imagery.
Keywords:
image quality assessment (IQA); visual saliency; Inception-ResNet-v2; saliency map;
no-reference IQA (NR-IQA)
1. Introduction
With the rapid rise of digital technology, more people utilize applications related to
multimedia content, which includes images and video-related content [
1
]. Unlike in the
past, now people have also included another item in their multimedia consumption, which
includes 3D images and video content. People are inclined toward 3D content more as
it focuses on providing a near-reality experience and perception. An end user demands
multimedia content, which could be either images, videos or 3D content. However, the
crucial part is that the transmitted content should be of a high quality [
2
]. The development
continued, as users expect high-quality images. The process starts with image acquisition,
where an image is simply captured and converted into a digital format using reliable
cameras. Furthermore, preprocessing is performed on the images, and they are transmitted
over the channel [
3
]. Finally, image restoration is performed, after which an end user can
have an experience with the multimedia content [
4
]. The problem is that each of these
subsystems from image acquisition to displaying the image can induce certain types of
distortions in the image. Therefore, the end resultant image will be a distorted version.
On the other hand, the end user requires higher-quality images and multimedia content.
A lot of research has been conducted to develop solutions to assess the quality of images
in a way that should be quite accurate, and this assessment needs to be automatic [
5
].
Nonetheless, image quality assessment (IQA) is essential to ensure that the image is not
affected by any type of distortion before the image reaches the end user. IQA can play a
vital role in improving the quality of an image and in image restoration as well.
Further down the line, IQA can be divided up into subjective quality scores and
objective quality scores. In subjective image quality assessment (SIQA), the quality of an
image is assessed from a human visual perspective. As the human eye can perceive and
Appl. Sci. 2022, 12, 9567. https://doi.org/10.3390/app12199567 https://www.mdpi.com/journal/applsci
