Citation: Sajun, A.R.; Zualkernan, I.;
Sankalpa, D. Investigating the
Performance of FixMatch for
COVID-19 Detection in Chest X-rays.
Appl. Sci. 2022, 12, 4694.
https://doi.org/10.3390/
app12094694
Academic Editors: Keun Ho Ryu
and Nipon Theera-Umpon
Received: 6 April 2022
Accepted: 2 May 2022
Published: 6 May 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Investigating the Performance of FixMatch for COVID-19
Detection in Chest X-rays
Ali Reza Sajun * , Imran Zualkernan and Donthi Sankalpa
Computer Science and Engineering Department, American University of Sharjah,
Sharjah P.O. Box 26666, United Arab Emirates; izualkernan@aus.edu (I.Z.); g00062902@aus.edu (D.S.)
* Correspondence: b00068908@aus.edu
Featured Application: Semi-supervised learning can effectively be used to detect the chest X-rays
affected by COVID-19.
Abstract:
The advent of the COVID-19 pandemic has resulted in medical resources being stretched
to their limits. Chest X-rays are one method of diagnosing COVID-19; they are used due to their
high efficacy. However, detecting COVID-19 manually by using these images is time-consuming and
expensive. While neural networks can be trained to detect COVID-19, doing so requires large amounts
of labeled data, which are expensive to collect and code. One approach is to use semi-supervised
neural networks to detect COVID-19 based on a very small number of labeled images. This paper
explores how well such an approach could work. The FixMatch algorithm, which is a state-of-the-art
semi-supervised classification algorithm, was trained on chest X-rays to detect COVID-19, Viral
Pneumonia, Bacterial Pneumonia and Lung Opacity. The model was trained with decreasing levels of
labeled data and compared with the best supervised CNN models, using transfer learning. FixMatch
was able to achieve a COVID F1-score of 0.94 with only 80 labeled samples per class and an overall
macro-average F1-score of 0.68 with only 20 labeled samples per class. Furthermore, an exploratory
analysis was conducted to determine the performance of FixMatch to detect COVID-19 when trained
with imbalanced data. The results show a predictable drop in performance as compared to training
with uniform data; however, a statistical analysis suggests that FixMatch may be somewhat robust
to data imbalance, as in many cases, and the same types of mistakes are made when the amount of
labeled data is decreased.
Keywords: COVID-19; chest X-rays; deep learning; semi-supervised learning; FixMatch
1. Introduction
COVID-19 was first declared a global pandemic in March 2020 by the director of the
World Health Organization (WHO) [1], and the world is still suffering from its impact. To
this date, almost 425 million people have been infected worldwide, with almost 6 million
deaths being recorded [
2
]. The disease has cold-like symptoms and is spread through
droplets in the air when people cough, sneeze or even talk [
3
]. Although the most commonly
used test to detect COVID-19 is the reverse-transcriptase polymerase chain reaction (RT-
PCR) test, as it is said to have sufficient analytical sensitivity to detect the viral infection in
the pre-infectious stage in an infected individual [
4
]; a study in Canada shows that the test
had a false negative rate (FNR) of 9.3% [
5
]. Despite the small value FNR, considering how
fast the disease spreads, the FNR could become a big issue. Hence, an alternative method
of testing is required for people who show symptoms but get negative test results. One
such way is the usage of chest X-rays (CXRs), as they are less costly compared to the other
radiological imaging methods and have the least risk, due to low amounts of radiation.
There is considerable work applying machine learning and medical imaging tech-
niques to reduce the burden on radiologists [
6
]. Due to the recent introduction of COVID-19,
Appl. Sci. 2022, 12, 4694. https://doi.org/10.3390/app12094694 https://www.mdpi.com/journal/applsci
