Citation: Nassif, A.B.; Nasir, Q.; Talib,
M.A.; Gouda, O.M. Improved Optical
Flow Estimation Method for
Deepfake Videos. Sensors 2022, 22,
2500. https://doi.org/10.3390/
s22072500
Academic Editors: Sławomir
Nowaczyk, Rita P. Ribeiro and
Grzegorz Nalepa
Received: 26 February 2022
Accepted: 23 March 2022
Published: 24 March 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Improved Optical Flow Estimation Method for
Deepfake Videos
Ali Bou Nassif
1,
* , Qassim Nasir
2
, Manar Abu Talib
3
and Omar Mohamed Gouda
1
1
Department of Computer Engineering, University of Sharjah, Sharjah 27272, United Arab Emirates;
u19104867@sharjah.ac.ae
2
Department of Electrical Engineering, University of Sharjah, Sharjah 27272, United Arab Emirates;
nasir@sharjah.ac.ae
3
Department of Computer Science, University of Sharjah, Sharjah 27272, United Arab Emirates;
mtalib@sharjah.ac.ae
* Correspondence: anassif@sharjah.ac.ae
Abstract:
Creating deepfake multimedia, and especially deepfake videos, has become much easier
these days due to the availability of deepfake tools and the virtually unlimited numbers of face
images found online. Research and industry communities have dedicated time and resources to
develop detection methods to expose these fake videos. Although these detection methods have
been developed over the past few years, synthesis methods have also made progress, allowing for
the production of deepfake videos that are harder and harder to differentiate from real videos. This
research paper proposes an improved optical flow estimation-based method to detect and expose the
discrepancies between video frames. Augmentation and modification are experimented upon to try
to improve the system’s overall accuracy. Furthermore, the system is trained on graphics processing
units (GPUs) and tensor processing units (TPUs) to explore the effects and benefits of each type of
hardware in deepfake detection. TPUs were found to have shorter training times compared to GPUs.
VGG-16 is the best performing model when used as a backbone for the system, as it achieved around
82.0% detection accuracy when trained on GPUs and 71.34% accuracy on TPUs.
Keywords:
deepfake; optical flow; tensor processing units (TPU); GPU; convolutional neural
networks (CNNs)
1. Introduction
Deepfake multimedia (manipulated images, video and audio) have grown to become
more and more of a threat to public opinion [
1
,
2
]. These fake multimedia are easily spread
all over the world thanks to social media platforms that connect people with a click of a
button [
3
]. Seeing a manipulated deepfake video of a public figure can alter a citizen’s
opinions or political stance within seconds. The term deepfake refers to manipulated
multimedia generated using artificial intelligence (AI)-based tools [4].
The most disruptive type of deepfake is a manipulated video in which a target person’s
face is replaced by another face while keeping the target’s facial expression [
5
]. Although
these generated videos can be very realistic and hard to detect, they are very easy to
create. The availability of a wide variety of images and online videos has helped to provide
enough data to create a huge number of fake videos. Anyone can generate these videos by
combining the data available with free and open-source tools such as FaceApp [
6
]. Some
positive applications of deepfake tools can be seen in movie productions, photography,
and even video games [
7
]. However, deepfake technology has been infamously used for
malicious purposes, such as creating fake news [
8
]. To address the problem of the malicious
use of deepfake technology, the research and commercial communities have developed a
number of methods to verify the integrity of multimedia files and to detect deepfake videos.
Most of the methods attempt to detect deepfake videos by analyzing pixel values [
9
,
10
].
Sensors 2022, 22, 2500. https://doi.org/10.3390/s22072500 https://www.mdpi.com/journal/sensors
