Citation: Yan, Z.; Jin, Z.; Teng, S.;
Chen, G.; Bassir, D. Measurement of
Bridge Vibration by UAVs Combined
with CNN and KLT Optical-Flow
Method. Appl. Sci. 2022, 12, 5181.
https://doi.org/10.3390/app12105181
Academic Editor:
Giuseppe Lacidogna
Received: 26 April 2022
Accepted: 19 May 2022
Published: 20 May 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Measurement of Bridge Vibration by UAVs Combined with
CNN and KLT Optical-Flow Method
Zhaocheng Yan
1
, Zihan Jin
1
, Shuai Teng
1
, Gongfa Chen
1,
* and David Bassir
2,3
1
School of Civil and Transportation Engineering, Guangdong University of Technology,
Guangzhou 510006, China; 13612318991@163.com (Z.Y.); jin15155955794@163.com (Z.J.);
1112009002@mail2.gdut.edu.cn (S.T.)
2
Centre Borelli, ENS-University of Paris-Saclay, 91190 Gif-sur-Yvette, France; david.bassir@utbm.fr
3
UTBM, IRAMAT UMR 7065-CNRS, Rue de Leupe, CEDEX, 90010 Belfort, France
* Correspondence: gongfa.chen@gdut.edu.cn; Tel.: +86-136-6248-3527
Abstract:
A measurement method of bridge vibration by unmanned aerial vehicles (UAVs) combined
with convolutional neural networks (CNNs) and Kanade–Lucas–Tomasi (KLT) optical-flow method is
proposed. In this method, the stationary reference points in the structural background are required, a
UAV is used to shoot the structure video, and the KLT optical-flow method is used to track the target
points on the structure and the background reference points in the video to obtain the coordinates of
these points on each frame. Then, the characteristic relationship between the reference points and
the target points can be learned by a CNN according to the coordinates of the reference points and
the target points, so as to correct the displacement time–history curves of target points containing
the false displacement caused by the UAV’s egomotion. Finally, operational modal analysis (OMA)
is used to extract the natural frequency of the structure from the displacement signal. In addition,
the reliability of UAV measurement combined with CNN is proved by comparing the measurement
results of the fixed camera and those of UAV combined with CNN, and the reliability of the KLT
optical-flow method is proved by comparing the tracking results of the digital image correlation
(DIC) and KLT optical-flow method in the experiment of this paper.
Keywords:
bridge vibration; unmanned aerial vehicles; Kanade–Lucas–Tomasi optical-flow method;
operational modal analysis; convolutional neural networks
1. Introduction
The long-term use of bridges may lead to structural damages; hence, it is necessary to
detect damages regularly. Vibration measurement is an important step in structural damage
detections. In recent years, some noncontact-measurement methods have been proposed,
such as Global Positioning System (GPS) [
1
] and laser Doppler vibrometer (LDV) [
2
], to
replace the traditional contact-measurement methods (such as acceleration sensors [
3
]
and strain gauges [
4
]). However, the GPS is of low accuracy [
5
] and LDV is costly and
time-consuming [
6
]. With the development of computer-vision technology, digital image
correlation (DIC) is more and more widely used in bridge vibration measurement [
7
,
8
].
Compared with GPS and LDV, DIC technology has the advantages of low cost, high preci-
sion, and high efficiency. DIC is also used for deformation and displacement measurement
of other engineering structures [
9
]. However, the measurement accuracy of the DIC method
is limited due to the errors caused by pixel interpolation [
10
]. Also as a computer-vision
method, the optical-flow method is widely used in bridge vibration measurement, and its
accuracy has been confirmed [11,12].
The Kanade–Lucas–Tomasi (KLT) optical-flow method [
13
] is proposed on the basis of
the Lucas–Kanade optical-flow method [
14
]. The concept of optical flow was first proposed
by Gibson [
15
], and represents the velocity of a moving object in a time-varying image.
According to the idea of optical flow, the KLT optical-flow method matches and tracks the
Appl. Sci. 2022, 12, 5181. https://doi.org/10.3390/app12105181 https://www.mdpi.com/journal/applsci
