Citation: Zhai, R.; Yuan, Y. A Method
of Vision Aided GNSS Positioning
Using Semantic Information in
Complex Urban Environment.
Remote Sens. 2022, 14, 869. https://
doi.org/10.3390/rs14040869
Academic Editor: Giuseppe Casula
Received: 29 December 2021
Accepted: 8 February 2022
Published: 11 February 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
A Method of Vision Aided GNSS Positioning Using Semantic
Information in Complex Urban Environment
Rui Zhai
1,2
and Yunbin Yuan
1,
*
1
State Key Laboratory of Geodesy and Earth’s Dynamics, Innovation Academy for Precision Measurement
Science and Technology, Chinese Academy of Sciences, Wuhan 430071, China; zhairui@apm.ac.cn
2
University of Chinese Academy of Sciences, Beijing 100049, China
* Correspondence: yybgps@asch.whigg.ac.cn
Abstract:
High-precision localization through multi-sensor fusion has become a popular research
direction in unmanned driving. However, most previous studies have performed optimally only in
open-sky conditions; therefore, high-precision localization in complex urban environments required
an urgent solution. The complex urban environments employed in this study include dynamic envi-
ronments, which result in limited visual localization performance, and highly occluded environments,
which yield limited global navigation satellite system (GNSS) performance. In order to provide high-
precision localization in these environments, we propose a vision-aided GNSS positioning method
using semantic information by integrating stereo cameras and GNSS into a loosely coupled naviga-
tion system. To suppress the effect of dynamic objects on visual positioning accuracy, we propose a
dynamic-simultaneous localization and mapping (Dynamic-SLAM) algorithm to extract semantic
information from images using a deep learning framework. For the GPS-challenged environment, we
propose a semantic-based dynamic adaptive Kalman filtering fusion (S-AKF) algorithm to develop
vision aided GNSS and achieve stable and high-precision positioning. Experiments were carried out
in GNSS-challenged environments using the open-source KITTI dataset to evaluate the performance
of the proposed algorithm. The results indicate that the dynamic-SLAM algorithm improved the
performance of the visual localization algorithm and effectively suppressed the error spread of the vi-
sual localization algorithm. Additionally, after vision was integrated, the loosely-coupled navigation
system achieved continuous high-accuracy positioning in GNSS-challenged environments.
Keywords:
vision/GNSS integration; adaptive Kalman filter; semantic segmentation; high-precision;
stereo camera
1. Introduction
The Global Navigation Satellite System (GNSS) can provide highly reliable, globally
valid and highly accurate position information, carrier velocities and precise times [
1
,
2
]. It
has gradually become the foundation of most positioning and navigation applications, in-
cluding autonomous driving vehicles (ADVs) [
3
,
4
] and guided weapons. As the positioning
performance of GNSS depends on the continuous tracking of the passible radio signal, the
accuracy, availability, and continuity deteriorate in GNSS-challenged environments [
5
,
6
].
Therefore, the low availability is compensated with additional sensors in complex environ-
ments like urban canyons or tunnels. Additionally, there is an increasing attractiveness to
keeping high-precision positioning for multi-sensor navigation systems when GPS fails
due to automobile and UAS crashes.
In the field of combined navigation, most researches were focused on the integration
of GNSS and inertial navigation system (INS) [
7
–
11
]. With the advances in microelec-
tromechanical system (MEMS) inertial sensor technologies, low-cost GNSS/MEMS-IMU
(inertial measurement units) integration can achieve high-accuracy positioning in open-sky
environments [12–14]. However, the rapid divergence of estimation errors in the low-cost
Remote Sens. 2022, 14, 869. https://doi.org/10.3390/rs14040869 https://www.mdpi.com/journal/remotesensing
