Citation: Cheng, X.; Zhang, S.;
Cheng, S.; Xia, Q.; Zhang, J.
Path-Following and Obstacle
Avoidance Control of Nonholonomic
Wheeled Mobile Robot Based on
Deep Reinforcement Learning. Appl.
Sci. 2022, 12, 6874. https://doi.org/
10.3390/app12146874
Academic Editor: Dario Richiedei
Received: 24 May 2022
Accepted: 5 July 2022
Published: 7 July 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Path-Following and Obstacle Avoidance Control of
Nonholonomic Wheeled Mobile Robot Based on Deep
Reinforcement Learning
Xiuquan Cheng
1
, Shaobo Zhang
2
, Sizhu Cheng
1
, Qinxiang Xia
2,
* and Junhao Zhang
1,2
1
Guangzhou Civil Aviation College, Guangzhou 510403, China; chengxiuquan@gcac.edu.cn (X.C.);
chengsizhu@gcac.edu.cn (S.C.); junhaozhang28@163.com (J.Z)
2
School of Mechanical & Automotive Engineering, South China University of Technology,
Guangzhou 510641, China; shaobozhang5@163.com
* Correspondence: meqxxia@scut.edu.cn
Abstract:
In this paper, a novel path-following and obstacle avoidance control method is given for
nonholonomic wheeled mobile robots (NWMRs), based on deep reinforcement learning. The model
for path-following is investigated first, and then applied to the proposed reinforcement learning
control strategy. The proposed control method can achieve path-following control through interacting
with the environment of the set path. The path-following control method is mainly based on the
design of the state and reward function in the training of the reinforcement learning. For extra
obstacle avoidance problems in following, the state and reward function is redesigned by utilizing
both distance and directional perspective aspects, and a minimum representative value is proposed
to deal with the occurrence of multiple obstacles in the path-following environment. Through
the reinforcement learning algorithm deep deterministic policy gradient (DDPG), the NWMR can
gradually achieve the path it is required to follow and avoid the obstacles in simulation experiments,
and the effectiveness of the proposed algorithm is verified.
Keywords: path-following; obstacle avoidance; NWMRs; reinforcement learning; DDPG
1. Introduction
Path-following has been considered as an alternative problem formulation for trajec-
tory tracking problems [
1
]. The main task of path-following is to develop control laws for
following a predefined path with minimum position error. In contrast to the trajectory
tracking problem, path-following research focuses on the fact that the path is specified by a
relatively independent timing control law, making it more flexible in terms of the control
of the tracked object. Therefore, the path-following problem has been extensively studied
in the field of control, for applications such as wheeled mobile robots [
2
,
3
], autonomous
underwater vehicles [4,5], and quadrotors [6,7].
Currently, numerous control methods have been referenced in the study of path-
following problems, such as guiding vector field (GVF) [
2
,
8
], model predictive control
(MPC) [
7
,
9
], sliding mode control (SMC) [
2
], etc. The GVF approach has been proposed to
achieve path-following for a nonholonomic mobile robot, and global convergence condi-
tions were established to demonstrate the proposed algorithm [
8
]. Linear constrained MPC
has been proposed to solve the path-following problem for quadrotor unmanned aerial
vehicles [
7
]. There are also studies on model predictive control methods using models for
other control strategies; for instance, the information-aware Lyapunov-based MPC strat-
egy was utilized to achieve classic robot control tasks in a feedback–feedforward control
scheme [
9
]. A nonsingular terminal sliding mode control scheme was constructed to solve
the problem of the omnidirectional mobile robot with mecanum wheels [
2
]. There are
also numerous intelligent computing methods that have been widely used in the research
Appl. Sci. 2022, 12, 6874. https://doi.org/10.3390/app12146874 https://www.mdpi.com/journal/applsci
