Article
A Novel Hybrid Path Planning Method Based on Q-Learning
and Neural Network for Robot Arm
Ali Abdi , Dibash Adhikari and Ju Hong Park *
Citation: Abdi, A.; Adhikari, D.;
Park, J.H. A Novel Hybrid Path
Planning Method Based on
Q-Learning and Neural Network for
Robot Arm. Appl. Sci. 2021, 11, 6770.
https://doi.org/10.3390/app11156770
Academic Editor: Amerigo Capria
Received: 22 June 2021
Accepted: 21 July 2021
Published: 23 July 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Department of Convergence IT Engineering, Pohang University of Science and Technology (POSTECH),
77 Cheongam-ro, Nam-gu, Pohang 37673, Gyeongbuk, Korea; abdiali@postech.ac.kr (A.A.);
dibash@postech.ac.kr (D.A.)
* Correspondence: juhpark@postech.ac.kr; Tel.: +82-54-279-8875
Abstract:
Path planning for robot arms to reach a target and avoid obstacles has had a crucial
role in manufacturing automation. Although many path planning algorithms, including RRT, APF,
PRM, and RL-based, have been presented, they have many problems: a time-consuming process,
high computational costs, slowness, non-optimal paths, irregular paths, failure to find a path, and
complexity. Scholars have tried to address some of these issues. However, those methods still suffer
from slowness and complexity. In order to address these two limitations, this paper presents a new
hybrid path planning method that contains two separate parts: action-finding (active approach)
and angle-finding (passive approach). In the active phase, the Q-learning algorithm is used to
find a sequence of simple actions, including up, down, left, and right, to reach the target cell in a
gridded workspace. In the passive phase, the joints angles of the robot arm, with respect to the found
actions, are obtained by the trained neural network. The simulation and test results show that this
hybrid approach significantly improves the slowness and complexity due to using the simplified
agent-environment interaction in the active phase and simple computing the joints angles in the
passive phase.
Keywords:
path planning; hybrid method; Q-learning; neural network; robot arm; target reaching;
obstacle avoidance
1. Introduction
Currently, the use of robot arms has dramatically increased in various applications,
including manufacturing, logistics, medicine, home, education, defense, and factories. due to
their plentiful benefits such as high quality, fast production, less waste, and excellent safety
(removing employees from hazardous working conditions, and handling heavy parts).
Because of their crucial role, path planning, including target reaching and obstacle
avoidance, is essential. Therefore, this article tries to address the issues of path planning
methods to improve the efficiency of their operation. In this regard, since all robot arms’
activities are dependent on their end-effector, target reaching (i.e., getting the end-effector
to the desired position) is a crucial task for them. Additionally, obstacle avoidance is
another equally vital task for these robots because robot arms must avoid self-collisions
and any objects within their workspace.
For the past decades, scholars have presented many algorithms in robot arm path
planning to improve those tasks. Artificial Potential Field (APF), Probabilistic Road Maps
(PRM), Rapidly exploring Random Tree (RRT), Reinforcement Learning based (RL-based)
are some of the well-known examples.
In this regard, some of the authors used the APF method for path planning. Sheng
Nan Gai et al. presented a whole-arm path planning algorithm for a 6-DOF industrial
robot based on the artificial potential field method [
1
]. Hsien-I Lin et al. proposed a path
planning method and gave an idea for collision avoidance by a three-dimensional artificial
Appl. Sci. 2021, 11, 6770. https://doi.org/10.3390/app11156770 https://www.mdpi.com/journal/applsci
