Citation: Huang, R.; Guan, W.; Zhai,
G.; He, J.; Chu, X. Deep Graph
Reinforcement Learning Based
Intelligent Traffic Routing Control for
Software-Defined Wireless Sensor
Networks. Appl. Sci. 2022, 12, 1951.
https://doi.org/10.3390/app12041951
Academic Editors: Alvaro Araujo
Pinto and Hacene Fouchal
Received: 24 December 2021
Accepted: 9 February 2022
Published: 13 February 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Deep Graph Reinforcement Learning Based Intelligent
Traffic Routing Control for Software-Defined Wireless
Sensor Networks
Ru Huang
1,
* , Wenfan Guan
1
, Guangtao Zhai
2
, Jianhua He
3
and Xiaoli Chu
4
1
School of Information Science & Engineering, East China University of Science and Technology,
Shanghai 200237, China; Y30190690@mail.ecust.edu.cn
2
Institute of Image Communication and Information Processing, Shanghai Jiao Tong University,
Shanghai 200240, China; zhaiguangtao@sjtu.edu.cn
3
School of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ, UK;
j.he@essex.ac.uk
4
Department of Electronic and Electrical Engineering, University of Sheffield, Sheffield S1 3JD, UK;
x.chu@sheffield.ac.uk
* Correspondence: huangrabbit@ecust.edu.cn
Abstract:
Software-defined wireless sensor networks (SDWSN), where the data and control planes
are decoupled, are more suited to handling big sensor data and effectively monitoring dynamic
environments and events. To overcome the limitations of using static routing tables under high traffic
intensity, such as network congestion, high packet loss rate, low throughput, etc., it is critical to
design intelligent traffic routing control for the SDWSNs. In this paper we propose a deep graph
reinforcement learning (DGRL) model-based intelligent traffic control scheme for SDWSNs, which
combines graph convolution with deterministic policy gradient. The model fits well for the task of
intelligent routing control for the SDWSN, as the process of data forwarding can be regarded as the
sampling of continuous action space and the traffic data has strong graph features. The intelligent
control policies are made by the SDWSN controller and implemented at the sensor nodes to optimize
the data forwarding process. Simulation experiments performed on the Omnet++ platform show
that, compared with the existing traffic routing algorithms for SDWSNs, the proposed intelligent
routing control method can effectively reduce packet transmission delay, increase packet delivery
ratio, and reduce the probability of network congestion.
Keywords:
software-defined wireless sensor network; intelligent routing control; deep reinforcement
learning; graph convolutional network
1. Introduction
With major technological advances in communication, computing, and sensing, sensor
networks play important roles for modern society. Sensor nodes help in collecting data
from environment or devices, which can be used to monitor environments, develop and
implement intelligent control systems, such as smart cities, smart factories, and intelligent
surveillance systems. For example, there are about 70 million surveillance cameras in
the U.S.. While the fast-growing number of sensors provide the data needed for big data
analytics and intelligence, the massive data traffic also presents a big challenge for data
transport and networking, e.g., increased network congestion and poor network quality of
services. One of the promising networking approaches to tackle these challenges is software-
defined networking (SDN) [
1
]. In the SDN paradigm, the control plane is decoupled from
the data plane to provide flexible traffic control and simplify the network operation and
management. An investigation of the SDN technology for the Internet of Things (IoT) was
reported in [2,3].
Appl. Sci. 2022, 12, 1951. https://doi.org/10.3390/app12041951 https://www.mdpi.com/journal/applsci
