Citation: Asaf, K.; Khan, B.; Kim,
G.-Y. Wireless Lan Performance
Enhancement Using Double Deep
Q-Networks. Appl. Sci. 2022, 12, 4145.
https://doi.org/10.3390/
app12094145
Academic Editor: Amadeo
Benavent-Climent
Received: 11 March 2022
Accepted: 15 April 2022
Published: 20 April 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Wireless Lan Performance Enhancement Using Double
Deep Q-Networks
Khizra Asaf
1,†
, Bilal Khan
1,†
and Ga-Young Kim
2,
*
1
Department of Computer Science, National University of Computer and Emerging Sciences,
Chiniot-Faisalabad Campus, Chiniot 35400, Pakistan; khizraasaf94@gmail.com (K.A.);
khan.bilal@nu.edu.pk (B.K.)
2
Faculty of General Education, Kangnam University, Yongin-si 16979, Korea
* Correspondence: dolga2000@kangnam.ac.kr
† These authors contributed equally to this work.
Abstract:
Due to the exponential growth in the use of Wi-Fi networks, it is necessary to study its
usage pattern in dense environments for which the legacy IEEE 802.11 MAC (Medium Access Control)
protocol was not specially designed. Although 802.11ax aims to improve Wi-Fi performance in dense
scenarios due to modifications in the physical layer (PHY), however, MAC layer operations remain
unchanged, and are not capable enough to provide stable performance in dense scenarios. Potential
applications of Deep Learning (DL) to Media Access Control (MAC) layer of WLAN has now been
recognized due to their unique features. Deep Reinforcement Learning (DRL) is a technique focused
on behavioral sensitivity and control philosophy. In this paper, we have proposed an algorithm
for setting optimal contention window (CW) under different network conditions called DRL-based
Contention Window Optimization (DCWO). The proposed algorithm operates in three steps. In the
initial step, Wi-Fi is being controlled by the 802.11 standards. In the second step, the agent makes
the decisions concerning the value of CW after the TRAIN procedure for the proposed algorithm.
The final phase begins after the training, defined by a time duration specified by the user. Now, the
agent is fully trained, and no updates will be no longer received. Now the CW is updated via the
OPTIMIZE process of DCWO. We have selected total network throughput, instantaneous network
throughput, fairness index, and cumulative reward, and compared our proposed scheme DCWO
with the Centralized Contention window Optimization with DRL (CCOD). Simulation results show
that DCWO with Double Deep Q-Networks (DDQN) performs better than CCOD with (i) Deep
Deterministic Policy Gradient (DDPG) and (ii) Deep Q-Network (DQN). More specifically, DCWO
with DDQN gives on average 28% and 23% higher network throughput than CCOD in static and
dynamic scenarios. Whereas in terms of instantaneous network throughput DCWO gives around
10% better results than the CCOD. DCWO achieves almost near to optimal fairness in static scenarios
and better than DQN and DDPG with CCOD in dynamic scenarios. Similarly, while the cumulative
reward achieved by DCWO is almost the same with CCOD with DDPG, the uptrend of DCWO is
still encouraging.
Keywords: WLAN; 802.11; DRL; DDQN; DCWO
1. Introduction
Wireless networks have seen increasing and continuous popularity resulting in in-
creased data traffic over all the networks [
1
]. Wi-Fi networks have experienced incredible
growth concerning traffic consumption. Due to the increased use of mobile devices, it is
expected that 63% of the mobile data traffic will be shifted to Wi-Fi networks by the year
2021 [2].
IEEE 802.11 is a set of progressively improved standards to continue developing
further modifications to overcome the limitations once discovered. Due to flexibility,
Appl. Sci. 2022, 12, 4145. https://doi.org/10.3390/app12094145 https://www.mdpi.com/journal/applsci
