Citation: Kim, M.; Chung, K.
Reinforcement Learning-Based
Adaptive Streaming Scheme with
Edge Computing Assistance. Sensors
2022, 22, 2171. https://doi.org/
10.3390/s22062171
Academic Editors: Nikolaos Thomos,
Eirina Bourtsoulatze and Yang Yue
Received: 19 January 2022
Accepted: 9 March 2022
Published: 10 March 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Reinforcement Learning-Based Adaptive Streaming Scheme
with Edge Computing Assistance
Minsu Kim and Kwangsue Chung *
Department of Electronics and Communications Engineering, Kwangwoon University, Seoul 01897, Korea;
mskim@cclab.kw.ac.kr
* Correspondence: kchung@kw.ac.kr
Abstract:
Dynamic Adaptive Streaming over HTTP (DASH) is a promising scheme for improving
the Quality of Experience (QoE) of users in video streaming. However, the existing schemes do
not perform coordination among clients and depend on fixed heuristics. In this paper, we propose
an adaptive streaming scheme with reinforcement learning in edge computing environments. The
proposed scheme improves the overall QoE of clients and QoE fairness among clients based on
a state-of-the-art reinforcement learning algorithm. Edge computing assistance plays a role in
providing client-side observations to the mobile edge, making agents utilize this information when
generating a policy for multi-client adaptive streaming. We evaluated the proposed scheme through
simulation-based experiments under various network conditions. The experimental results show
that the proposed scheme achieves better performance than the existing schemes.
Keywords:
Dynamic Adaptive Streaming over HTTP (DASH); Quality of Experience (QoE); mobile
edge computing (MEC); reinforcement learning
1. Introduction
According to the Cisco Annual Internet Report (2018–2023), the total number of global
mobile subscribers will increase from 66% of the population in 2018 to 71% of the population
by 2023 [
1
]. Video services such as Netflix and YouTube contribute to the major portion
of Internet traffic. Since these services deliver videos over the Internet, providing a high
Quality of Experience (QoE) to users is an important challenge in terms of network and
service [
2
]. Dynamic Adaptive Streaming over HTTP (DASH) was standardized in 2011 as
a solution for efficient and smooth video streaming. Using the existing HTTP infrastructure,
DASH adapts the bitrate of video segments delivered over the network to improve resource
utilization and the QoE for users [
3
–
5
]. Moreover, DASH has high scalability owing to the
client-driven scheme that does not impose any modification to the HTTP server. Various
studies based on DASH have been conducted over several years [
6
–
10
]. These schemes
perform bitrate adaptation by determining the video bitrate fit to the measured available
bandwidth, current buffer level, or other predicted conditions.
QoE refers to the degree of delight or annoyance of a user for an application or service.
Many factors associated with network, device, user expectation, and content affect the
QoE. In adaptive streaming, the QoE is determined by considering startup delay, packet
loss rates, average video quality, video re-buffering events, and quality variations, etc.
According to studies for the QoE, a user wants to view a video as clearly and smoothly
as possible. The long startup delay prevents a user from viewing a video smoothly after
the streaming session starts. The packet loss rate and average video quality are related
to the clarity of play-back scenes to be watched by users. Once the video re-buffering
events occur, the QoE is significantly degraded compared with the other influencing factors.
The quality variations leading to abrupt changes of perceived quality also have negative
effect on the QoE. Various schemes have been proposed to measure and evaluate the QoE,
Sensors 2022, 22, 2171. https://doi.org/10.3390/s22062171 https://www.mdpi.com/journal/sensors
