Citation: Phan, B.C.; Lee, M.-T.; Lai,
Y.-C. Intelligent Deep-Q-Network-
Based Energy Management for an
Isolated Microgrid. Appl. Sci. 2022,
12, 8721. https://doi.org/10.3390/
app12178721
Academic Editors: Luis Hernández-
Callejo, Sara Gallardo Saavedra and
Sergio Nesmachnow
Received: 30 July 2022
Accepted: 27 August 2022
Published: 31 August 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Intelligent Deep-Q-Network-Based Energy Management for an
Isolated Microgrid
Bao Chau Phan
1
, Meng-Tse Lee
2
and Ying-Chih Lai
1,3,
*
1
Department of Aeronautics and Astronautics, College of Engineering, National Cheng Kung University,
Tainan 701, Taiwan
2
Department of Automation Engineering, College of Engineering, National Formosa University,
Yunlin 632, Taiwan
3
Institute of Civil Aviation, National Cheng Kung University, Tainan 701, Taiwan
* Correspondence: yingclai@mail.ncku.edu.tw; Tel.: +886-6-275-7575 (ext. 63648)
Abstract:
The development of hybrid renewable energy systems (HRESs) can be the most feasible
solution for a stable, environment-friendly, and cost-effective power generation, especially in rural
and island territories. In this studied HRES, solar and wind energy are used as the major resources.
Moreover, the electrolyzed hydrogen is utilized to store energy for the operation of a fuel cell. In
case of insufficiency, battery and fuel cell are storage systems that supply energy, while a diesel
generator adds a backup system to meet the load demand under bad weather conditions. An
isolated HRES energy management system (EMS) based on a Deep Q Network (DQN) is introduced
to ensure the reliable and efficient operation of the system. A DQN can deal with the problem
of continuous state spaces and manage the dynamic behavior of hybrid systems without exact
mathematical models. Following the power consumption data from Basco island of the Philippines,
HOMER software is used to calculate the capacity of each component in the proposed power plant.
In MATLAB/Simulink, the plant and its DQN-based EMS are simulated. Under different load profile
scenarios, the proposed method is compared to the convectional dispatch (CD) control for a validation.
Based on the outstanding performances with fewer fuel consumption, DQN is a very powerful and
potential method for energy management.
Keywords:
hybrid renewable energy system (HRES); isolated microgrid; energy management system
(EMS); Deep Q Network (DQN); HOMER software
1. Introduction
The worldwide increase in energy demand leads to the consideration of using renew-
able energy types such as solar, wind, tidal, and geothermal. Currently, fossil fuels are
still the major reliable power sources especially for rural and island electrification. On the
other hand, fossil fuel price is constantly increasing, and fossil fuels are responsible for
global environmental pollution. Consequently, many countries have recently opted for the
long-term sustainable development of renewable energy. By 2025, the Ministry of Economic
Affairs (Taiwan) aims at increasing the share of renewable energy to 20% within the total
power generation, as well as phasing out nuclear energy. Several developing countries such
as Philippines, Thailand, and Vietnam have changed their power development plan based
on green energy. We consider them some of the most typical countries for the deployment
of renewable energy power plants [1].
The recent development of solar and wind energy has recently been considered
because of the available amount of solar radiation and wind distribution. These energy
types are environment-friendly and cost effective, but unpredictable and uncontrollable
as well due to the significant dependence on weather conditions. In order to improve the
operational ability and efficiency of these power systems, the concept of a hybrid renewable
energy system (HRES) was created [
2
]. In terms of power generation for rural and island
Appl. Sci. 2022, 12, 8721. https://doi.org/10.3390/app12178721 https://www.mdpi.com/journal/applsci
