Citation: Alibabaei, K.; Gaspar, P.D.;
Assunção, E.; Alirezazadeh, S.;
Lima, T.M.; Soares, V.N.G.J.;
Caldeira, J.M.L.P. Comparison of
On-Policy Deep Reinforcement
Learning A2C with Off-Policy DQN
in Irrigation Optimization: A Case
Study at a Site in Portugal. Computers
2022, 11, 104. https://doi.org/
10.3390/computers11070104
Academic Editors: Phivos Mylonas,
Katia Lida Kermanidis and Manolis
Maragoudakis
Received: 30 May 2022
Accepted: 21 June 2022
Published: 24 June 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Comparison of On-Policy Deep Reinforcement Learning A2C
with Off-Policy DQN in Irrigation Optimization: A Case Study
at a Site in Portugal
Khadijeh Alibabaei
1,2
, Pedro D. Gaspar
1,2
, Eduardo Assunção
1,2
, Saeid Alirezazadeh
3
,
Tânia M. Lima
1,2
, Vasco N. G. J. Soares
4,5,
* and João M. L. P. Caldeira
4,5
1
C-MAST Center for Mechanical and Aerospace Science and Technologies, University of Beira Interior,
6201-001 Covilhã, Portugal; k.alibabaei@ubi.pt (K.A.); dinis@ubi.pt (P.D.G.); eduardo.assuncao@ubi.pt (E.A.);
tmlima@ubi.pt (T.M.L.)
2
Department of Electromechanical Engineering, University of Beira Interior, Rua Marquês d’Ávila e Bolama,
6201-001 Covilhã, Portugal
3
C4—Cloud Computing Competence Centre (C4-UBI), University of Beira Interior, Rua Marquês d’Ávila e
Bolama, 6201-001 Covilhã, Portugal; saeid.zadeh@ubi.pt
4
Polytechnic Institute of Castelo Branco, Av. Pedro Álvares Cabral nº 12, 6000-084 Castelo Branco, Portugal;
jcaldeira@ipcb.pt
5
Instituto de Telecomunicações, Rua Marquês d’Ávila e Bolama, 6201-001 Covilhã, Portugal
* Correspondence: vasco.g.soares@ipcb.pt
Abstract:
Precision irrigation and optimization of water use have become essential factors in agricul-
ture because water is critical for crop growth. The proper management of an irrigation system should
enable the farmer to use water efficiently to increase productivity, reduce production costs, and maxi-
mize the return on investment. Efficient water application techniques are essential prerequisites for
sustainable agricultural development based on the conservation of water resources and preservation
of the environment. In a previous work, an off-policy deep reinforcement learning model, Deep
Q-Network, was implemented to optimize irrigation. The performance of the model was tested
for tomato crop at a site in Portugal. In this paper, an on-policy model, Advantage Actor–Critic,
is implemented to compare irrigation scheduling with Deep Q-Network for the same tomato crop.
The results show that the on-policy model Advantage Actor–Critic reduced water consumption by
20% compared to Deep Q-Network with a slight change in the net reward. These models can be
developed to be applied to other cultures with high production in Portugal, such as fruit, cereals, and
wine, which also have large water requirements.
Keywords:
agriculture; deep learning; on-policy deep reinforcement learning; irrigation optimization
1. Introduction
Water deficiency directly or indirectly affects all physiological processes in plants,
some of which have a major impact on crop growth, development, and productivity [
1
,
2
].
The effect of water stress on transpiration, photosynthesis, and the subsequent absorption
of water and nutrients by plants has a profound impact on crops and their potential
productivity [1,2].
The Food and Agriculture Organization (FAO) reports that agriculture is the sector
where the greatest need for action is to reduce water consumption, as about 60% of the water
used for irrigation is lost as waste [
3
]. The same studies indicate that reducing this loss by
10% would be enough to supply twice the current world population, based on statistical
averages [
3
]. Hence, an efficient water management system is essential. With the use of
the Internet of Things (IoT) [
4
] in agriculture, systems are being developed to effectively
manage fields [
5
,
6
]. The IoT sensors enable monitoring of light, humidity, temperature, soil
moisture, and analysis of water among other parameters [5–9].
Computers 2022, 11, 104. https://doi.org/10.3390/computers11070104 https://www.mdpi.com/journal/computers
