Citation: Wang, S.; Liu, Z.; Xie, R.;
Ran, L. Reinforcement Learning for
Compressed-Sensing Based
Frequency Agile Radar in the
Presence of Active Interference.
Remote Sens. 2022, 14, 968. https://
doi.org/10.3390/rs14040968
Academic Editors: Yangquan Chen,
Subhas Mukhopadhyay,
Nunzio Cennamo, M. Jamal Deen,
Junseop Lee and Simone Morais
Received: 21 January 2022
Accepted: 14 February 2022
Published: 16 February 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Reinforcement Learning for Compressed-Sensing Based
Frequency Agile Radar in the Presence of Active Interference
Shanshan Wang, Zheng Liu *, Rong Xie and Lei Ran
National Laboratory of Radar Signal Processing, Xidian University, Xi’an 710071, China;
ssw@stu.xidian.edu.cn (S.W.); rxie@mail.xidian.edu.cn (R.X.); rl@xidian.edu.cn (L.R.)
* Correspondence: lz@xidian.edu.cn
Abstract:
Compressed sensing (CS)-based frequency agile radar (FAR) is attractive due to its superior
data rate and target measurement performance. However, traditional frequency strategies for CS-
based FAR are not cognitive enough to adapt well to the increasingly severe active interference
environment. In this paper, we propose a cognitive frequency design method for CS-based FAR using
reinforcement learning (RL). Specifically, we formulate the frequency design of CS-based FAR as a
model-free partially observable Markov decision process (POMDP) to cope with the non-cooperation
of the active interference environment. Then, a recognizer-based belief state computing method is
proposed to relieve the storage and computation burdens in solving the model-free POMDP. This
method is independent of the environmental knowledge and robust to the sensing scenario. Finally,
the double deep Q network-based method using the exploration strategy integrating the CS-based
recovery metric into the
e
-greedy strategy (DDQN-CSR-
e
-greedy) is proposed to solve the model-free
POMDP. This can achieve better target measurement performance while avoiding active interference
compared to the existing techniques. A number of examples are presented to demonstrate the
effectiveness and advantage of the proposed design.
Keywords:
compressed-sensing-based frequency agile radar; cognitive design; anti-interference;
target measurement
1. Introduction
In electronic warfare scenarios, hostile jammers emit active interference by intercepting
and imitating radar signals [1,2], having a significant negative effect on radar functioning.
Hence, it is necessary to equip radar systems with anti-jamming techniques. In addition,
since an ever-growing number of electromagnetic systems require access to the limited
frequency resource, especially after the wide deployment of the fifth generation (5G),
minimizing the active co-frequency interference between different radiators becomes an
attractive consideration. Frequency agile radar (FAR), which transmits pulses with dif-
ferent carrier frequencies in a coherent processing interval (CPI), possesses anti-jamming
capabilities and has the potential to realize spectrum compatibility [
3
,
4
]. Random fre-
quency is a common strategy for FAR with the thumbtack-type ambiguity function, but
it cannot avoid active interference flexibly due to the lack of utilization of environmental
information. Sense-and-avoid (SAA) techniques can be employed to select unoccupied
frequency bands automatically [
5
,
6
], but the selection is based on the active interference
knowledge sensed in the previous time. This cannot handle the anti-interference design
in a dynamically changing environment. Therefore, it is of great significance to learn the
interference dynamics and design a more cognitive frequency strategy for FAR.
Reinforcement learning (RL) is a branch of machine learning that aims at making the
agent learn a control strategy through interaction with the environment [
7
–
9
]. It has been
widely studied in the cognitive communication field to learn spectrum sense and access
strategies [
10
,
11
]. Inspired by these investigations, some researchers have attempted to
Remote Sens. 2022, 14, 968. https://doi.org/10.3390/rs14040968 https://www.mdpi.com/journal/remotesensing
