Deep Reinforcement Learning Control for Radar Detection and Tracking in Congested Spectral Environments

Charles E. Thornton,Mark A. Kozy,R. Michael Buehrer,Anthony F. Martone,Kelly D. Sherbondy
DOI: https://doi.org/10.1109/tccn.2020.3019605
IF: 6.359
2020-12-01
IEEE Transactions on Cognitive Communications and Networking
Abstract:This work addresses dynamic non-cooperative coexistence between a cognitive pulsed radar and nearby communications systems by applying nonlinear value function approximation via deep reinforcement learning (Deep RL) to develop a policy for optimal radar performance. The radar learns to vary the bandwidth and center frequency of its linear frequency modulated (LFM) waveforms to mitigate interference with other systems for improved target detection performance while also sufficiently utilizing available frequency bands to achieve a fine range resolution. We demonstrate that this approach, based on the Deep ${Q}$ -Learning (DQL) algorithm, enhances several radar performance metrics more effectively than policy iteration or sense-and-avoid (SAA) approaches in several realistic coexistence environments. The DQL-based approach is also extended to incorporate Double ${Q}$ -learning and a recurrent neural network to form a Double Deep Recurrent ${Q}$ -Network (DDRQN), which yields favorable performance and stability compared to DQL and policy iteration. The practicality of the proposed scheme is demonstrated through experiments performed on a software defined radar (SDRadar) prototype system. Experimental results indicate that the proposed Deep RL approach significantly improves radar detection performance in congested spectral environments compared to policy iteration and SAA.
telecommunications
What problem does this paper attempt to address?