Abstract:The frequency hopping pattern of the existing frequency hopping communication system is not designed according to the electromagnetic interference environment, resulting in blind anti-jamming. Therefore, to address this problem, a “three-variable” frequency-hopping pattern is proposed, where the frequency, hopping rate, and instantaneous bandwidth of the frequency-hopping signal vary randomly based on the background electromagnetic interference. The decision-making problem of the “three-variable” frequency-hopping pattern is modeled as a Markov decision process (MDP) by constructing the state-action-reward tuple. The designed frequency varies continuously within a small frequency band selected from a pseudo-random sequence to alleviate the problem of dimension explosion in decision-making. At the same time, discrete values for the hopping rate and instantaneous bandwidth are designed. To solve this MDP problem efficiently, a combined deep reinforcement learning algorithm (OC-CDRL) based on optimistic exploration and conservative estimation is proposed, which combines the features of TD3 and D3QN algorithms and designs the corresponding states, actions, and rewards to deal with continuous and discrete action spaces, respectively. To address the problem that the D3QN algorithm tends to fall into local optimal solutions, an optimistic exploration strategy (OES) for action selection is proposed to improve the degree of exploration. Moreover, the loss function is improved by conservatively estimating state–action pairs outside the experience replay buffer, reducing the overestimation of the optimistic action-value function and increasing the stability and convergence of the algorithm. Comparative simulation results of the algorithms in different electromagnetic interference environments show that the OC-CDRL algorithm effectively avoids most regions with higher interference and has better adaptability and anti-jamming capability.

Intelligent Anti-jamming Decision Algorithm for Frequency Hopping Network Based on Multi-agent Fuzzy Deep Reinforcemnet Learning

An Intelligent Strategy Decision Method for Collaborative Jamming Based On Hierarchical Multi-Agent Reinforcement Learning

Multi-Agent Reinforcement Learning Based UAV Swarm Communications Against Jamming

Intelligent Frequency Decision Communication with Two-Agent Deep Reinforcement Learning

Intelligent Decision-Making for a “Three-Variable” Frequency-Hopping Pattern Based on OC-CDRL

Intelligent Anti-Jamming Decision with Continuous Action and State in Bivariate Frequency Agility Communication System

A New Frequency Hopping Strategy Based on Federated Reinforcement Learning for FANET

Intelligent Anti-Jamming Decision Algorithm of Bivariate Frequency Hopping Pattern Based on DQN with PER and Pareto

Deep Reinforcement Learning-Based Anti-Jamming Algorithm Using Dual Action Network

Intelligent Anti-Jamming Communication with Continuous Action Decision for Ultra-Dense Network

Joint Channel Selection And Data Scheduling In Hf Jamming Environment: An Interference-Aware Reinforcement Learning Approach

Efficient Jamming Resource Allocation Against Frequency-Hopping Spread Spectrum in WSNs with Asynchronous Deep Reinforcement Learning

Radar Anti-Jamming Decision-Making Method Based on DDPG-MADDPG Algorithm

A Heterogeneous Information Fusion Deep Reinforcement Learning for Intelligent Frequency Selection of HF Communication

Deep Reinforcement Learning Based Multi-User Anti Jamming Strategy

Deep Reinforcement Learning Based Hopping Strategy for Wideband Anti-Jamming Wireless Communications

Intelligent Anti-Jamming Communication for Wireless Sensor Networks: A Multi-Agent Reinforcement Learning Approach

Frequency Diversity Array Radar and Jammer Intelligent Frequency Domain Power Countermeasures Based on Multi-Agent Reinforcement Learning

An Intelligent Frequency Decision Method for a Frequency Agile Radar Based on Deep Reinforcement Learning

Against Jamming Attack in Wireless Communication Networks: A Reinforcement Learning Approach

A Centralized Multi-User Anti-Composite Intelligent Interference Algorithm Based on Improved Q-Learning