Abstract:In this paper, we consider a point-to-point integrated sensing and communication (ISAC) system, where a transmitter conveys a message to a receiver over a channel with memory and simultaneously estimates the state of the channel through the backscattered signals from the emitted waveform. Using Massey's concept of directed information for channels with memory, we formulate the capacity-distortion tradeoff for the ISAC problem when sensing is performed in an online fashion. Optimizing the transmit waveform for this system to simultaneously achieve good communication and sensing performance is a complicated task, and thus we propose a deep reinforcement learning (RL) approach to find a solution. The proposed approach enables the agent to optimize the ISAC performance by learning a reward that reflects the difference between the communication gain and the sensing loss. Since the state-space in our RL model is à priori unbounded, we employ deep deterministic policy gradient algorithm (DDPG). Our numerical results suggest a significant performance improvement when one considers unbounded state-space as opposed to a simpler RL problem with reduced state-space. In the extreme case of degenerate state-space only memoryless signaling strategies are possible. Our results thus emphasize the necessity of well exploiting the memory inherent in ISAC systems.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in the Integrated Sensing and Communication (ISAC) system, how to optimize the transmit waveform to achieve good communication and sensing performance simultaneously. Specifically, the paper focuses on a point - to - point ISAC system, where the transmitter sends messages to the receiver through a channel with memory characteristics and simultaneously estimates the channel state through the echo signal. The key challenges of this problem are: 1. **Trade - off between communication and sensing**: It is necessary to find a suitable transmit waveform so that it can transmit information efficiently and estimate the channel state accurately. 2. **Online estimation**: The sensing task must be carried out online, that is, immediately estimate the current channel state after each transmission, instead of waiting until the end of communication to estimate. 3. **Memory characteristics of the channel**: The channel has memory characteristics, which means that the current output depends not only on the current input, but also on past inputs and states. To solve these problems, the paper adopts an approach based on Deep Reinforcement Learning (DRL), especially using the Deep Deterministic Policy Gradient (DDPG) algorithm. This method allows the agent to optimize the performance of the ISAC system by learning the reward function, where the reward function reflects the difference between the communication gain and the sensing loss. ### Formula summary - **Capacity - distortion trade - off formula**: \[ C(D)=\lim_{n\rightarrow\infty}\frac{1}{n}\sup_{\{P_{X_{i}|X_{i - 1}Z_{i - 1}}\}_{i = 1}^{n}}\sum_{i = 1}^{n}I(X_{i};Y_{i}|Z_{i - 1}) \] where \(I(X_{i};Y_{i}|Z_{i - 1})\) represents the conditional mutual information, and the constraint condition is: \[ \frac{1}{n}\sum_{i = 1}^{n}E[d(S_{i},g_{i}^{*}(X_{i},Z_{i}))]\leq D \] - **Optimal estimator**: \[ \hat{S}_{i}=\arg\min_{\hat{s}}\sum_{s}P_{S_{i}|X_{i}Z_{i}}(s|x_{i},z_{i})d(s,\hat{s}) \] - **Reward function in MDP**: \[ r_{i}=I(X_{i},U_{i - 1};Y_{i}|Y_{i - 1})-\beta E[d(S_{i},\hat{S}_{i})] \] Through these formulas and methods, the paper shows how to use deep reinforcement learning to optimize the performance of the ISAC system, especially when dealing with channels with memory characteristics. Compared with simplified models (such as finite or degenerate state spaces), the performance in the complete (unbounded) state space is significantly improved.

A Memory-Based Reinforcement Learning Approach to Integrated Sensing and Communication

Improved Information-Theoretic Bound for Multiple-Access Integrated Sensing and Communication Systems

Information-Theoretic Limits of Integrated Sensing and Communication with Correlated Sensing and Channel States for Vehicular Networks

Using Deep Reinforcement Learning to Enhance Channel Sampling Patterns in Integrated Sensing and Communication

End-to-End Learning for SLP-Based ISAC Systems

Deep Learning for Secure Transmission in Reconfigurable Intelligent Surface-Assisted Communications.

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Joint Task Offloading and Resource Allocation for Intelligent Reflecting Surface-Aided Integrated Sensing and Communication Systems Using Deep Reinforcement Learning Algorithm

Enhancing Battlefield Awareness: An Aerial RIS-assisted ISAC System with Deep Reinforcement Learning

An Analysis of Capacity-Distortion Trade-Offs in Memoryless ISAC Systems

Integrated Sensing and Communication in the Finite Blocklength Regime

SINR-Aware Deep Reinforcement Learning for Distributed Dynamic Channel Allocation in Cognitive Interference Networks

Deep Learning-Based Cramér-Rao Bound Optimization for Integrated Sensing and Communication in Vehicular Networks

Reinforcement Learning-powered Semantic Communication via Semantic Similarity

Efficient Communications for Multi-Agent Reinforcement Learning in Wireless Networks

Integrated Sensing and Communications for Low-Altitude Economy: A Deep Reinforcement Learning Approach

Refined-Deep Reinforcement Learning for MIMO Bistatic Backscatter Resource Allocation

Doubly-Dynamic ISAC Precoding for Vehicular Networks: A Constrained Deep Reinforcement Learning (CDRL) Approach

Bistatic Integrated Sensing and Communication over Memoryless Relay Channels

Dynamic Multichannel Sensing in Cognitive Radio: Hierarchical Reinforcement Learning