Abstract:Integrated sensing and communication (ISAC) technology plays a crucial role in vehicular networks. However, the communication channel within this context exhibits time-varying characteristics, and potential targets may move rapidly, resulting in double dynamics. These presents significant challenges for real-time ISAC precoding design that have not been thoroughly explored. While optimization-based precoding methods have been extensively studied, they are computationally complex and heavily rely on perfect prior information that is rarely available in situations with double dynamics. In this paper, we propose a synesthesia of machine (SoM)-enhanced precoding paradigm, where the base station leverages various modalities such as positioning and channel information to adapt to double dynamics, and effectively utilizes environmental information to stretch ISAC performance boundaries through a deep reinforcement learning framework. Additionally, a parameter-shared actor-critic architecture is tailored to expedite training in complex state and action spaces. Extensive experimental validation has demonstrated the multifaceted superiority of our method over existing approaches.

What problem does this paper attempt to address?

This paper attempts to address the double - dynamics challenges faced by Integrated Sensing and Communication (ISAC) technology in Vehicular Networks. Specifically: 1. **Time - varying channel characteristics**: In the vehicular network environment, the wireless communication channel exhibits time - varying characteristics, which increases the difficulty of real - time ISAC precoding design. 2. **Fast - moving targets**: Potential targets may move rapidly, further exacerbating the complexity of the system. These problems pose significant challenges to real - time ISAC precoding design. In particular, optimization methods usually rely on perfect prior information, and it is difficult to obtain such information in a double - dynamic environment. Moreover, existing optimization - based precoding methods have high computational complexity and rely heavily on immediate environmental information (such as instantaneous channel state information and real - time target location), which is difficult to achieve in a double - dynamic environment. To solve these problems, the paper proposes a Deep Reinforcement Learning (DRL) framework enhanced by Synesthesia of Machines (SoM). This framework adapts to double - dynamics by leveraging various modal information of the base station (BS) (such as location and channel information) and effectively uses environmental information to expand the ISAC performance boundary. Specifically: - **SoM - enhanced precoding paradigm**: The base station uses various modal information (such as location and channel information) to adapt to double - dynamics and effectively use environmental information. - **Deep Reinforcement Learning framework**: Through the Deep Reinforcement Learning framework, the base station can perform efficient learning and optimization in a complex decision - space. - **Actor - Critic architecture with parameter sharing**: To accelerate training, an Actor - Critic architecture with parameter sharing is designed, which reduces network parameters and speeds up the training. In summary, this paper aims to solve the real - time precoding design problem of ISAC in a double - dynamic environment by introducing the SoM - enhanced DRL framework, thereby improving communication and sensing performance, reducing computational complexity, and reducing dependence on immediate environmental information.

Synesthesia of Machines (SoM)-Enhanced ISAC Precoding for Vehicular Networks with Double Dynamics

Intelligent Multi-Modal Sensing-Communication Integration: Synesthesia of Machines

Information-Theoretic Limits of Integrated Sensing and Communication with Correlated Sensing and Channel States for Vehicular Networks

Improved Information-Theoretic Bound for Multiple-Access Integrated Sensing and Communication Systems

Doubly-Dynamic ISAC Precoding for Vehicular Networks: A Constrained Deep Reinforcement Learning (CDRL) Approach

Extending 5G NR V2X Mode 2 to Enable Integrated Sensing and Communication for Vehicular Networks

Intelligent Surface Enabled Sensing-Assisted Communication

Experimental Study on ISAC Performance with Different Sensing Sequences

Secure Precoding Optimization for NOMA-Aided Integrated Sensing and Communication

Soft Actor-Critic-Based Multi-User Multi-TTI MIMO Precoding in Multi-Modal Real-Time Broadband Communications

Deep-learning methods for integrated sensing and communication in vehicular networks

Collaborative Precoding Design for Adjacent Integrated Sensing and Communication Base Stations

Synesthesia of Machines (SoM)-Enhanced Wideband Multi-User CSI Learning With LiDAR Sensing

Multi-Modal Intelligent Channel Modeling: A New Modeling Paradigm via Synesthesia of Machines

Sensing-Assisted Communication in Vehicular Networks with Intelligent Surface

ISAC-NET: Model-driven Deep Learning for Integrated Passive Sensing and Communication

Semantic Communication for Cooperative Perception based on Importance Map

Double Kill: Compressive-Sensing-Based Narrow-Band Interference and Impulsive Noise Mitigation for Vehicular Communications.

Optimizing Radio Access Technology Selection and Precoding in CV-Aided ISAC Systems

Toward Multiple Integrated Sensing and Communication Base Station Systems: Collaborative Precoding Design with Power Constraint