Abstract:Today's advanced automotive systems are turning into intelligent Cyber-Physical Systems (CPS), bringing computational intelligence to their cyber-physical context. Such systems power advanced driver assistance systems (ADAS) that observe a vehicle's surroundings for their functionality. However, such ADAS have clear limitations in scenarios when the direct line-of-sight to surrounding objects is occluded, like in urban areas. Imagine now automated driving (AD) systems that ideally could benefit from other vehicles' field-of-view in such occluded situations to increase traffic safety if, for example, locations about pedestrians can be shared across vehicles. Current literature suggests vehicle-to-infrastructure (V2I) via roadside units (RSUs) or vehicle-to-vehicle (V2V) communication to address such issues that stream sensor or object data between vehicles. When considering the ongoing revolution in vehicle system architectures towards powerful, centralized processing units with hardware accelerators, foreseeing the onboard presence of large language models (LLMs) to improve the passengers' comfort when using voice assistants becomes a reality. We are suggesting and evaluating a concept to complement the ego vehicle's field-of-view (FOV) with another vehicle's FOV by tapping into their onboard LLM to let the machines have a dialogue about what the other vehicle ``sees''. Our results show that very recent versions of LLMs, such as GPT-4V and GPT-4o, understand a traffic situation to an impressive level of detail, and hence, they can be used even to spot traffic participants. However, better prompts are needed to improve the detection quality and future work is needed towards a standardised message interchange format between vehicles.

Embodied AI-Enhanced Vehicular Networks: An Integrated Large Language Models and Reinforcement Learning Method

Deep Learning based Wireless Resource Allocation with Application to Vehicular Networks

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles

Receive, Reason, and React: Drive as You Say, With Large Language Models in Autonomous Vehicles

Reinforcement Learning in V2I Communication Assisted Autonomous Driving

Multi-agent Reinforcement Learning Based Resource Allocation for Vehicular Networks.

Hybrid LLM-DDQN based Joint Optimization of V2I Communication and Autonomous Driving

Optimization of Resource Allocation for V2X Security Communication based on Multi-Agent Reinforcement Learning

DNN Partitioning, Task Offloading, and Resource Allocation in Dynamic Vehicular Networks: A Lyapunov-Guided Diffusion-Based Reinforcement Learning Approach

Resource Allocation for Delay-Sensitive Vehicle-to-Multi-Edges (v2es) Communications in Vehicular Networks: A Multi-Agent Deep Reinforcement Learning Approach.

Enhancing the Minimum Awareness Failure Distance in V2X Communications: A Deep Reinforcement Learning Approach

Augmented Mixed Vehicular Platoon Control with Dense Communication Reinforcement Learning for Traffic Oscillation Alleviation

Tapping in a Remote Vehicle's onboard LLM to Complement the Ego Vehicle's Field-of-View

Improving the Performance of Tasks Offloading for Internet of Vehicles Via Deep Reinforcement Learning Methods

Large Language Model guided Deep Reinforcement Learning for Decision Making in Autonomous Driving

Deep Reinforcement Learning-aided Transmission Design for Energy-efficient Link Optimization in Vehicular Communications

Multi-Agent RL Enables Decentralized Spectrum Access in Vehicular Networks

Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles

Deep Reinforcement Learning Based Intelligent Resource Allocation in Hybrid Vehicle Scenario

Multiagent Deep-Reinforcement-Learning-Based Resource Allocation for Heterogeneous QoS Guarantees for Vehicular Networks