Abstract:Internet of Robotic Things (IoRT) emphasizes the integrated robotic, artificial intelligence computing, and communication technologies, enabling more sophisticated operations and decision-making. As a crucial element of IoRT, mission-critical applications, such as industrial manufacturing and emergency services, impose stringent requirements on ultra-reliable and low-latency communication (URLLC). The paper focuses on addressing URLLC challenges in the context of IoRT, particularly when autonomous mobile robots (AMRs) coexist with static sensors. We prioritize safe and efficient AMRs’ travel through trajectory design and communication resource allocation in IoRT systems without the need of any prior knowledge. To enhance network connectivity and exploit diversity gains, we introduce the flexible decoding and free clustering as the next-generation multiple access technologies in spectrum-limited downlink IoRT system. Then, aiming at minimizing the decoding error probability and travel time, we formulate a long-term multi-objective optimization problem by jointly designing AMRs’ trajectory and communication resource. To accommodate the inherent dynamics and unpredictability in the IoRT system, we introduce a multi-agent actor-critic deep reinforcement learning (DRL) framework, offering four distinct implementations, each accompanied by comprehensive complexity analyses. Simulation results reveal the following insights: 1) in terms of DRL implementations, off-policy algorithms with deterministic policies outperform their on-policy counterparts, achieving approximately a 67% increase in rewards; 2) In terms of communication schemes, our proposed flexible decoding and free clustering strategies under designed trajectories can effectively reduce decoding errors; 3) In terms of algorithm optimality, our DRL framework shows superior flexibility and adaptability in communication environments compared to traditional A* search and heuristic methods.

Optimizing Communication in Deep Reinforcement Learning with XingTian

ZiXia: A Reinforcement Learning Approach via Adjusted Ranking Reward for Internet Congestion Control

Optimizing Deep-Space DTN Congestion Control Via Deep Reinforcement Learning

Deep Reinforcement Learning Enables Joint Trajectory and Communication in Internet of Robotic Things

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Sim-to-Real Optimization of Complex Real World Mobile Network with Imperfect Information via Deep Reinforcement Learning from Self-play

Deep Reinforcement Learning for Computation and Communication Resource Allocation in Multiaccess MEC Assisted Railway IoT Networks

An Offline-Transfer-Online Framework for Cloud-Edge Collaborative Distributed Reinforcement Learning

Deep Reinforcement Learning-aided Transmission Design for Energy-efficient Link Optimization in Vehicular Communications

Decision Transformers for Wireless Communications: A New Paradigm of Resource Management

Optimal parallelization strategies for active flow control in deep reinforcement learning-based computational fluid dynamics

Power Optimization in Device-to-Device Communications: A Deep Reinforcement Learning Approach with Dynamic Reward.

Deep Reinforcement Learning for Online Resource Allocation in IoT Networks: Technology, Development, and Future Challenges

Experience-driven Networking: A Deep Reinforcement Learning based Approach

Robust and energy-efficient RPL optimization algorithm with scalable deep reinforcement learning for IIoT

Deep Reinforcement Learning: Framework, Applications, and Embedded Implementations

Learning Multi-Agent Communication with Double Attentional Deep Reinforcement Learning

A Framework for Mapping DRL Algorithms with Prioritized Replay Buffer onto Heterogeneous Platforms

TCLiVi: Transmission Control in Live Video Streaming Based on Deep Reinforcement Learning

DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agents