Abstract:Internet of Robotic Things (IoRT) emphasizes the integrated robotic, artificial intelligence computing, and communication technologies, enabling more sophisticated operations and decision-making. As a crucial element of IoRT, mission-critical applications, such as industrial manufacturing and emergency services, impose stringent requirements on ultra-reliable and low-latency communication (URLLC). The paper focuses on addressing URLLC challenges in the context of IoRT, particularly when autonomous mobile robots (AMRs) coexist with static sensors. We prioritize safe and efficient AMRs’ travel through trajectory design and communication resource allocation in IoRT systems without the need of any prior knowledge. To enhance network connectivity and exploit diversity gains, we introduce the flexible decoding and free clustering as the next-generation multiple access technologies in spectrum-limited downlink IoRT system. Then, aiming at minimizing the decoding error probability and travel time, we formulate a long-term multi-objective optimization problem by jointly designing AMRs’ trajectory and communication resource. To accommodate the inherent dynamics and unpredictability in the IoRT system, we introduce a multi-agent actor-critic deep reinforcement learning (DRL) framework, offering four distinct implementations, each accompanied by comprehensive complexity analyses. Simulation results reveal the following insights: 1) in terms of DRL implementations, off-policy algorithms with deterministic policies outperform their on-policy counterparts, achieving approximately a 67% increase in rewards; 2) In terms of communication schemes, our proposed flexible decoding and free clustering strategies under designed trajectories can effectively reduce decoding errors; 3) In terms of algorithm optimality, our DRL framework shows superior flexibility and adaptability in communication environments compared to traditional A* search and heuristic methods.

Autonomous Rate Control for Mobile Internet of Things: A Deep Reinforcement Learning Approach

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Deep Reinforcement Learning for Online Resource Allocation in IoT Networks: Technology, Development, and Future Challenges

Deep Multiagent Reinforcement-Learning-Based Resource Allocation for Internet of Controllable Things

Deep Reinforcement Learning for Computation and Communication Resource Allocation in Multiaccess MEC Assisted Railway IoT Networks

Intelligent User Association for Symbiotic Radio Networks Using Deep Reinforcement Learning

Deep Reinforcement Learning Enables Joint Trajectory and Communication in Internet of Robotic Things

Deep Reinforcement Learning-Based Resource Allocation for Satellite Internet of Things with Diverse QoS Guarantee

Distributed DRL for Device Access in Symbiotic Radio-Aided High-Speed Railway Networks.

Intelligent Routing Strategy in the Internet of Things Based on Deep Reinforcement Learning

Dynamic multiple access based on deep reinforcement learning for Internet of Things

Energy-efficient deep Q-network: reinforcement learning for efficient routing protocol in wireless internet of things

Energy‐efficient resource allocation over wireless communication systems through deep reinforcement learning

Deep Reinforcement Learning Driven Joint Dynamic TDD and RRC Connection Management Scheme in Massive IoT Networks

A Joint Strategy for CUAV-based Traffic Offloading Via Deep Reinforcement Learning

Deep Reinforcement Learning for Multi-User Access Control in Non-Terrestrial Networks.

Deep Reinforcement Learning Optimal Transmission Algorithm for Cognitive Internet of Things with RF Energy Harvesting

Deep reinforcement learning-based task scheduling and resource allocation for NOMA-MEC in Industrial Internet of Things

Deep Reinforcement Learning for Channel and Power Allocation in UAV-enabled IoT Systems

Deep Reinforcement Learning for Communication Flow Control in Wireless Mesh Networks