Abstract:Internet of Robotic Things (IoRT) emphasizes the integrated robotic, artificial intelligence computing, and communication technologies, enabling more sophisticated operations and decision-making. As a crucial element of IoRT, mission-critical applications, such as industrial manufacturing and emergency services, impose stringent requirements on ultra-reliable and low-latency communication (URLLC). The paper focuses on addressing URLLC challenges in the context of IoRT, particularly when autonomous mobile robots (AMRs) coexist with static sensors. We prioritize safe and efficient AMRs’ travel through trajectory design and communication resource allocation in IoRT systems without the need of any prior knowledge. To enhance network connectivity and exploit diversity gains, we introduce the flexible decoding and free clustering as the next-generation multiple access technologies in spectrum-limited downlink IoRT system. Then, aiming at minimizing the decoding error probability and travel time, we formulate a long-term multi-objective optimization problem by jointly designing AMRs’ trajectory and communication resource. To accommodate the inherent dynamics and unpredictability in the IoRT system, we introduce a multi-agent actor-critic deep reinforcement learning (DRL) framework, offering four distinct implementations, each accompanied by comprehensive complexity analyses. Simulation results reveal the following insights: 1) in terms of DRL implementations, off-policy algorithms with deterministic policies outperform their on-policy counterparts, achieving approximately a 67% increase in rewards; 2) In terms of communication schemes, our proposed flexible decoding and free clustering strategies under designed trajectories can effectively reduce decoding errors; 3) In terms of algorithm optimality, our DRL framework shows superior flexibility and adaptability in communication environments compared to traditional A* search and heuristic methods.

Deep Reinforcement Learning for Online Resource Allocation in IoT Networks: Technology, Development, and Future Challenges

Deep Reinforcement Learning for Internet of Things: A Comprehensive Survey

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Deep Reinforcement Learning-Based Resource Allocation for Content Distribution in IoT-Edge-Cloud Computing Environments

Deep Reinforcement Learning for Computation and Communication Resource Allocation in Multiaccess MEC Assisted Railway IoT Networks

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

Deep Multiagent Reinforcement-Learning-Based Resource Allocation for Internet of Controllable Things

Deep Reinforcement Learning Enables Joint Trajectory and Communication in Internet of Robotic Things

Deep Reinforcement Learning (DRL)-based Resource Management in Software-Defined and Virtualized Vehicular Ad Hoc Networks

Deep Reinforcement Learning Multi-Agent System for Resource Allocation in Industrial Internet of Things

DeepEdge: A New QoE-Based Resource Allocation Framework Using Deep Reinforcement Learning for Future Heterogeneous Edge-IoT Applications

Toward Enhanced Reinforcement Learning-Based Resource Management via Digital Twin: Opportunities, Applications, and Challenges

Deep Reinforcement Learning-Based Resource Allocation for Satellite Internet of Things with Diverse QoS Guarantee

Distributed Deep Reinforcement Learning for Resource Allocation in Digital Twin Networks

A DRL-Driven Intelligent Optimization Strategy for Resource Allocation in Cloud-Edge-End Cooperation Environments

Deep Reinforcement Learning for Resource Management in Network Slicing

Deep Reinforcement Learning: Framework, Applications, and Embedded Implementations

Autonomous Resource Management in Construction Companies Using Deep Reinforcement Learning Based on IoT

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Broad Reinforcement Learning for Supporting Fast Autonomous IoT