Abstract:Internet of Robotic Things (IoRT) emphasizes the integrated robotic, artificial intelligence computing, and communication technologies, enabling more sophisticated operations and decision-making. As a crucial element of IoRT, mission-critical applications, such as industrial manufacturing and emergency services, impose stringent requirements on ultra-reliable and low-latency communication (URLLC). The paper focuses on addressing URLLC challenges in the context of IoRT, particularly when autonomous mobile robots (AMRs) coexist with static sensors. We prioritize safe and efficient AMRs’ travel through trajectory design and communication resource allocation in IoRT systems without the need of any prior knowledge. To enhance network connectivity and exploit diversity gains, we introduce the flexible decoding and free clustering as the next-generation multiple access technologies in spectrum-limited downlink IoRT system. Then, aiming at minimizing the decoding error probability and travel time, we formulate a long-term multi-objective optimization problem by jointly designing AMRs’ trajectory and communication resource. To accommodate the inherent dynamics and unpredictability in the IoRT system, we introduce a multi-agent actor-critic deep reinforcement learning (DRL) framework, offering four distinct implementations, each accompanied by comprehensive complexity analyses. Simulation results reveal the following insights: 1) in terms of DRL implementations, off-policy algorithms with deterministic policies outperform their on-policy counterparts, achieving approximately a 67% increase in rewards; 2) In terms of communication schemes, our proposed flexible decoding and free clustering strategies under designed trajectories can effectively reduce decoding errors; 3) In terms of algorithm optimality, our DRL framework shows superior flexibility and adaptability in communication environments compared to traditional A* search and heuristic methods.

Intelligent Orchestrating of IoT Microservices Based on Reinforcement Learning

Deep Reinforcement Learning for Online Resource Allocation in IoT Networks: Technology, Development, and Future Challenges

TF-DDRL: A Transformer-enhanced Distributed DRL Technique for Scheduling IoT Applications in Edge and Cloud Computing Environments

Intelligent Routing Strategy in the Internet of Things Based on Deep Reinforcement Learning

DRLS: A Deep Reinforcement Learning Based Scheduler for Time-Triggered Ethernet

Deep Reinforcement Learning for Scheduling in an Edge Computing-Based Industrial Internet of Things

Blockchain-Aided Network Resource Orchestration in Intelligent Internet of Things

GTD3-NET: A deep reinforcement learning-based routing optimization algorithm for wireless networks

IoT Microservice Deployment in Edge-Cloud Hybrid Environment Using Reinforcement Learning

Performance Comparison of Different Deep Reinforcement Learning Algorithms for Task Scheduling Problem in Blockchain-Enabled Internet of Vehicles

A Deep Reinforcement Learning Approach for Composing Moving IoT Services

Intelligent User Association for Symbiotic Radio Networks Using Deep Reinforcement Learning

Optimal Scheduling in IoT-Driven Smart Isolated Microgrids Based on Deep Reinforcement Learning

A hierarchical reinforcement learning approach for energy‐aware service function chain dynamic deployment in IoT

A deep reinforcement learning-based multi-optimality routing scheme for dynamic IoT networks

A DRL-Driven Intelligent Optimization Strategy for Resource Allocation in Cloud-Edge-End Cooperation Environments

Deep Reinforcement Learning Driven Joint Dynamic TDD and RRC Connection Management Scheme in Massive IoT Networks

Joint Service Migration and Resource Allocation in Edge IoT System Based on Deep Reinforcement Learning

Deep Reinforcement Learning Enables Joint Trajectory and Communication in Internet of Robotic Things

Scheduling of decentralized robot services in cloud manufacturing with deep reinforcement learning

A Distributed Intelligent Service Trusted Provision Approach for IoT