Abstract:In this paper, we present an innovative method for replacing wired communication within an operational agile production cell with wireless communication, using a minimal number of access points. Our approach utilizes Reinforcement Learning (RL) to optimize the layout and minimize the required number of access points. We introduce an Artificial Intelligence (AI) agent that facilitates cooperative interaction between cameras and access points, optimizing camera stream utilization and radio coverage. This solution benefits from the recent standardization of The fifth generation of mobile communications (5G) Network Resource Management (NRM) features of the User Equipment (UE) attached to the cameras. To accommodate the dynamic nature of the production cell, our optimization algorithms run during cell operation, necessitating extended training times. Our contributions extend beyond the state-of-the-art by offering a Digital Twin (DT) environment equipped with physics and radio propagation capabilities, fast raytracing for radio propagation modeling, curriculum-learning-based reinforcement learning, and a Physical Twin (PT) of the radio-connected camera with self-adjusting scene setup capabilities. This comprehensive proposal combines the efficiency of 5G radio utilization with precise modeling and high-resolution simulations, while also significantly improving training speed. In the Agile Robotics for Industrial Automation Competition (ARIAC) 2020 environment, our methods showed compelling results, with curriculum learning achieving a remarkable 3x speed-up compared to non-curriculum approaches. We also demonstrated that high Quality of Service (QoS) demands can be reduced to just 2% of total time while maintaining consistent productivity Key Performance Indicators (KPIs). Furthermore, we explored the integration of these methods into Internet of Things (IoT) devices with limited performance capabilities. In conclusion, our approach offers significant benefits and savings for IoT devices, networks, edge clouds, and industry players.

Intelligent wireless resource management in industrial camera systems: Reinforcement Learning-based AI-extension for efficient network utilization

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

Maximizing throughput in NOMA-enable industrial IoT networks using digital twin and reinforcement learning

Multiagent RL Aided Task Offloading and Resource Management in Wi-Fi 6 and 5G Coexisting Industrial Wireless Environment

Joint Trajectory and Radio Resource Optimization for Autonomous Mobile Robots Exploiting Multi-Agent Reinforcement Learning

Multi-agent deep reinforcement learning for end—edge orchestrated resource allocation in industrial wireless networks

Learning-Based Energy-Efficient Resource Management by Heterogeneous RF/VLC for Ultra-Reliable Low-Latency Industrial IoT Networks

Deep Reinforcement Learning Enables Joint Trajectory and Communication in Internet of Robotic Things

Reinforcement Learning based Multi-connectivity Resource Allocation in Factory Automation Systems

Green Deep Reinforcement Learning for Radio Resource Management: Architecture, Algorithm Compression and Challenge

Deployment options of AI components for network resource management in 5G‐enabled agile industrial production cell

Digital Twin and Meta RL Empowered Fast-Adaptation of Joint User Scheduling and Task Offloading for Mobile Industrial IoT

Deep Reinforcement Learning Multi-Agent System for Resource Allocation in Industrial Internet of Things

Digital twin-based reinforcement learning framework: application to autonomous mobile robot dispatching

Efficient and Flexible Management for Industrial Internet of Things: A Federated Learning Approach.

Visualizing Multi-Agent Reinforcement Learning for Robotic Communication in Industrial IoT Networks

Toward Enhanced Reinforcement Learning-Based Resource Management via Digital Twin: Opportunities, Applications, and Challenges

Multi-agent reinforcement learning for intelligent resource allocation in IIoT networks

Resource-Efficient Sensor Data Management for Autonomous Systems Using Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning Based Resource Allocation for Ultra-Reliable Low-Latency Internet of Controllable Things