Abstract:Metaverse has emerged as a revolutionary technique for transforming the way people interact with digital content, which relies on a distributed computing and communication infrastructure, encompassing terminal users, edge servers, and cloud servers. However, the rapid evolution of the Metaverse presents challenges that surpass the capabilities of existing communication and network infrastructures, particularly on network bandwidth and latency. Additionally, human experience becomes a critical factor in this domain. Therefore, we introduce a human-aware hierarchical software defined network (SDN) architecture consisting of a Metaverse cloud layer, a mobile edge computing (MEC) server empowered edge layer, and a distributed terminal layer. Each MEC server dynamically controls a multi-antenna base station (BS) and several reconfigurable intelligent surfaces (RISs) according to the terminal immersive experience requirements in real-time. To overcome the bandwidth limitation, we propose a novel smart reconfigurable spatial reuse new radio in unlicensed spectrum (NR-U) framework, which can realize customizable communications through flexibly and coordinately reconfiguring beams among the coordination between BSs and RISs. The objective function is formulated as a Lyapunov optimization based decentralized partially-observable Markov decision process (Dec-POMDP) problem to maximize the spectral efficiency while guaranteeing the latency and reliability requirements in Metaverse, via a joint user selection, phase-shift control, and beam coordination strategy. To solve the above non-convex, strongly coupled, and mixed integer nonlinear programming (MINLP), we propose a novel multi-agent hierarchical deep reinforcement learning (MAHDRL) algorithm that integrates deep Q-network (DQN) to solve discrete problems, deep deterministic policy gradient (DDPG) to solve continuous problems, and mixing network to capture complex interactions between multiple agents. Numerical results demonstrate the effectiveness of the proposed algorithm and verify the performance improvements compared to traditional multi-agent deep reinforcement learning (MADRL) algorithms.

Sketching an open invisible space with multiagent reinforcement learning

Autonomous Aeroamphibious Invisibility Cloak with Stochastic-Evolution Learning

Dynamic metasurface control using Deep Reinforcement Learning

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

METRA: Scalable Unsupervised RL with Metric-Aware Abstraction

Decentralized multi-agent reinforcement learning algorithm using a cluster-synchronized laser network

Learning Curricula in Open-Ended Worlds

Three‐Dimensional Intelligent Cloaked Vehicle Equipped with Thousand‐Level Reconfigurable Full‐Polarization Metasurfaces

Multi-UAV Pursuit-Evasion with Online Planning in Unknown Environments by Deep Reinforcement Learning

Human-Aware Dynamic Hierarchical Network Control for Distributed Metaverse Services

Decision-Making for Satellite Anti-Interception Missions Leveraging Multi-Agent Reinforcement Learning

Subspace-Aware Exploration for Sparse-Reward Multi-Agent Tasks.

3D Intelligent Cloaked Vehicle Equipped with Thousand-Level Reconfigurable Full-Polarization Metasurfaces.

Meta Reinforcement Learning Based Sensor Scanning in 3D Uncertain Environments for Heterogeneous Multi-Robot Systems

Environment as Policy: Learning to Race in Unseen Tracks

MetaUrban: An Embodied AI Simulation Platform for Urban Micromobility

A Multi-agent Reinforcement Learning Method for Swarm Robots in Space Collaborative Exploration

Emergence of Collective Open-Ended Exploration from Decentralized Meta-Reinforcement Learning

Electromagnetic Illusion in Smart Environments

Learning Efficient Multi-Agent Cooperative Visual Exploration