Abstract:To promote better performance-bandwidth trade-off for multi-agent perception, we propose a novel distilled collaboration graph (DiscoGraph) to model trainable, pose-aware, and adaptive collaboration among agents. Our key novelties lie in two aspects. First, we propose a teacher-student framework to train DiscoGraph via knowledge distillation. The teacher model employs an early collaboration with holistic-view inputs; the student model is based on intermediate collaboration with single-view inputs. Our framework trains DiscoGraph by constraining post-collaboration feature maps in the student model to match the correspondences in the teacher model. Second, we propose a matrix-valued edge weight in DiscoGraph. In such a matrix, each element reflects the inter-agent attention at a specific spatial region, allowing an agent to adaptively highlight the informative regions. During inference, we only need to use the student model named as the distilled collaboration network (DiscoNet). Attributed to the teacher-student framework, multiple agents with the shared DiscoNet could collaboratively approach the performance of a hypothetical teacher model with a holistic view. Our approach is validated on V2X-Sim 1.0, a large-scale multi-agent perception dataset that we synthesized using CARLA and SUMO co-simulation. Our quantitative and qualitative experiments in multi-agent 3D object detection show that DiscoNet could not only achieve a better performance-bandwidth trade-off than the state-of-the-art collaborative perception methods, but also bring more straightforward design rationale. Our code is available on <a class="link-external link-https" href="https://github.com/ai4ce/DiscoNet" rel="external noopener nofollow">this https URL</a>.

Gaussian process decentralized data fusion meets transfer learning in large-scale distributed cooperative perception

Collective Online Learning of Gaussian Processes in Massive Multi-Agent Systems

Multiagent Information Fusion and Cooperative Control in Target Search

Fast Transfer Gaussian Process Regression with Large-Scale Sources

An Adaptive Fusion Strategy for Distributed Information Estimation Over Cooperative Multi-Agent Networks.

Scalable, Federated Gaussian Process Training for Decentralized Multi-Agent Systems

A Collaborative Perception Network Based on Dynamic Multi-scale Fusion

CoSense3D: an Agent-based Efficient Learning Framework for Collective Perception

Learning Distilled Collaboration Graph for Multi-Agent Perception

Distributed NeRF Learning for Collaborative Multi-Robot Perception

Multi-Agent Visualization for Explaining Federated Learning.

Breaking Data Silos: Cross-Domain Learning for Multi-Agent Perception from Independent Private Sources

Toward Cooperative Multi-Agent Video Streaming Perception

Multi-Agent Fusion With Different Limited Fields-of-View

On the Federated Learning Framework for Cooperative Perception

Hierarchical Perception-Improving for Decentralized Multi-Robot Motion Planning in Complex Scenarios

Efficient Distributed Transfer Learning for Large‐Scale Gaussian Graphic Models

Exploiting Structure for Optimal Multi-Agent Bayesian Decentralized Estimation

Towards Full-scene Domain Generalization in Multi-agent Collaborative Bird's Eye View Segmentation for Connected and Autonomous Driving

Distributed Online Sparse Gaussian Process Regression for Multi-Agent Coverage Control

Distributed Fusion of Highly Maneuvering Multi-Target under Limited Field of View Sensors