Abstract:Collaborative trajectory prediction can comprehensively forecast the future motion of objects through multi-view complementary information. However, it encounters two main challenges in multi-drone collaboration settings. The expansive aerial observations make it difficult to generate precise Bird's Eye View (BEV) representations. Besides, excessive interactions can not meet real-time prediction requirements within the constrained drone-based communication bandwidth. To address these problems, we propose a novel framework named "Drones Help Drones" (DHD). Firstly, we incorporate the ground priors provided by the drone's inclined observation to estimate the distance between objects and drones, leading to more precise BEV generation. Secondly, we design a selective mechanism based on the local feature discrepancy to prioritize the critical information contributing to prediction tasks during inter-drone interactions. Additionally, we create the first dataset for multi-drone collaborative prediction, named "Air-Co-Pred", and conduct quantitative and qualitative experiments to validate the effectiveness of our DHD framework.The results demonstrate that compared to state-of-the-art approaches, DHD reduces position deviation in BEV representations by over 20% and requires only a quarter of the transmission ratio for interactions while achieving comparable prediction performance. Moreover, DHD also shows promising generalization to the collaborative 3D object detection in CoPerception-UAVs.

What problem does this paper attempt to address?

The paper proposes a new framework called "Drones Help Drones" (DHD) to address the problem of collaborative trajectory prediction for multiple drones. In collaborative drone operations, there are challenges in generating and exchanging precise Bird's Eye View (BEV) representations in real-time. The DHD framework improves the accuracy of BEV generation by utilizing ground priors obtained from drones' tilted observations to estimate the distance between objects and drones. Additionally, it designs a selective mechanism based on local feature differences to prioritize processing key information that contributes to the prediction task, thereby reducing communication bandwidth requirements during interactions. The main problems mentioned in the paper include: 1. The vast aerial observations make it difficult to generate precise BEV representations. 2. Excessive interactions cannot meet the communication constraints for real-time prediction among drones. The DHD framework addresses these problems in the following ways: - Ground prior-guided BEV generation module: Utilizing drones' tilted perspective to estimate the distance between objects and drones improves the accuracy of BEV representations. - Sliding window sparse interaction module: Dynamically evaluating the amount of information based on feature differences and prioritizing key regions for interactions to enhance prediction accuracy. The paper also introduces the first multi-drone collaborative prediction dataset, "Air-Co-Pred," and conducts quantitative and qualitative experiments to demonstrate the effectiveness of the DHD framework. Compared to existing methods, DHD reduces the positional deviation in BEV representations by over 20% and decreases the proportion of interaction transmissions by 75%, while maintaining comparable prediction performance. Furthermore, DHD shows potential in collaborative 3D object detection.

Drones Help Drones: A Collaborative Framework for Multi-Drone Object Trajectory Prediction and Beyond

Collaborative Recognition of Feasible Region with Aerial and Ground Robots Through DPCN

UVCPNet: A UAV-Vehicle Collaborative Perception Network for 3D Object Detection

Multi-Drone Collaborative Trajectory Optimization for Large-Scale Aerial 3D Scanning

Control-Aware Trajectory Predictions for Communication-Efficient Drone Swarm Coordination in Cluttered Environments

UCDNet: Multi-UAV Collaborative 3D Object Detection Network by Reliable Feature Mapping

Multi-Drone-Based Single Object Tracking With Agent Sharing Network

Environmental-Adaptive Trajectory Prediction of Multi-Object for UAV Networks with RGBD Camera

Aerial Monocular 3D Object Detection

Based on improved joint detection and tracking of UAV for multi-target detection of livestock

Target Trajectory Prediction-Based UAV Swarm Cooperative for Bird-Driving Strategy at Airport

Robust Trajectory Planning for Spatial-Temporal Multi-Drone Coordination in Large Scenes

DroneMOT: Drone-based Multi-Object Tracking Considering Detection Difficulties and Simultaneous Moving of Drones and Objects

CoDRMA: Collaborative Depth Refinement Via Dual-Mask and Dual-Attention for Bird’s Eye View Collaborative 3D Object Detection

HOLA-Drone: Hypergraphic Open-ended Learning for Zero-Shot Multi-Drone Cooperative Pursuit

3D Trajectory Reconstruction of Drones using a Single Camera

Virtual Reality Aided High-Quality 3D Reconstruction by Remote Drones

Towards Aerial Collaborative Stereo: Real-Time Cross-Camera Feature Association and Relative Pose Estimation for UAVs

Aerial Vision-and-Dialog Navigation

BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Driving

TAFormer: A Unified Target-Aware Transformer for Video and Motion Joint Prediction in Aerial Scenes