TrinitySLAM: On-board Real-time Event-image Fusion SLAM System for Drones

Xinjun Cai,Jingao Xu,Kuntian Deng,Hongbo Lan,Yue Wu,Xiangwen ZhuGe,Zheng Yang

DOI: https://doi.org/10.1145/3696420

2024-01-01

ACM Transactions on Sensor Networks

Abstract:Drones have witnessed extensive popularity among diverse smart applications, and visual SLAM technology is commonly used to estimate 6-DoF pose for the drone flight control system. However, traditional image-based SLAM cannot ensure the flight safety of drones, especially in challenging environments such as high-speed flight and high dynamic range scenarios. Event camera, a new vision sensor, holds the potential to enable drones to overcome the above challenging scenarios if fused into the image-based SLAM. Unfortunately, the computational demands of event-image fusion SLAM have grown manifold compared to image-based SLAM. Existing research on visual SLAM acceleration cannot achieve real-time operation of event-image fusion SLAM on on-board computing platforms for drones. To fill this gap, we present TrinitySLAM, a high accuracy, real-time, low energy consumption event-image fusion SLAM acceleration framework utilizing Xilinx Zynq, an on-board heterogeneous computing platform. The key innovations of TrinitySLAM include a fine-grained computation allocation strategy, several novel hardware-software co-acceleration designs, and an efficient data exchange mechanism. We fully implement TrinitySLAM on the latest Zynq UltraScale+ platform and evaluate its performance under one self-made drone dataset and four official datasets covering various scenarios. Comprehensive experiments show TrinitySLAM improves the pose estimation accuracy by 28% with half end-to-end latency and 1.2 × energy consumption reduction, compared to the most comparable SOTA heterogeneous computing platform acceleration baseline.