Performance Assessment of Lidar Odometry Frameworks: A Case Study at the Australian Botanic Garden Mount Annan

Mohamed Mourad Ouazghire,Julie Stephany Berrio,Mao Shan,Stewart Worrall

2024-11-26

Abstract:Autonomous vehicles are being tested in diverse environments worldwide. However, a notable gap exists in evaluating datasets representing natural, unstructured environments such as forests or gardens. To address this, we present a study on localisation at the Australian Botanic Garden Mount Annan. This area encompasses open grassy areas, paved pathways, and densely vegetated sections with trees and other objects. The dataset was recorded using a 128-beam LiDAR sensor and GPS and IMU readings to track the ego-vehicle. This paper evaluates the performance of two state-of-the-art LiDARinertial odometry frameworks, COIN-LIO and LIO-SAM, on this dataset. We analyse trajectory estimates in both horizontal and vertical dimensions and assess relative translation and yaw errors over varying distances. Our findings reveal that while both frameworks perform adequately in the vertical plane, COINLIO demonstrates superior accuracy in the horizontal plane, particularly over extended trajectories. In contrast, LIO-SAM shows increased drift and yaw errors over longer distances.

Robotics

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to evaluate and compare the performance of two state - of - the - art LiDAR - inertial odometry frameworks, COIN - LIO and LIO - SAM, in the complex environment of the Australian Botanic Garden Mount Annan. Specifically, this research attempts to solve the following problems:

1. **Insufficient dataset evaluation in natural, unstructured environments**:
- Most of the current tests for autonomous vehicles are concentrated in urban or artificial environments, while the dataset evaluation in natural, unstructured environments (such as forests or gardens) is relatively scarce. This paper fills this gap by conducting experiments in the Australian Botanic Garden Mount Annan.

2. **Accuracy of long - distance trajectory estimation**:
- The research focuses on the performance of the two frameworks when dealing with long - distance trajectories, especially their performance in terms of cumulative drift. By analyzing the trajectory estimation in the horizontal and vertical dimensions, their relative translation and yaw errors are evaluated.

3. **Effectiveness of the loop closure mechanism**:
- In a complex environment, loop closure is crucial for maintaining global consistency. This paper explores the effectiveness of the two frameworks in loop closure detection and correction, especially in an environment with complex terrain and rich vegetation.

4. **Adaptability and robustness of different frameworks**:
- COIN - LIO and LIO - SAM show different advantages in different scenarios. COIN - LIO is more suitable for geometrically - degraded environments (such as tunnels and flat fields), while LIO - SAM performs well in mixed indoor - outdoor and urban environments. This paper demonstrates these differences through experiments and explores the reasons behind them.

### Main conclusions

Through a detailed evaluation of the dataset in the Australian Botanic Garden Mount Annan, the research finds that:
- **COIN - LIO** shows higher accuracy in the horizontal (x - y) plane, especially in long - distance trajectories, can better maintain trajectory fidelity, and has less drift.
- **LIO - SAM** performs well on the vertical (z) axis, but shows significant drift in the horizontal plane, especially in the last quarter of the trajectory, which indicates that its loop closure mechanism has challenges in a complex environment.

In conclusion, this paper provides a valuable reference for future research by evaluating two advanced LiDAR - inertial odometry frameworks in a complex natural environment and points out the direction for further improvement.