Lost in Tracking Translation: A Comprehensive Analysis of Visual SLAM in Human-Centered XR and IoT Ecosystems

Yasra Chandio,Khotso Selialia,Joseph DeGol,Luis Garcia,Fatima M. Anwar

2024-11-12

Abstract:Advancements in tracking algorithms have empowered nascent applications across various domains, from steering autonomous vehicles to guiding robots to enhancing augmented reality experiences for users. However, these algorithms are application-specific and do not work across applications with different types of motion; even a tracking algorithm designed for a given application does not work in scenarios deviating from highly standard conditions. For example, a tracking algorithm designed for robot navigation inside a building will not work for tracking the same robot in an outdoor environment. To demonstrate this problem, we evaluate the performance of the state-of-the-art tracking methods across various applications and scenarios. To inform our analysis, we first categorize algorithmic, environmental, and locomotion-related challenges faced by tracking algorithms. We quantitatively evaluate the performance using multiple tracking algorithms and representative datasets for a wide range of Internet of Things (IoT) and Extended Reality (XR) applications, including autonomous vehicles, drones, and humans. Our analysis shows that no tracking algorithm works across different applications and scenarios within applications. Ultimately, using the insights generated from our analysis, we discuss multiple approaches to improving the tracking performance using input data characterization, leveraging intermediate information, and output evaluation.

Robotics,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to solve the problem of poor performance of visual SLAM (Simultaneous Localization and Mapping) algorithms in different application scenarios and across applications within a scenario. Specifically:

1. **Application - specificity**: Existing tracking algorithms are usually designed for specific applications and cannot adapt well to other types of applications or scenarios that deviate from standard conditions. For example, a tracking algorithm designed for indoor robot navigation cannot work effectively in an outdoor environment.

2. **Environmental complexity and diversity**: Challenges faced by tracking systems include environmental complexity (such as the number of objects, lighting conditions, occlusions, weather, reflective surfaces, and scene changes), different motion requirements (such as differences in the motion patterns of vehicles, robots, and humans), and sensor limitations (such as time drift of IMU sensors, sensitivity of depth cameras to lighting and reflective surfaces, and sensitivity of RGB cameras to lighting changes).

3. **Special challenges in XR (Extended Reality)**:
- **Unpredictable motion**: Human actions are often sudden and irregular, such as sharp turns or rapid gestures, which can affect tracking accuracy.
- **Inter - individual differences**: Different users interact with XR systems in different ways, depending on their familiarity with the technology, physical abilities, and personal preferences.
- **Environmental changes**: XR systems need to operate in a variety of environments, from bright indoor spaces to outdoor areas, requiring the system to be continuously recalibrated.
- **Cognitive load and real - time processing**: Complex interactions increase the user's cognitive load, requiring the tracking system to be intuitive and minimize interference.
- **Body part occlusion**: The user's arms or other body parts may occlude the sensors and interrupt tracking.
- **Physical safety**: XR systems must maintain situation awareness to prevent collisions or unsafe interactions with real - world objects.

4. **Limitations of existing methods**: Traditional SLAM methods rely on carefully designed features and manually designed system components and lack robustness; end - to - end learning methods, although more adaptable, may perform poorly when encountering unseen situations or environments not in the training data; hybrid methods combine the advantages of both but usually sacrifice optimal performance.

### Research contributions

To address the above challenges, the paper makes the following contributions:

1. **Challenge classification**: A classification system is constructed, which describes in detail the algorithm - related, environment - related, and motion - related challenges faced by tracking systems and their impact on XR applications.

2. **Performance evaluation**: The performance of the state - of - the - art tracking algorithms is quantitatively evaluated through three different datasets (representing different application fields, environments, motions, and tracking targets).

3. **Dataset feature analysis**: Based on the quantitative evaluation of traditional, end - to - end learning, and hybrid tracking systems, a preliminary dataset feature analysis is carried out, emphasizing the importance of understanding dataset features for tracking performance and proposing adaptive solutions for specific environments and use cases.

4. **Improvement strategies**: Three strategies for enhancing the performance of SLAM systems in dynamic environments are proposed: input feature configuration, intermediate information utilization, and output evaluation.

Through these studies, the paper not only reveals the unique challenges of tracking systems in different application fields but also provides a solid foundation for developing more robust and versatile tracking solutions, especially in XR applications, ensuring that the tracking system can adapt to the unpredictability of human behavior.