Abstract:IMU based inertial tracking plays an indispensable role in many mobility centric tasks, such as robotic control, indoor navigation and virtual reality gaming. Despite its mature application in rigid machine mobility (e.g., robot and aircraft), tracking human users via mobile devices remains a fundamental challenge due to the intractable gait/posture patterns. Recent data-driven models have tackled sensor drifting, one key issue that plagues inertial tracking. However, these systems still assume the devices are held or attached to the user body with a relatively fixed posture. In practice, natural body activities may rotate/translate the device which may be mistaken as whole body movement. Such motion artifacts remain as the dominating factor that fails existing inertial tracing systems in practical uncontrolled settings. Inspired by the observation that human heads induces far less intensive movement relative to the body during walking, compared to other parts, we propose a novel multi-stage sensor fusion pipeline called DeepIT, which realizes inertial tracking by synthesizing the IMU measurements from a smartphone and an associated earbud. DeepIT introduces a data-driven reliability aware attention model, which assesses the reliability of each IMU and opportunistically synthesizes their data to mitigate the impacts of motion noise. Furthermore, DeepIT uses a reliability aware magnetometer compensation scheme to combat the angular drifting problem caused by unrestricted motion artifacts. We validate DeepIT on the first large-scale inertial navigation dataset involving both smartphone and earbud IMUs. The evaluation results show that DeepIT achieves multiple folds of accuracy improvement on the challenging uncontrolled natural walking scenarios, compared with state-of-the-art closed-form and data-driven models.

A Rotation Based Method for Detecting On-Body Positions of Mobile Devices

On-Body Position Recognition Method for Mobile Devices Using Rotation Patterns

Smartphone-based Activity Recognition Independent of Device Orientation and Placement.

An Interpretable Orientation and Placement Invariant Approach for Smartphone Based Activity Recognition.

Activity Recognition on an Accelerometer Embedded Mobile Phone with Varying Positions and Orientations

Posture Tracking Meets Fitness Coaching: A Two-Phase Optimization Approach with Wearable Devices

Human Body Recognition And Positioning With Multiple Cameras Based On "Vibration Signals" From Skin Surfaces

A Scalable and Wearable Self-Sensing IMU Sensor Network for Personalized Human Motion and Deformation Capture

Identifying typical physical activity on smartphone with varying positions and orientations

A Human Movement Detection Algorithm on Mobile Phone Platform

Combining Accelerometer and Gyroscope Data in Smartphone-Based Activity Recognition using Movelets

IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds

A Portable System for ToF Camera Based Human Body Detection and Pose Estimation

Robust Inertial Motion Tracking Through Deep Sensor Fusion Across Smart Earbuds and Smartphone.

Contactless Measurement of Pose in the Earth Frame Using A Mobile Phone

On Motion-Sensor Behavior Analysis for Human-Activity Recognition Via Smartphones

Inertial Motion Tracking on Mobile and Wearable Devices: Recent Advancements and Challenges

A Generic Framework for Human Motion Recognition Based on Smartphones

Position-Aware Indoor Human Activity Recognition Using Multisensors Embedded in Smartphones

Research on the Human Motion Recognition Method Based on Wearable

It's the Human that Matters: Accurate User Orientation Estimation for Mobile Computing Applications