Abstract:Due to the sudden movement during the camera shoot, the videos retrieved from the hand-held mobile devices often suffer from undesired frame jitters, leading to the loss of video quality. In this paper, we present a video stabilization solution in mobile devices via inertial-visual state tracking. Specifically, during the video shoot, we use the gyroscope to estimate the rotation of camera, and use the structure-from-motion among the image frames to estimate the translation of camera. We build a camera projection model by considering the rotation and translation of the camera, and the camera motion model to depict the relationship between the inertial-visual state and the camera's 3D motion. By fusing the inertial measurement (IMU)-based method and the computer vision (CV)-based method, our solution is robust to the fast movement and violent jitters, moreover, it greatly reduces the computation overhead in video stabilization. In comparison to the IMU-based solution, our solution can estimate the translation in a more accurate manner, since we use the feature point pairs in adjacent image frames, rather than the error-prone accelerometers, to estimate the translation. In comparison to the CV-based solution, our solution can estimate the translation with less number of feature point pairs, since the number of undetermined degrees of freedom in the 3D motion directly reduces from 6 to 3. We implemented a prototype system on smart glasses and smart phones, and evaluated the performance under real scenarios, i.e., the human subjects used mobile devices to shoot videos while they were walking, climbing or riding. The experiment results show that our solution achieves 32 percent better performance than the state-of-art solutions in regard to video stabilization. Moreover, the average processing time latency is 32.6ms, which is lower than the conventional inter-frame time interval, i.e., 33ms, and thus meets the real-ti-e requirement for online processing.

The video stabilization algorithm based on MEMS gyroscope

Image Stabilization Using Motion Estimation and Micro-mechanical Compensation

Optical Image Mosaic Methods Based on MEMS Gyroscope

Real-time Digital Image Stabilization Method Based on Projection Algorithm and Block Matching

Image Stabilization with Support Vector Machine

A Fast Approach Against Large Foreground Motion in Real-time Image Stabilization

Video stabilisation based on modelling of motion imaging.

Performance test of optical and electronic image stabilizer for digital imaging system

Video stabilization algorithm based on human visual characteristics

Real-time Video Stabilization Based on Minimal Spanning Tree and Modified Kalman Filter

Real-Time Digital Image Stabilization System Based on Gray Projection Algorithm

Study on Performance Evaluation and Frequency Characteristic for Real-Time Image Stabilization System

Time-delay compensation for stabilization imaging system

Camera Stabilization Based on 2.5D Motion Estimation and Inertial Motion Filtering

A Fast Digital Image Stabilization Algorithm Based on Multiple-candidate Three-step Search

A Comprehensive Motion Estimation Technique for the Improvement of EIS Methods Based on the SURF Algorithm and Kalman Filter

Digital Video Sequence Stabilization Based on 2.5d Motion Estimation and Inertial Motion Filtering

Video Stabilization for Camera Shoot in Mobile Devices via Inertial-Visual State Tracking

Real-time Electronic Image Stabilization for Spherical Robot Based on Inertial Measurement Unit and Gray Projection Method

A Circular-Projection based digital image stabilization algorithm and its implementation for rotating image sequences

Global Motion Estimation for Video Stabilization