Abstract:Due to the sudden movement during the camera shoot, the videos retrieved from the hand-held mobile devices often suffer from undesired frame jitters, leading to the loss of video quality. In this paper, we present a video stabilization solution in mobile devices via inertial-visual state tracking. Specifically, during the video shoot, we use the gyroscope to estimate the rotation of camera, and use the structure-from-motion among the image frames to estimate the translation of camera. We build a camera projection model by considering the rotation and translation of the camera, and the camera motion model to depict the relationship between the inertial-visual state and the camera's 3D motion. By fusing the inertial measurement (IMU)-based method and the computer vision (CV)-based method, our solution is robust to the fast movement and violent jitters, moreover, it greatly reduces the computation overhead in video stabilization. In comparison to the IMU-based solution, our solution can estimate the translation in a more accurate manner, since we use the feature point pairs in adjacent image frames, rather than the error-prone accelerometers, to estimate the translation. In comparison to the CV-based solution, our solution can estimate the translation with less number of feature point pairs, since the number of undetermined degrees of freedom in the 3D motion directly reduces from 6 to 3. We implemented a prototype system on smart glasses and smart phones, and evaluated the performance under real scenarios, i.e., the human subjects used mobile devices to shoot videos while they were walking, climbing or riding. The experiment results show that our solution achieves 32 percent better performance than the state-of-art solutions in regard to video stabilization. Moreover, the average processing time latency is 32.6ms, which is lower than the conventional inter-frame time interval, i.e., 33ms, and thus meets the real-ti-e requirement for online processing.

Vehicle Video Stabilization Combined with Vestibular Reflex Effect

Performance test of optical and electronic image stabilizer for digital imaging system

A Fast Approach Against Large Foreground Motion in Real-time Image Stabilization

Image Stabilization Using Motion Estimation and Micro-mechanical Compensation

Video Image Stabilization Algorithm for Remote Control Driving

Interline image revision method for electrical image stabilization

Video stabilisation based on modelling of motion imaging.

Real-time Electronic Image Stabilization for Spherical Robot Based on Inertial Measurement Unit and Gray Projection Method

Imaging Simulation and Restoration for Mobile-based Long-focus Camera with Optical Stabilization

Harris Corner Detection-based Stereo Visual Odometry

Robust and Efficient Vehicles Motion Estimation with Low-Cost Multi-Camera and Odometer-Gyroscope

Time-delay compensation for stabilization imaging system

Video stabilization algorithm based on human visual characteristics

Video Stabilization for Camera Shoot in Mobile Devices via Inertial-Visual State Tracking

A stable vision system for moving vehicles

Camera Stabilization Based on 2.5D Motion Estimation and Inertial Motion Filtering

Image stabilization technology based on inertial sensors and piezoelectric fast pendulum mirrors

On the Benefits of Visual Stabilization for Frame- and Event-based Perception

An Approach of Electronic Image Stabilization Based on the Representative Point Matching

Tightly Coupled Visual-Inertial Fusion with Image Enhancement for Robust Positioning

A Comprehensive Motion Estimation Technique for the Improvement of EIS Methods Based on the SURF Algorithm and Kalman Filter