Abstract:Due to the sudden movement during the camera shoot, the videos retrieved from the hand-held mobile devices often suffer from undesired frame jitters, leading to the loss of video quality. In this paper, we present a video stabilization solution in mobile devices via inertial-visual state tracking. Specifically, during the video shoot, we use the gyroscope to estimate the rotation of camera, and use the structure-from-motion among the image frames to estimate the translation of camera. We build a camera projection model by considering the rotation and translation of the camera, and the camera motion model to depict the relationship between the inertial-visual state and the camera's 3D motion. By fusing the inertial measurement (IMU)-based method and the computer vision (CV)-based method, our solution is robust to the fast movement and violent jitters, moreover, it greatly reduces the computation overhead in video stabilization. In comparison to the IMU-based solution, our solution can estimate the translation in a more accurate manner, since we use the feature point pairs in adjacent image frames, rather than the error-prone accelerometers, to estimate the translation. In comparison to the CV-based solution, our solution can estimate the translation with less number of feature point pairs, since the number of undetermined degrees of freedom in the 3D motion directly reduces from 6 to 3. We implemented a prototype system on smart glasses and smart phones, and evaluated the performance under real scenarios, i.e., the human subjects used mobile devices to shoot videos while they were walking, climbing or riding. The experiment results show that our solution achieves 32 percent better performance than the state-of-art solutions in regard to video stabilization. Moreover, the average processing time latency is 32.6ms, which is lower than the conventional inter-frame time interval, i.e., 33ms, and thus meets the real-ti-e requirement for online processing.

The Analysis of Image Stabilization Technology Based on Small-UAV Airborne Video

Electronic Image Stabilization Algorithms Based on Flight Characteristics of the Small UAV

Image Stabilization Using Motion Estimation and Micro-mechanical Compensation

Evaluation and Selection of Video Stabilization Techniques for UAV-Based Active Infrared Thermography Application

The Adaptive Compensation Algorithm for Small UAV Image Stabilization

A Method of Intentional Movement Estimation of Oblique Small-Uav Videos Stabilized Based on Homography Model

Real-Time Digital Image Stabilization System Based on Gray Projection Algorithm

Mini-Uav Borne Hyperspectral Remote Sensing: A Review

An intended motion estimation method based on unmanned aerial vehicle aviation video image

Intended Motion Estimation Using Fuzzy Kalman Filtering for UAV Image Stabilization with Large Drifting

Video Stabilization and Target Tracking for Airship Monitoring System

Application of Video Capture in UAV Image Processing Based on MPEG2

Video Stabilization via Piecewise Linear L1 Optimization for Airship Earth Observation.

Overview of imaging stabilization

An Overview of Video Stabilization

Image Motion Compensation Control Technology of Unmanned Aerial Vehicle (UAV) Airborne Photoelectric Reconnaissance Platform

Research on Multi-depth-of-field Electronic Image Stabilization Algorithm for In-Vehicle Videos

Video-Based Vibration Measurement Using an Unmanned Aerial Vehicle: An Anti-Disturbance Algorithm for the Shaking of Airborne Cameras

REAL-TIME VIBRATION REDUCTION IN UAV'S IMAGE SENSORS USING EFFICIENT HOUGH TRANSFORM

Real-Time Visual Tracking of Moving Targets Using a Low-Cost Unmanned Aerial Vehicle with a 3-Axis Stabilized Gimbal System

Video Stabilization for Camera Shoot in Mobile Devices via Inertial-Visual State Tracking