Abstract:Motion estimation is a fundamental step in dynamic medical image processing for the assessment of target organ anatomy and function. However, existing image-based motion estimation methods, which optimize the motion field by evaluating the local image similarity, are prone to produce implausible estimation, especially in the presence of large motion. In addition, the correct anatomical topology is difficult to be preserved as the image global context is not well incorporated into motion estimation. In this study, we provide a novel motion estimation framework of dense-sparse-dense (DSD), which comprises two stages. In the first stage, we process the raw dense image to extract sparse landmarks to represent the target organ's anatomical topology, and discard the redundant information that is unnecessary for motion estimation. For this purpose, we introduce an unsupervised 3-D landmark detection network to extract spatially sparse but representative landmarks for the target organ's motion estimation. In the second stage, we derive the sparse motion displacement from the extracted sparse landmarks of two images of different time points. Then, we present a motion reconstruction network to construct the motion field by projecting the sparse landmarks' displacement back into the dense image domain. Furthermore, we employ the estimated motion field from our two-stage DSD framework as initialization and boost the motion estimation quality in light-weight yet effective iterative optimization. We evaluate our method on two dynamic medical imaging tasks to model cardiac motion and lung respiratory motion, respectively. Our method has produced superior motion estimation accuracy compared to the existing comparative methods. Besides, the extensive experimental results demonstrate that our solution can extract well-representative anatomical landmarks without any requirement of manual annotation. Our code is publicly available online: https://github.com/yyguo-sjtu/DSD-3D-Unsupervised-Landmark-Detection-Based-Motion-Estimation.

Motion energy model for estimating range and motion using spatio-temporal textural images

Qualitative estimations of range and motion using spatio-temporal textural images

Visual Odometry in dynamical scenes

Visual Tracking Via Subspace Motion Model

Motion Detection for Rapidly Moving Cameras in Fully 3D Scenes

KINETIC OCCLUSION MODELING AND MOTION ORIENTATION DETECTION IN EPIPOLAR PLANE IMAGES

Spatial-Related Sensors Matters: 3D Human Motion Reconstruction Assisted with Textual Semantics

Physics-Informed Spatio-Temporal Model for Human Mobility Prediction

Motion Recovery Using the Image Interpolation Algorithm and an RGB-D Camera

A variational approach for ego-motion estimation and segmentation based on 3D TOF camera

Dynamic Texture Recognition by Spatio-Temporal Multiresolution Histograms

Optical Flow Estimation Based on Texture Constraints and Parametric Motion Model

Feature Regions Segmentation Based RGB-D Visual Odometry in Dynamic Environment

Three-dimensional motion estimation by straight-line and endpoint optical flow

Human Action Recognition Using Multi-Velocity STIPs and Motion Energy Orientation Histogram.

Unsupervised Landmark Detection-Based Spatiotemporal Motion Estimation for 4-D Dynamic Medical Images

Estimating Structural Motions in Extreme Environmental Conditions——a Dynamic Correlation Filter Based Computer Vision Approach

Describing Local Reference Frames for 3-D Motion Trajectory Recognition

Contextual Motion Field-Based Distance for Video Analysis

Direct Method for Motion Estimation and Structure Reconstruction Based on Optical Flow

Unsupervised Landmark Detection Based Spatiotemporal Motion Estimation for 4D Dynamic Medical Images