SplatPose+: Real-time Image-Based Pose-Agnostic 3D Anomaly Detection

Yizhe Liu,Yan Song Hu,Yuhao Chen,John Zelek
2024-10-16
Abstract:Image-based Pose-Agnostic 3D Anomaly Detection is an important task that has emerged in industrial quality control. This task seeks to find anomalies from query images of a tested object given a set of reference images of an anomaly-free object. The challenge is that the query views (a.k.a poses) are unknown and can be different from the reference views. Currently, new methods such as OmniposeAD and SplatPose have emerged to bridge the gap by synthesizing pseudo reference images at the query views for pixel-to-pixel comparison. However, none of these methods can infer in real-time, which is critical in industrial quality control for massive production. For this reason, we propose SplatPose+, which employs a hybrid representation consisting of a Structure from Motion (SfM) model for localization and a 3D Gaussian Splatting (3DGS) model for Novel View Synthesis. Although our proposed pipeline requires the computation of an additional SfM model, it offers real-time inference speeds and faster training compared to SplatPose. Quality-wise, we achieved a new SOTA on the Pose-agnostic Anomaly Detection benchmark with the Multi-Pose Anomaly Detection (MAD-SIM) dataset.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to achieve image - based, pose - agnostic 3D anomaly detection in industrial quality control. Specifically, the goal of this task is to detect anomalies from the query images of test objects, given a set of reference images of non - anomalous objects. The challenge lies in the fact that the query view (i.e., pose) is unknown and may be different from the reference view. Current methods such as OmniposeAD and SplatPose can achieve pixel - level comparison by synthesizing pseudo - reference images in the query view, but these methods cannot achieve real - time inference, which is crucial for industrial quality control in large - scale production. Therefore, this paper proposes SplatPose +, a method that combines the Structure from Motion (SfM) model for localization and the 3D Gaussian Splatting (3DGS) model for novel view synthesis to achieve efficient and real - time anomaly detection.