ShapeMaker: Self-Supervised Joint Shape Canonicalization, Segmentation, Retrieval and Deformation
Yan Di,Chenyangguang Zhang,Chaowei Wang,Ruida Zhang,Guangyao Zhai,Yanyan Li,Bowen Fu,Xiangyang Ji,Shan Gao
DOI: https://doi.org/10.1109/cvpr52733.2024.01986
2024-01-01
Computer Vision and Pattern Recognition
Abstract:In this paper, we present ShapeMatcher, a unified self-supervised learningframework for joint shape canonicalization, segmentation, retrieval anddeformation. Given a partially-observed object in an arbitrary pose, we firstcanonicalize the object by extracting point-wise affine-invariant features,disentangling inherent structure of the object with its pose and size. Theselearned features are then leveraged to predict semantically consistent partsegmentation and corresponding part centers. Next, our lightweight retrievalmodule aggregates the features within each part as its retrieval token andcompare all the tokens with source shapes from a pre-established database toidentify the most geometrically similar shape. Finally, we deform the retrievedshape in the deformation module to tightly fit the input object by harnessingpart center guided neural cage deformation. The key insight of ShapeMaker isthe simultaneous training of the four highly-associated processes:canonicalization, segmentation, retrieval, and deformation, leveragingcross-task consistency losses for mutual supervision. Extensive experiments onsynthetic datasets PartNet, ComplementMe, and real-world dataset Scan2CADdemonstrate that ShapeMaker surpasses competitors by a large margin.