Jigsaw++: Imagining Complete Shape Priors for Object Reassembly

Jiaxin Lu,Gang Hua,Qixing Huang

2024-10-16

Abstract:The automatic assembly problem has attracted increasing interest due to its complex challenges that involve 3D representation. This paper introduces Jigsaw++, a novel generative method designed to tackle the multifaceted challenges of reconstruction for the reassembly problem. Existing approach focusing primarily on piecewise information for both part and fracture assembly, often overlooking the integration of complete object prior. Jigsaw++ distinguishes itself by learning a category-agnostic shape prior of complete objects. It employs the proposed "retargeting" strategy that effectively leverages the output of any existing assembly method to generate complete shape reconstructions. This capability allows it to function orthogonally to the current methods. Through extensive evaluations on Breaking Bad dataset and PartNet, Jigsaw++ has demonstrated its effectiveness, reducing reconstruction errors and enhancing the precision of shape reconstruction, which sets a new direction for future reassembly model developments.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper attempts to solve the key problem in object reassembly, that is, how to use the complete shape prior to reconstruct partially assembled or inaccurately assembled objects. Existing methods mainly focus on local information and patch matching, ignoring the complete shape information of the whole object. This leads to the difficulty in achieving high - quality shape reconstruction in practical applications, especially when only a limited number of fragments are available.

### Specific Problem Description

1. **Limitations of Existing Methods**:
- Existing methods mainly rely on patch information for matching and lack an understanding of the overall shape prior of the complete object.
- In real - world scenarios, usually only some fragments can be obtained, and complete shape information cannot be provided.
- The reconstruction process highly depends on the prior knowledge of the complete object, and existing methods are deficient in this regard.

2. **Challenges in Application Scenarios**:
- The object reassembly problem is widely used in digital archaeology, robot furniture assembly, and the medical field (such as fracture repair).
- These application scenarios require the ability to recover the complete object structure from partial or inaccurate inputs.

3. **Specific Problem Classification**:
- **Part Assembly**: Deals with semantically meaningful parts.
- **Fractured Assembly**: Deals with fragments broken by strong external forces.

### Solutions Proposed in the Paper

To solve the above problems, the paper introduces Jigsaw++, a novel generative method, which aims to meet the complex reassembly challenges by learning the complete shape prior. The main contributions of Jigsaw++ include:

- **Learning the Complete Shape Prior**: Jigsaw++ learns the complete object - shape prior independent of categories through a generative model, solving the problem that existing methods only rely on patch information.
- **Retargeting Strategy**: This strategy effectively utilizes the output of existing assembly methods to generate complete shape reconstruction. This method can be orthogonally combined with existing methods to improve reconstruction accuracy.
- **Point - Cloud Generation Module**: An object - level point - cloud generation module that can adapt to any size of input and output number of points has been developed, solving the problems of data sparsity and category - specificity in point - cloud generation.
- **Flexibility and Efficiency**: By adopting the rectified flow, the tuning cost is reduced and the generation efficiency is improved.

### Summary

Jigsaw++ aims to solve the problem of the lack of complete shape prior in existing object reassembly methods. By learning the complete shape prior and introducing the "retargeting" strategy, it significantly improves the reconstruction quality of partially assembled or inaccurately assembled objects.