PReP: Efficient context-based shape retrieval for missing parts

Vlassis Fotis,Ioannis Romanelis,Georgios Mylonas,Athanasios Kalogeras,Konstantinos Moustakas

2024-10-18

Abstract:In this paper we study the problem of shape part retrieval in the point cloud domain. Shape retrieval methods in the literature rely on the presence of an existing query object, but what if the part we are looking for is not available? We present Part Retrieval Pipeline (PReP), a pipeline that creatively utilizes metric learning techniques along with a trained classification model to measure the suitability of potential replacement parts from a database, as part of an application scenario targeting circular economy. Through an innovative training procedure with increasing difficulty, it is able to learn to recognize suitable parts relying only on shape context. Thanks to its low parameter size and computational requirements, it can be used to sort through a warehouse of potentially tens of thousand of spare parts in just a few seconds. We also establish an alternative baseline approach to compare against, and extensively document the unique challenges associated with this task, as well as identify the design choices to solve them.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is in the point - cloud domain, when a part of an object is missing, how to find a suitable replacement part based on the existing shape context. Specifically, traditional shape retrieval methods rely on the existence of existing query objects, but if the missing part is unavailable, how to find a suitable substitute relying only on the shape context is a challenge. The author proposes a new method named Part Retrieval Pipeline (PReP) for efficiently retrieving missing parts in a large spare - parts library to support circular economy applications.

### Main Problem Description

1. **Shape Part Retrieval Problem**:
- In the point - cloud domain, when a part of an object is missing, how to find a suitable replacement part.
- Traditional methods rely on the existence of complete query objects, but this paper studies the situation where the missing part is unavailable.

2. **Specific Application Scenarios**:
- Imagine a manufacturing center where users can bring in damaged items and find replacement parts from a large spare - parts warehouse.
- The goal is to promote the circular economy by repairing these items and give a second life to items that are no longer in use.

3. **Technical Difficulties**:
- The missing part cannot be scanned and directly matched, and only the context information of the existing shape can be relied on.
- It is necessary to quickly and accurately find the most suitable replacement part among a large number of spare parts.

### Solution Overview

The author proposes PReP (Part Retrieval Pipeline), which is a pipeline that uses metric learning techniques and a trained classification model to evaluate the applicability of potential replacement parts. The specific steps are as follows:

1. **Encoder Design**:
- Use a deep encoder similar to PointNet to map point - cloud parts to a high - dimensional feature space.
- Ensure that parts with similar geometries are close in the feature space.

2. **Feature Fusion and Scoring**:
- Use a shallow Transformer module to process the feature vectors of multiple parts and generate an overall fitness score.
- When pre - encoded spare parts are introduced, evaluate the entire object and provide a fitness score.

3. **Two - stage Training**:
- First stage: Train the encoder to map parts of the same category to close feature points.
- Second stage: End - to - end train the entire model for classification to ensure that the Transformer can predict the composition of objects of a specific category based on the relationships between parts.

4. **Data Preparation and Pre - processing**:
- Use the DBSCAN algorithm to separate parts in the point cloud to ensure that each part is processed separately.
- Create data sets containing complete objects and part IDs, called Items and Warehouse data sets respectively.

5. **Experimental Verification**:
- Conduct experiments on two benchmark data sets, Shapenet - part and PartNet, to verify the effectiveness of PReP.
- The results show that, despite the differences in the number of points and the mis - clustering problem, PReP can still select meaningful replacement parts.

### Formula Representation

- **Feature Encoding Formula**:
\[
v_i = f(p_i; \theta), \quad p_i \in \mathbb{R}^{N_i\times3}\rightarrow\mathbb{R}^d
\]
where \( f \) is a neural network encoder with parameter \( \theta \), mapping the point cloud \( p_i \) to a \( d \)-dimensional feature vector \( v_i \).

- **Similarity Calculation**:
\[
S = P\cdot P^T, \quad \|p_i\| = 1
\]
where \( S \) is the normalized point - pair similarity matrix.

- **Loss Function**:
\[
L=\frac{1}{N^2}\sum_{i = 0}^{N - 1}\sum_{j = 0}^{N - 1}(S_{ij}-G_{ij})^2
\]
where \( G_{ij} = 1 \) if \( l_i = l_j \), otherwise