Putri A. van der Linden,David W. Romero,Erik J. Bekkers
Abstract:Neural operations that rely on neighborhood information are much more expensive when deployed on point clouds than on grid data due to the irregular distances between points in a point cloud. In a grid, on the other hand, we can compute the kernel only once and reuse it for all query positions. As a result, operations that rely on neighborhood information scale much worse for point clouds than for grid data, specially for large inputs and large neighborhoods.
In this work, we address the scalability issue of point cloud methods by tackling its root cause: the irregularity of the data. We propose learnable gridification as the first step in a point cloud processing pipeline to transform the point cloud into a compact, regular grid. Thanks to gridification, subsequent layers can use operations defined on regular grids, e.g., Conv3D, which scale much better than native point cloud methods. We then extend gridification to point cloud to point cloud tasks, e.g., segmentation, by adding a learnable de-gridification step at the end of the point cloud processing pipeline to map the compact, regular grid back to its original point cloud form. Through theoretical and empirical analysis, we show that gridified networks scale better in terms of memory and time than networks directly applied on raw point cloud data, while being able to achieve competitive results. Our code is publicly available at <a class="link-external link-https" href="https://github.com/computri/gridifier" rel="external noopener nofollow">this https URL</a>.
What problem does this paper attempt to address?
### Problems the paper attempts to solve
The paper "Learned Gridification for Efficient Point Cloud Processing" aims to solve the scalability problem in point cloud processing. Specifically, neural operations are more costly when processing point cloud data than when processing grid data, mainly because the distances between points in the point cloud are irregular. This irregularity leads to a much higher computational complexity for operations based on neighborhood information (such as convolution) on point clouds than on grid data, especially when dealing with large - scale inputs and large neighborhoods.
### Main contributions
1. **Learnable Gridification**:
- A learnable gridification method is proposed to convert point clouds into compact and regular grids. Through this method, subsequent layers can use operations defined on regular grids (such as Conv3D), and the computational complexity of these operations is much lower than that of operations directly performed on point clouds.
- The gridification process is achieved through a convolutional message - passing layer, which establishes connections between point clouds and grids and propagates information.
2. **De - gridification**:
- To handle tasks from point cloud to point cloud (such as segmentation), a learnable de - gridification step is introduced to map the compact and regular grid back to the original point cloud form. This extension enables the gridified network to solve global prediction tasks (such as classification) and dense prediction tasks (such as segmentation and regression).
3. **Performance analysis**:
- Through theoretical and empirical analysis, it is proved that the gridified network is superior to the network directly applied to the original point cloud data in terms of memory and time complexity, and can achieve competitive performance at the same time.
### Method overview
1. **Point cloud and grid representations**:
- **Point cloud**: A point cloud \(P=\{(c_i^P, x_i^P)\}_{i = 1}^{N_P}\) is an unstructured set of coordinate - feature value pairs, scattered in space without a predefined pattern or connectivity.
- **Grid**: A grid \(G=\{(c_i^G, x_i^G)\}_{i = 1}^{N_G}\) can be regarded as a point cloud, in which the coordinate - feature value pairs are regularly arranged to form a lattice. The points in the grid are evenly distributed and aligned along predefined axes, so that the point - pair attributes can be calculated at one time and reused on all query points.
2. **Gridification process**:
- The goal is to map the sparse point cloud \(P\) onto a compact and regular grid \(G\). This process is formalized as an operation on a bipartite graph, which establishes the connection between the point cloud \(P\) and the grid \(G\).
- The gridification process is achieved through a convolutional message - passing layer, which contains a node embedding network \(\phi_{\text{node}}\), a position embedding network \(\phi_{\text{pos}}\) and a message embedding network \(\phi_{\text{msg}}\).
3. **De - gridification process**:
- The de - gridification step maps the grid \(G\) back to the original point cloud form \(P\). This process can be regarded as the inverse process of gridification and is achieved by reversing the connection scheme \(E_{P\rightarrow G}\).
4. **Requirements and properties**:
- To prevent information loss, the gridification process needs to meet multiple requirements, including maintaining the number of points, feature width, and high - frequency representation ability, etc.
- To encourage a compact and expressive grid representation, some additional requirements are also proposed, such as each point cloud point is connected to at least one grid point, and each grid point is connected to at least one point cloud point.
### Experimental results
1. **Random point cloud reconstruction**:
- The expressive ability of the method was evaluated through a simple network (only including gridification and de - gridification steps). The experimental results show that good reconstruction effects can be achieved by increasing the grid resolution or the number of channels. From the perspective of efficiency, it is more advantageous to use a low - resolution but multi - channel representation.
2. **ModelNet40 classification**:
- In