Abstract:Recently, leveraging pre-training techniques to enhance point cloud models has become a hot research topic. However, existing approaches typically require full fine-tuning of pre-trained models to achieve satisfied performance on downstream tasks, accompanying storage-intensive and computationally demanding. To address this issue, we propose a novel Parameter-Efficient Fine-Tuning (PEFT) method for point cloud, called PointGST (Point cloud Graph Spectral Tuning). PointGST freezes the pre-trained model and introduces a lightweight, trainable Point Cloud Spectral Adapter (PCSA) to fine-tune parameters in the spectral domain. The core idea is built on two observations: 1) The inner tokens from frozen models might present confusion in the spatial domain; 2) Task-specific intrinsic information is important for transferring the general knowledge to the downstream task. Specifically, PointGST transfers the point tokens from the spatial domain to the spectral domain, effectively de-correlating confusion among tokens via using orthogonal components for separating. Moreover, the generated spectral basis involves intrinsic information about the downstream point clouds, enabling more targeted tuning. As a result, PointGST facilitates the efficient transfer of general knowledge to downstream tasks while significantly reducing training costs. Extensive experiments on challenging point cloud datasets across various tasks demonstrate that PointGST not only outperforms its fully fine-tuning counterpart but also significantly reduces trainable parameters, making it a promising solution for efficient point cloud learning. It improves upon a solid baseline by +2.28%, 1.16%, and 2.78%, resulting in 99.48%, 97.76%, and 96.18% on the ScanObjNN OBJ BG, OBJ OBLY, and PB T50 RS datasets, respectively. This advancement establishes a new state-of-the-art, using only 0.67% of the trainable parameters.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of Parameter - Efficient Fine - Tuning (PEFT) in point cloud learning. Specifically, existing methods usually require fully fine - tuning (FFT) of pre - trained models to achieve satisfactory performance in downstream tasks. However, this method is accompanied by problems of high storage density and high computational cost because a large number of parameters need to be updated. To solve these problems, the authors propose a new parameter - efficient fine - tuning method called PointGST (Point cloud Graph Spectral Tuning). This method improves the existing PEFT methods in the following ways: 1. **Freeze the pre - trained model**: PointGST freezes the parameters of the pre - trained model and only introduces a small number of trainable parameters for fine - tuning. 2. **Introduce spectral - domain fine - tuning**: Different from traditional methods, PointGST converts point clouds from the spatial domain to the spectral domain for fine - tuning, thereby more effectively reducing the confusion between tokens and introducing the intrinsic information of downstream point clouds. 3. **Light - weight adapter**: PointGST introduces a light - weight point cloud spectral adapter (PCSA) for adjusting parameters in the spectral domain. Through these improvements, PointGST not only significantly reduces the training cost but also improves the performance of downstream tasks. Experimental results show that PointGST achieves better performance than fully fine - tuning on multiple challenging point cloud datasets while using only a very small number of trainable parameters. ### Core innovation points - **Spectral - domain fine - tuning**: Utilize the uniqueness of spectral representation to convert point clouds from the spatial domain to the spectral domain, making it easier to decorrelate tokens and introduce intrinsic information. - **Light - weight adapter**: Introduce a light - weight point cloud spectral adapter (PCSA) that can improve performance while maintaining efficiency. - **Multi - scale graph construction**: Construct multi - scale point cloud graphs to capture global and local structural information and further improve the performance of the model. ### Experimental results Experimental results show that PointGST outperforms existing methods on multiple point cloud datasets. For example, it achieves an accuracy of 99.48% on the ScanObjectNN OBJ BG dataset while using only 0.67% of the trainable parameters. This marks an important breakthrough of this method in the field of point cloud learning.

Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning

Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models

Parameter Efficient Point Cloud Prompt Tuning for Unified Point Cloud Understanding

Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis

Parameter-Efficient Fine-Tuning via Selective Discrete Cosine Transform

Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning

Pre-training Everywhere: Parameter-Efficient Fine-Tuning for Medical Image Analysis via Target Parameter Pre-training

Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies

Gradient-based Parameter Selection for Efficient Fine-Tuning

See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition

Fine-Tuning Point Cloud Transformers with Dynamic Aggregation

UPetu: A Unified Parameter-Efficient Fine-Tuning Framework for Remote Sensing Foundation Model

Positional Prompt Tuning for Efficient 3D Representation Learning

Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey

Parameter Efficient Fine Tuning: A Comprehensive Analysis Across Applications

GIST: Improving Parameter Efficient Fine Tuning via Knowledge Interaction

Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Models

Parameter Efficient Fine-tuning Via Cross Block Orchestration for Segment Anything Model

Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning

An Empirical Study of Parameter Efficient Fine-tuning on Vision-Language Pre-train Model

Spectral Adapter: Fine-Tuning in Spectral Space