Abstract:Recently, leveraging pre-training techniques to enhance point cloud models has become a hot research topic. However, existing approaches typically require full fine-tuning of pre-trained models to achieve satisfied performance on downstream tasks, accompanying storage-intensive and computationally demanding. To address this issue, we propose a novel Parameter-Efficient Fine-Tuning (PEFT) method for point cloud, called PointGST (Point cloud Graph Spectral Tuning). PointGST freezes the pre-trained model and introduces a lightweight, trainable Point Cloud Spectral Adapter (PCSA) to fine-tune parameters in the spectral domain. The core idea is built on two observations: 1) The inner tokens from frozen models might present confusion in the spatial domain; 2) Task-specific intrinsic information is important for transferring the general knowledge to the downstream task. Specifically, PointGST transfers the point tokens from the spatial domain to the spectral domain, effectively de-correlating confusion among tokens via using orthogonal components for separating. Moreover, the generated spectral basis involves intrinsic information about the downstream point clouds, enabling more targeted tuning. As a result, PointGST facilitates the efficient transfer of general knowledge to downstream tasks while significantly reducing training costs. Extensive experiments on challenging point cloud datasets across various tasks demonstrate that PointGST not only outperforms its fully fine-tuning counterpart but also significantly reduces trainable parameters, making it a promising solution for efficient point cloud learning. It improves upon a solid baseline by +2.28%, 1.16%, and 2.78%, resulting in 99.48%, 97.76%, and 96.18% on the ScanObjNN OBJ BG, OBJ OBLY, and PB T50 RS datasets, respectively. This advancement establishes a new state-of-the-art, using only 0.67% of the trainable parameters.

UPetu: A Unified Parameter-Efficient Fine-Tuning Framework for Remote Sensing Foundation Model

AiRs: Adapter in Remote Sensing for Parameter-Efficient Transfer Learning

Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning

Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning

See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition

Parameter Efficient Point Cloud Prompt Tuning for Unified Point Cloud Understanding

Parameter-Efficient Fine-Tuning via Selective Discrete Cosine Transform

Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning

Exploring Parameter-Efficient Fine-Tuning to Enable Foundation Models in Federated Learning

Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models

Parameter Efficient Fine-tuning Via Cross Block Orchestration for Segment Anything Model

Advancing Parameter Efficiency in Fine-tuning via Representation Editing

FedPEAT: Convergence of Federated Learning, Parameter-Efficient Fine Tuning, and Emulator Assisted Tuning for Artificial Intelligence Foundation Models with Mobile Edge Computing

Positional Prompt Tuning for Efficient 3D Representation Learning

HUT: A More Computation Efficient Fine-Tuning Method With Hadamard Updated Transformation

SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels

Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies

Parameter-efficient fine-tuning for single image snow removal

Embedded Prompt Tuning: Towards Enhanced Calibration of Pretrained Models for Medical Images

Parameter-Efficient Fine-Tuning With Adapters

Parameter-efficient Fine-tuning in Hyperspherical Space for Open-vocabulary Semantic Segmentation