Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Need

Martin Wistuba,Prabhu Teja Sivaprasad,Lukas Balles,Giovanni Zappella

2024-06-05

Abstract:Recent Continual Learning (CL) methods have combined pretrained Transformers with prompt tuning, a parameter-efficient fine-tuning (PEFT) technique. We argue that the choice of prompt tuning in prior works was an undefended and unablated decision, which has been uncritically adopted by subsequent research, but warrants further research to understand its implications. In this paper, we conduct this research and find that the choice of prompt tuning as a PEFT method hurts the overall performance of the CL system. To illustrate this, we replace prompt tuning with LoRA in two state-of-the-art continual learning methods: Learning to Prompt and S-Prompts. These variants consistently achieve higher accuracy across a wide range of domain-incremental and class-incremental benchmarks, while being competitive in inference speed. Our work highlights a crucial argument: unexamined choices can hinder progress in the field, and rigorous ablations, such as the PEFT method, are required to drive meaningful adoption of CL techniques in real-world applications.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The paper primarily explores the choice of Parameter-Efficient Fine-Tuning (PEFT) techniques in Continual Learning (CL), particularly for pre-trained Transformer models. The paper points out that in many CL studies, Prompt Tuning is widely adopted as a PEFT method, but this choice has not been sufficiently validated and evaluated.

The main findings of the paper include:
1. **Limitations of Prompt Tuning**: The authors demonstrate through experiments that Prompt Tuning performs worse in several benchmarks compared to another PEFT method, LoRA, especially in Domain-Incremental and Class-Incremental tasks.
2. **Effectiveness of Alternatives**: The paper proposes replacing two CL methods based on Prompt Tuning (S-Prompts and L2P) with methods based on LoRA, and shows consistent performance improvements of these modified versions in different scenarios.
3. **Questioning Existing Practices**: The authors believe that the choice of Prompt Tuning may be due to habitual use and lack of critical evaluation rather than inherent advantages. They suggest that more consideration should be given to other PEFT methods like LoRA in CL research.

In summary, the paper aims to remind researchers not to blindly follow the practice of Prompt Tuning but to more comprehensively evaluate different PEFT techniques to advance the practical application and development of the CL field.