Abstract:Continual learning is a subfield of machine learning, which aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting what was learned in the past. In this work, we take a step back, and ask: "Why should one care about continual learning in the first place?". We set the stage by examining recent continual learning papers published at four major machine learning conferences, and show that memory-constrained settings dominate the field. Then, we discuss five open problems in machine learning, and even though they might seem unrelated to continual learning at first sight, we show that continual learning will inevitably be part of their solution. These problems are model editing, personalization and specialization, on-device learning, faster (re-)training and reinforcement learning. Finally, by comparing the desiderata from these unsolved problems and the current assumptions in continual learning, we highlight and discuss four future directions for continual learning research. We hope that this work offers an interesting perspective on the future of continual learning, while displaying its potential value and the paths we have to pursue in order to make it successful. This work is the result of the many discussions the authors had at the Dagstuhl seminar on Deep Continual Learning, in March 2023.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve The paper "Continual Learning: Applications and the Road Forward" aims to explore the importance and future development directions of continual learning in machine learning. Specifically, the authors focus on the following aspects: 1. **Why continual learning is needed**: - Continual learning allows machine learning models to accumulate knowledge without forgetting previously learned content as they continuously receive new data. This is different from the traditional machine learning assumption that all training and testing data are independent and identically distributed (i.i.d.). - Deep learning models particularly need the ability for continual learning because they are highly dependent on data. When data distribution changes over time, models often adapt to the most recent data, leading to "catastrophic forgetting." 2. **Current state of continual learning research**: - The authors analyzed continual learning papers published in four major machine learning conferences (ECCV, NeurIPS, CVPR, ICML) and found that most research focuses on memory-constrained settings, while computational costs are often overlooked. - Existing continual learning methods typically optimize performance by limiting the amount of stored data but rarely consider computational costs. 3. **Five open problems and their relation to continual learning**: - **Model editing**: The need to selectively edit models to correct erroneous predictions or update outdated information without forgetting other relevant knowledge. - **Personalization and specialization**: Large pre-trained models may perform poorly in specific domains or personal applications, requiring fine-tuning or personalized adjustments. - **On-device learning**: Many deep learning applications need to be updated on deployment devices, especially when cloud computing is unavailable or user privacy needs to be protected. - **Faster retraining**: In industrial environments, when new data becomes available or a change in data distribution is detected, models are often retrained from scratch, which is time-consuming and resource-intensive. Continual learning can significantly reduce retraining time and resource consumption. - **Reinforcement learning**: In reinforcement learning, agents learn through interaction with the environment, leading to natural non-stationarity, requiring continual learning to cope with new environment explorations. 4. **Future research directions**: - By comparing the needs of these unresolved issues with the assumptions of current continual learning methods, the authors propose four future research directions aimed at improving the balance between performance and efficiency in continual learning. In summary, this paper attempts to demonstrate the potential value of continual learning by analyzing its applications in real-world problems and proposes future research directions to advance the development of continual learning.

Continual Learning: Applications and the Road Forward

Continual Learning with Deep Learning Methods in an Application-Oriented Context

Continual Learning as Computationally Constrained Reinforcement Learning

A Comprehensive Survey of Continual Learning: Theory, Method and Application

Is Continual Learning Ready for Real-world Challenges?

A Unified Framework for Continual Learning and Machine Unlearning

Recent Advances of Continual Learning in Computer Vision: An Overview

Continual Learning in Human Activity Recognition: an Empirical Analysis of Regularization

Dissecting Continual Learning a Structural and Data Analysis

CVPR 2020 Continual Learning in Computer Vision Competition: Approaches, Results, Current Challenges and Future Directions

Rethinking Continual Learning for Autonomous Agents and Robots

Continuum: Simple Management of Complex Continual Learning Scenarios

Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges

Advancing continual lifelong learning in neural information retrieval: definition, dataset, framework, and empirical evaluation

Continual Learning for Class- and Domain-Incremental Semantic Segmentation

Continual Learning of Large Language Models: A Comprehensive Survey

A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning

Beyond Supervised Continual Learning: a Review

Continual Learning with Neuromorphic Computing: Theories, Methods, and Applications