Abstract:Abstract knowledge is deeply grounded in many computer-based applications. An important research area of Artificial Intelligence (AI) deals with the automatic derivation of knowledge from data. Machine learning offers the according algorithms. One area of research focuses on the development of biologically inspired learning algorithms. The respective machine learning methods are based on neurological concepts so that they can systematically derive knowledge from data and store it. One type of machine learning algorithms that can be categorized as "deep learning" model is referred to as Deep Neural Networks (DNNs). DNNs consist of multiple artificial neurons arranged in layers that are trained by using the backpropagation algorithm. These deep learning methods exhibit amazing capabilities for inferring and storing complex knowledge from high-dimensional data. However, DNNs are affected by a problem that prevents new knowledge from being added to an existing base. The ability to continuously accumulate knowledge is an important factor that contributed to evolution and is therefore a prerequisite for the development of strong AIs. The so-called "catastrophic forgetting" (CF) effect causes DNNs to immediately loose already derived knowledge after a few training iterations on a new data distribution. Only an energetically expensive retraining with the joint data distribution of past and new data enables the abstraction of the entire new set of knowledge. In order to counteract the effect, various techniques have been and are still being developed with the goal to mitigate or even solve the CF problem. These published CF avoidance studies usually imply the effectiveness of their approaches for various continual learning tasks. This dissertation is set in the context of continual machine learning with deep learning methods. The first part deals with the development of an ...

Continual Learning and Catastrophic Forgetting

Progressive Learning without Forgetting

Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting

Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy

Continual Lifelong Learning with Neural Networks: A Review

Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes

Challenging Common Assumptions about Catastrophic Forgetting

A Methodology-Oriented Study of Catastrophic Forgetting in Incremental Deep Neural Networks

Wide Neural Networks Forget Less Catastrophically

Measuring Catastrophic Forgetting in Neural Networks

A Neural Network Model of Continual Learning with Cognitive Control

Quantum Continual Learning Overcoming Catastrophic Forgetting

Knowledge Accumulation in Continually Learned Representations and the Issue of Feature Forgetting

Statistical Mechanical Analysis of Catastrophic Forgetting in Continual Learning with Teacher and Student Networks

Automating Continual Learning

Dissecting Catastrophic Forgetting in Continual Learning by Deep Visualization

Rethinking Continual Learning for Autonomous Agents and Robots

Deep Generative Dual Memory Network for Continual Learning

Learning to Continually Learn with the Bayesian Principle

Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping

Continual Learning with Deep Learning Methods in an Application-Oriented Context