Abstract:Consider the lifelong machine learning paradigm whose objective is to learn a sequence of tasks depending on previous experiences, e.g., knowledge library or deep network weights. However, the knowledge libraries or deep networks for most recent lifelong learning models are of prescribed size and can degenerate the performance for both learned tasks and coming ones when facing with a new task environment (cluster). To address this challenge, we propose a novel incremental clustered lifelong learning framework with two knowledge libraries: feature learning library and model knowledge library, called Flexible Clustered Lifelong Learning (FCL3). Specifically, the feature learning library modeled by an autoencoder architecture maintains a set of representation common across all the observed tasks, and the model knowledge library can be self-selected by identifying and adding new representative models (clusters). When a new task arrives, our FCL3 model firstly transfers knowledge from these libraries to encode the new task, i.e., effectively and selectively soft-assigning this new task to multiple representative models over feature learning library. Then: 1) the new task with a higher outlier probability will be judged as a new representative, and used to redefine both feature learning library and representative models over time; or 2) the new task with lower outlier probability will only refine the feature learning library. For model optimization, we cast this lifelong learning problem as an alternating direction minimization problem as a new task comes. Finally, we evaluate the proposed framework by analyzing several multitask data sets, and the experimental results demonstrate that our FCL3 model can achieve better performance than most lifelong learning frameworks, even batch clustered multitask learning models.

What and How: Generalized Lifelong Spectral Clustering Via Dual Memory

Lifelong Spectral Clustering.

Lifelong Multi-view Spectral Clustering

Deep Spectral Clustering with Constrained Laplacian Rank

Attention Non-Negative Spectral Clustering

Fast spectral clustering with self-adapted bipartite graph learning

Clustered Lifelong Learning Via Representative Task Selection

Rank-Constrained Spectral Clustering with Flexible Embedding

Global discriminative-based nonnegative spectral clustering.

Lifelong Visual-Tactile Spectral Clustering for Robotic Object Perception

Spectral Clustering on Multiple Manifolds

Fast Spectral Clustering Learning with Hierarchical Bipartite Graph for Large-Scale Data

Spectral clustering with linear embedding: A discrete clustering method for large-scale data

Spectral Clustering With Adaptive Neighbors for Deep Learning

Representative Task Self-Selection for Flexible Clustered Lifelong Learning

Unified Spectral Clustering with Optimal Graph

Structured Graph Optimization for Joint Spectral Embedding and Clustering

Discrete and Balanced Spectral Clustering with Scalability

Deep Spectral Clustering with Regularized Linear Embedding for Hyperspectral Image Clustering

Spectral clustering based on iterative optimization for large-scale and high-dimensional data.

Large Graph Clustering with Simultaneous Spectral Embedding and Discretization