Abstract:Dictionary learning is an important sparse representation algorithm which has been widely used in machine learning and artificial intelligence. However, for massive data in the big data era, classical dictionary learning algorithms are computationally expensive and even can be infeasible. To overcome this difficulty, we propose new dictionary learning methods based on randomized algorithms. The contributions of this work are as follows. First, we find that dictionary matrix is often numerically low-rank. Based on this property, we apply randomized singular value decomposition (RSVD) to the dictionary matrix, and propose a randomized algorithm for linear dictionary learning. Compared with the classical K-SVD algorithm, an advantage is that one can update all the elements of the dictionary matrix simultaneously. Second, to the best of our knowledge, there are few theoretical results on why one can solve the involved matrix computation problems inexactly in dictionary learning. To fill-in this gap, we show the rationality of this randomized algorithm with inexact solving, from a matrix perturbation analysis point of view. Third, based on the numerically low-rank property and Nyström approximation of the kernel matrix, we propose a randomized kernel dictionary learning algorithm, and establish the distance between the exact solution and the computed solution, to show the effectiveness of the proposed randomized kernel dictionary learning algorithm. Fourth, we propose an efficient scheme for the testing stage in kernel dictionary learning. By using this strategy, there is no need to form nor store kernel matrices explicitly both in the training and the testing stages. Comprehensive numerical experiments are performed on some real-world data sets. Numerical results demonstrate the rationality of our strategies, and show that the proposed algorithms are much efficient than some state-of-the-art dictionary learning algorithms. The MATLAB codes of the proposed algorithms are publicly available from https://github.com/Jiali-yang/RALDL_RAKDL.

Robust Kernel Dictionary Learning Using a Whole Sequence Convergent Algorithm.

Online Kernel Dictionary Learning for Object Recognition

Kernel Regularized Nonlinear Dictionary Learning for Sparse Coding

Self-representative kernel concept factorization

Sparse Representation over Overcomplete Dictionary Based on Bayesian Nonparametric Approximation

Dictionary Learning for Sparse Coding: Algorithms and Convergence Analysis

Online Kernel Learning with a Near Optimal Sparsity Bound

Equiangular Kernel Dictionary Learning with Applications to Dynamic Texture Analysis

Robust kernel adaptive filtering for nonlinear time series prediction

Robust Regularized Kernel Regression.

L0 Norm Based Dictionary Learning by Proximal Methods with Global Convergence

A Non-Convex Relaxation Approach to Sparse Dictionary Learning

Blockwise Coordinate Descent Schemes for Efficient and Effective Dictionary Learning.

Data-driven Robust Optimization Based on Kernel Learning.

Structure Parameter Optimized Kernel Based Online Prediction With a Generalized Optimization Strategy for Nonstationary Time Series

Tactile Object Recognition Using Supervised Dictionary Learning

Robust Non-Linear Matrix Factorization for Dictionary Learning, Denoising, and Clustering

An adaptive kernel dictionary-based low-rank representation method for subspace clustering

Randomized algorithms for large-scale dictionary learning

Optimizing Kernel Machines using Deep Learning

Reduced Kernel Dictionary Learning