Abstract: This work investigates the problem of efficiently learning discriminative low-dimensional representations of multiclass large-scale image objects. We propose a generic deep learning approach by taking advantages of Convolutional Neural Networks (CNN), sparse dictionary learning, and orthogonal projections. CNN is not only powerful on feature extraction, but also robust to spatial variance and changes. Sparse dictionary learning is well known for disentangling nonlinear underlying discriminative factors in data. The orthogonal projection is a notable efficient tool to project multi-class data onto low dimensional discriminative subspace. The proposed procedure can be summarized as follows. At first, a CNN is employed to extract high-dimensional (HD) preliminary convolutional features. Secondly, to avoid the high computational cost of direct sparse coding on HD CNN features, we present to learn sparse representation (SR) in an orthogonal projected space over a taskdriven sparsifying dictionary. We then exploit the discriminative projection on SR. The whole learning process is treated as a joint optimization problem of trace quotient maximization, which involves the CNN parameters, the orthogonal projection on CNN features, the dictionary and the discriminative projection on sparse codes. The related cost function is well defined on product manifold of the Stiefel manifold, the Oblique manifold, and the Grassmann manifold. It is optimized via a geometrical stochastic gradient descent algorithm. Finally, the quality of dictionary and projections learned by the proposed approach is further investigated in terms of image classification. The experimental results show that the approach can achieve a strongly competitive performance with state-of-the-art image classification methods.

Convergence of an Alternating Direction and Projection Method for Sparse Dictionary Learning

Alternating Direction Method of Multipliers for Solving Dictionary Learning Models

Dictionary Learning for Sparse Coding: Algorithms and Convergence Analysis

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Blockwise Coordinate Descent Schemes for Efficient and Effective Dictionary Learning.

L0 Norm Based Dictionary Learning by Proximal Methods with Global Convergence

Forward Basis Selection for Sparse Approximation over Dictionary

A Non-Convex Relaxation Approach to Sparse Dictionary Learning

Analysis Dictionary Learning Using Block Coordinate Descent Framework with Proximal Operators

Low-Rank Plus Sparse Reconstruction Using Dictionary Learning for 3D-Mri

A Novel Dictionary Learning Method for Sparse Representation with Nonconvex Regularizations.

ODD: an Algorithm of Online Directional Dictionary Learning for Sparse Representation

Joint Learning Dictionary and Discriminative Features for High Dimensional Data.

Convergence of alternating minimisation algorithms for dictionary learning

Simple Alternating Minimization Provably Solves Complete Dictionary Learning

Dictionary learning for sparse representation: Complexity and algorithms

Forward Basis Selection for Pursuing Sparse Representations over a Dictionary

Joint Learning of Discriminative Low-dimensional Image Representations Based on Dictionary Learning and Two-layer Orthogonal Projections

An Efficient Algorithm for Dictionary Learning with a Mixed-Norm Regularizer for Sparsity Based on Proximal Operator.

A Fast Algorithm for Learning Overcomplete Dictionary for Sparse Representation Based on Proximal Operators

A Convergent ADMM Framework for Efficient Neural Network Training