Supplementary Materials for the Paper "A Convergent Incoherent Dictionary Learning Algorithm for Sparse Coding, ECCV, Zurich, 2014"

Chenglong Bao,Yuhui Quan,Hui Ji

2014-01-01

Abstract:This document accompanies the paper referenced as "A convergent incoherent dictionary learning algorithm for sparse coding, ECCV, Zurich, 2014". We provide a complete proof of Theorem 1 with all the details.

What problem does this paper attempt to address?

Gradient-based Dictionary Optimization for Compressive Spectral Imaging

Chenning Tao,Peng Sun,Siqi Liu,Chang Wang,Jinlei Zhang,Zhanghao Ding,Zhenrong Zheng

DOI: https://doi.org/10.1117/12.2573307

2020-01-01

Abstract:In compressive spectral imaging, three-dimensional spatio-spectral data cubes are recovered from two-dimensional projections. The quality of the compressive-sensing-based reconstruction is dependent on the coherence of the sensing matrix, which is determined by the system projection and the sparse prior. Studies on the optimization of the system projection, which mainly deals with the coded aperture, successfully decreases the coherence of the sensing matrix and improves the reconstruction quality. However, the optimization of the sparse prior considering the relationship between the system projection and the sparse prior remains a challenge. In this paper, we propose a gradient-descent-based sparse prior optimization algorithm for the coherence minimization of the sensing matrix in compressive spectral imaging. The Frobenius norm coherence is introduced as the cost function for the optimization, and the overcomplete dictionary is chosen as the sparse prior to solve the optimal sparse representation in the reconstruction as it provides higher degree of freedom for optimization compared to common orthogonal bases. The optimized dictionary effectively decreases the coherence of the sensing matrix from 0.880 to 0.604 and significantly improves the quantitative image quality metrics of the reconstructed hyperspectral images with the corresponding peak signal-to-noise ratio (PSNR) increased by 9 dB, the structural similarity (SSIM) above 0.98, and the spectrum angular mapper (SAM) below 0.1. Furthermore, the requirement of the sampling snapshots is reduced, which is shown by similar image quality metrics between the reconstructed hyperspectral images of only 1 snapshot with the optimized dictionary and of more than 5 snapshots with the non-optimized dictionary.
Supplementary Material to “ Convolutional Sparse Coding for Image Super-resolution ”

Shuhang Gu,Wangmeng Zuo,Qi Xie,Deyu Meng,Xiangchu Feng,Lei Zhang

2015-01-01

Abstract:Shuhang Gu, Wangmeng Zuo, Qi Xie, Deyu Meng, Xiangchu Feng, Lei Zhang1,∗ Dept. of Computing, The Hong Kong Polytechnic University, Hong Kong, China School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China School of Mathematics and Statistics, Xi′an Jiaotong University, Xi′an, China Dept. of Applied Mathematics, Xidian University, Xian, China {cssgu, cslzhang}@comp.polyu.edu.hk, cswmzuo@gmail.com, qixie.liwu@stu.xjtu.edu.cn, dymeng@mail.xjtu.edu.cn, xcfeng@mail.xidian.edu.cn
Dictionary Learning for Sparse Coding: Algorithms and Convergence Analysis

Chenglong Bao,Yuhui Quan,Hui Ji

DOI: https://doi.org/10.1109/tpami.2015.2487966

IF: 23.6

2015-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:In recent years, sparse coding has been widely used in many applications ranging from image processing to pattern recognition. Most existing sparse coding based applications require solving a class of challenging non-smooth and non-convex optimization problems. Despite the fact that many numerical methods have been developed for solving these problems, it remains an open problem to find a numerical method which is not only empirically fast, but also has mathematically guaranteed strong convergence. In this paper, we propose an alternating iteration scheme for solving such problems. A rigorous convergence analysis shows that the proposed method satisfies the global convergence property: the whole sequence of iterates is convergent and converges to a critical point. Besides the theoretical soundness, the practical benefit of the proposed method is validated in applications including image restoration and recognition. Experiments show that the proposed method achieves similar results with less computation when compared to widely used methods such as K-SVD.
K-CPD: Learning of Overcomplete Dictionaries for Tensor Sparse Coding.

Guifang Duan,Hongcui Wang,Zhenyu Liu,Junping Deng,Yen-Wei Chen

2012-01-01

Abstract:Recently sparse coding has received expressions of interest in the field of pattern recognition. Most existing methods take the data-as-vector formulation, and deal with images (the second order tensor) or volumes (the third order tensor) by vectorization. However, such kind of vectorization will lose the original structure of the data and reduce the reliability of post processing, leading a poor representation. In this paper, we propose a new algorithm of overcomplete dictionary learning for tensor sparse coding, named K-CPD, by extension of K-SVD [8] from vector formulation to tensor formulation. A multilinear orthogonal matching pursuit (MOMP) algorithm is also developed for calculating sparse representation of tensor signal. We evaluate the performance of K-CPD for image denoising, and the results demonstrate that the proposed method surpasses the conventional methods.
Sparse Canonical Correlation Analysis: New Formulation and Algorithm.

Delin Chu,Li-Zhi Liao,Michael K. Ng,Xiaowei Zhang

DOI: https://doi.org/10.1109/tpami.2013.104

IF: 23.6

2013-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:In this paper, we study canonical correlation analysis (CCA), which is a powerful tool in multivariate data analysis for finding the correlation between two sets of multidimensional variables. The main contributions of the paper are: 1) to reveal the equivalent relationship between a recursive formula and a trace formula for the multiple CCA problem, 2) to obtain the explicit characterization for all solutions of the multiple CCA problem even when the corresponding covariance matrices are singular, 3) to develop a new sparse CCA algorithm, and 4) to establish the equivalent relationship between the uncorrelated linear discriminant analysis and the CCA problem. We test several simulated and real-world datasets in gene classification and cross-language document retrieval to demonstrate the effectiveness of the proposed algorithm. The performance of the proposed method is competitive with the state-of-the-art sparse CCA algorithms.
Supplemental Document for “Matrix Completion with Covariate Information”

Xiaojun Mao,Songxi Chen,R. K. Wong

2017-01-01

Abstract:This document provides supplementary material to the article “Matrix Completion with Covariate Information” written by the same authors. S1 Proof of Propositions Proof of Proposition 1. We have E ∥Xβ +B −W ◦Θ∗ ◦ Y ∥2F = ∥Xβ +B∥2F − 2 ⟨Xβ +B,E (W ◦Θ ∗ ◦ Y )⟩+ E ∥W ◦Θ∗ ◦ Y ∥2F = ∥(Xβ +B)− (Xβ0 +B0)∥2F − ∥Xβ0 +B0∥ 2 F + ∑ ij ( (Xβ0)ij +B0ij )2 + σ2 ij θij , due to Conditions C1(a) and C4. For any minimizer (βs,Bs) of R, we have Xβ0+B0 = Xβs+Bs, which implies X(β0−βs) = Bs−B0. Since Bs−B0 ∈ N (X), we can conclude both Xβs = Xβ0 ∗Xiaojun Mao is Ph.D. candidate, Department of Statistics, Iowa State University, Ames, IA 50011, USA (Email: mxjki@iastate.edu). †Author of Correspondence. Song Xi Chen is Chair Professor, Department of Business Statistics and Econometrics, Guanghua School of Management and Center for Statistical Science, Peking University, Beijing 100651, China (Email: csx@gsm.pku.edu.cn). His research is partially supported by Chinas National Key Research Special Program Grants 2016YFC0207701 and 2015CB856000, and National Natural Science Foundation of China grants 11131002, 71532001 and 71371016. ‡Raymond K. W. Wong is Assistant Professor, Department of Statistics, Texas A&M University, College Station, TX 77843, USA (Email: raywong@stat.tamu.edu). His research is partially supported by the National Science Foundation under Grants DMS-1612985 and DMS-1711952 (subcontract).
Self-Distillation for Few-Shot Image Captioning (Supplementary Materials)

Xianyu Chen,Ming Jiang,Qi Zhao

2020-01-01

Abstract:In this supplementary material, we provide additional information and results of our proposed few-shot image captioner on the Microsoft COCO dataset [7] and Flickr30K dataset [8]. In Section 1, we present the algorithm of our few-shot image captioning. Next, in Section 2, we show a comprehensive ablation study for the hyper-parameters and the effects on different base models. Further, in Section 3, we present the quantitative results for Flickr30K [8] with state-of-the-art approach and our different baselines. Finally, in Section 4, we report and discuss qualitative results of various baselines and state-of-the-art approaches.
Polarimetric Multi-View Stereo Supplementary Material

Zhaopeng Cui,Jinwei Gu,Boxin Shi,Ping Tan,Jan Kautz,Simon Fraser

2017-01-01

Abstract:The supplementary material contains the complete proof of the appendix in our paper and more experimental results as follows: Section 1 describes the detailed proof of Equations (1) and (2) as well as Proposition 1 in the main paper based on the Muller calculus [1]; Section 2 presents more quantitative evaluation on the synthetic data, the running time and more experiment on real data with the laser scanned ground truth.
Self-Explanatory Convex Sparse Representation for Image Classification

Bao-Di Liu,Yu-Xiong Wang,Bin Shen,Yu-Jin Zhang,Yan-Jiang Wang,Wei-Feng Liu

DOI: https://doi.org/10.1109/SMC.2013.363

2013-01-01

Abstract:Sparse representation technique has been widely used in various areas of computer vision over the last decades. Unfortunately, in the current formulations, there are no explicit relationship between the learned dictionary and the original data. By tracing back and connecting sparse representation with the K-means algorithm, a novel variation scheme termed as self-explanatory convex sparse representation (SCSR) has been proposed in this paper. To be specific, the basis vectors of the dictionary are refined as convex combination of the data points. The atoms now would capture a notion of centroids similar to K-means, leading to enhanced interpretability. Sparse representation and K-means are thus unified under the same framework in this sense. Besides, an appealing property also emerges that the weight and code matrices both tend to be naturally sparse without additional constraints. Compared with the standard formulations, SCSR is easier to be extended into the kernel space. To solve the corresponding sparse coding sub problem and dictionary learning sub problem, block-wise coordinate descent and Lagrange multipliers are proposed accordingly. To validate the proposed algorithm, it is implemented in image classification, a successful applications of sparse representation. Experimental results on several benchmark data sets, such as UIUC-Sports, Scene 15, and Caltech-256 demonstrate the effectiveness of our proposed algorithm.
Incoherent Dictionary Learning for Sparse Representation

Tong Lin,Shi Liu,Hongbin Zha

2012-01-01

Abstract:Recent years have witnessed a growing interest in the sparse representation problem. Prior work demonstrated that adaptive dictionary learning techniques can greatly improve the performance of sparse representation approaches. Existing techniques mainly focus on the reconstructive accuracies and the discriminative power of the learned dictionary, whereas the mutual incoherence between any two basis atoms has been rarely studied yet. This paper proposes a novel method by explicitly incorporating a correlation penalty into the dictionary learning model. Experiments show that the proposed method can remarkably reduce the correlation measure of the learned dictionaries, and at the same time achieve higher classification accuracies than state-of-the-art algorithms.
Transfer Sparse Coding for Robust Image Representation

Mingsheng Long,Guiguang Ding,Jianmin Wang,Jiaguang Sun,Yuchen Guo,Philip S. Yu

DOI: https://doi.org/10.1109/cvpr.2013.59

2013-01-01

Computer Vision and Pattern Recognition

Abstract:Sparse coding learns a set of basis functions such that each input signal can be well approximated by a linear combination of just a few of the bases. It has attracted increasing interest due to its state-of-the-art performance in BoW based image representation. However, when labeled and unlabeled images are sampled from different distributions, they may be quantized into different visual words of the codebook and encoded with different representations, which may severely degrade classification performance. In this paper, we propose a Transfer Sparse Coding (TSC) approach to construct robust sparse representations for classifying cross-distribution images accurately. Specifically, we aim to minimize the distribution divergence between the labeled and unlabeled images, and incorporate this criterion into the objective function of sparse coding to make the new representations robust to the distribution difference. Experiments show that TSC can significantly outperform state-of-the-art methods on three types of computer vision datasets.
Understanding L4-Based Dictionary Learning: Interpretation, Stability, and Robustness.

Yuexiang Zhai,Hermish Mehta,Zhengyuan Zhou,Yi Ma

2020-01-01

Abstract:Recently $\ell^4$-norm maximization has been proposed to solve the sparse dictionary learning (SDL) problem. The simple MSP (matching, stretching, and projection) algorithm proposed by \cite{zhai2019a} has shown to be surprisingly efficient and effective. This paper aims to better understand this algorithm from its strong geometric and statistical connections with the classic PCA and ICA, as well as their associated fixed-point style algorithms. Such connections provide a unified way of viewing problems that pursue {\em principal}, {\em independent}, or {\em sparse} components of high-dimensional data. Our studies reveal additional good properties of the $\ell^4$-maximization: not only is the MSP algorithm for sparse coding insensitive to small noise, but also robust to outliers, and resilient to sparse corruptions. We provide preliminary statistical justification for such inherently nice properties. To corroborate the theoretical analysis, we also provide extensive and compelling experimental evidence with both synthetic data and real images.
Online Convolutional Sparse Coding with Sample-Dependent Dictionary

Yaqing Wang,Quanming Yao,James T. Kwok,Lionel M. Ni

DOI: https://doi.org/10.48550/arxiv.1804.10366

2018-01-01

Abstract:Convolutional sparse coding (CSC) has been popularly used for the learning of shift-invariant dictionaries in image and signal processing. However, existing methods have limited scalability. In this paper, instead of convolving with a dictionary shared by all samples, we propose the use of a sample-dependent dictionary in which each filter is a linear combination of a small set of base filters learned from data. This added flexibility allows a large number of sample-dependent patterns to be captured, which is especially useful in the handling of large or high-dimensional data sets. Computationally, the resultant model can be efficiently learned by online learning. Extensive experimental results on a number of data sets show that the proposed method outperforms existing CSC algorithms with significantly reduced time and space complexities.
“Self-Paced Learning for Matrix Factorization”: Supplementary Material

Qian Zhao,Deyu Meng,Lu Jiang,Qi Xie,Zongben Xu,Alexander G Hauptmann

2015-01-01

Abstract:In this supplementary material, we give the proof of Theorem 1 in the maintext.
A Short Note on Strongly Convex Programming for Exact Matrix Completion and Robust Principal Component Analysis

Qingshan You,Qun Wan,Yipeng Liu

DOI: https://doi.org/10.3934/ipi.2013.7.305

2013-01-01

Inverse Problems and Imaging

Abstract:In paper 'Strongly Convex Programming for Exact Matrix Completion and Robust Principal Component Analysis', an explicit lower bound of $\tau$ is strongly based on Theorem 3.4. However, a coefficient is missing in the proof of Theorem 3.4, which leads to improper result. In this paper, we correct this error and provide the right bound of $\tau$.
Constrained Sparse Concept Coding Algorithm with Application to Image Representation.

Zhenqiu Shu,Chunxia Zhao,Pu Huang

DOI: https://doi.org/10.3837/tiis.2014.09.015

2014-01-01

KSII Transactions on Internet and Information Systems

Abstract:Recently, sparse coding has achieved remarkable success in image representation tasks. In practice, the performance of clustering can be significantly improved if limited label information is incorporated into sparse coding. To this end, in this paper, a novel semi-supervised algorithm, called constrained sparse concept coding (CSCC), is proposed for image representation. CSCC considers limited label information into graph embedding as additional hard constraints, and hence obtains embedding results that are consistent with label information and manifold structure information of the original data. Therefore, CSCC can provide a sparse representation which explicitly utilizes the prior knowledge of the data to improve the discriminative power in clustering. Besides, a kernelized version of our proposed CSCC, namely kernel constrained sparse concept coding (KCSCC), is developed to deal with nonlinear data, which leads to more effective clustering performance. The experimental evaluations on the MNIST, PIE and Yale image sets show the effectiveness of our proposed algorithms.
Designing Incoherent Dictionaries for Compressed Sensing: Algorithm Comparison

Eliyahu Osherovich

DOI: https://doi.org/10.48550/arXiv.1206.4192

2012-06-19

Abstract:A new method presented for design of incoherent dictionaries.

Information Theory,Data Structures and Algorithms
Corrigendum to "Random grid-based visual secret sharing with abilities of OR and XOR decryptions"[Journal of Visual Communication and Image Representation. 24(2013) 48-62]

Saurabh Bora,Aparajita Ojha

DOI: https://doi.org/10.48550/arXiv.1502.04827

2015-02-17

Abstract:It has been observed that the contrast values for (2, 3) VSS scheme, (2, 4) VSS scheme, (3, 5) VSS scheme and (4, 5) VSS scheme claimed by Wu and Sun (2013) are found to be incorrect. Since the same values are cited and compared by many other researchers in their works, we have calculated and presented the correct values of contrast in this note.

Cryptography and Security
Supplementary material : C 3 AE : Exploring the Limits of Compact Model for Age Estimation

Chao Zhang,Shuaicheng Liu,Xun Xu,Ce Zhu

2019-01-01

Abstract:In this section, some detailed results and explanations are included to compare with our plain model. In order to validate our claims that for small size image and model, standard convolution performs better than depth-wise convolution, several variants on MobileNet-V2 and ShuffleNetV2 are compared with the plain model of C3AE. In addition, we also compare with the competitive SSR. The curve of train/validation losses on MobileNet-V2, ShuffleNet-V2, SSR are given in Fig. 1, Fig. 2, Fig. 3 (Some partial results are shown in Fig. 4 (main manuscript)). On one hand, for any finetuning of factors in Tab. 2 (main manuscript), the plain model of C3AE consistently achieves better result. On the other hand, considering the gap between train and validation loss, our plain model is the lowest. We could argue that the plain model of C3AE is more robust on the generalization. From Fig. 3, our plain model performs better than SSR, both with standard convolution. In the experimental implementation, SSR adopts the full model instead of the plain model. This comparison further shows the superiority of our plain model. In general, Tab. 2 (main manuscript), Fig. 1, Fig. 2 and Fig. 3, with three large datasets IMDB (460,000 images), WIKI (62,000 images) and Morph II(55,000 images), supports the effectiveness of our claims.
Equiangular Kernel Dictionary Learning with Applications to Dynamic Texture Analysis

Yuhui Quan,Chenglong Bao,Hui Ji

DOI: https://doi.org/10.1109/cvpr.2016.40

2016-01-01

Abstract:Most existing dictionary learning algorithms consider a linear sparse model, which often cannot effectively characterize the nonlinear properties present in many types of visual data, e.g. dynamic texture (DT). Such nonlinear properties can be exploited by the so-called kernel sparse coding. This paper proposed an equiangular kernel dictionary learning method with optimal mutual coherence to exploit the nonlinear sparsity of high-dimensional visual data. Two main issues are addressed in the proposed method: (1) coding stability for redundant dictionary of infinite-dimensional space, and (2) computational efficiency for computing kernel matrix of training samples of high-dimensional data. The proposed kernel sparse coding method is applied to dynamic texture analysis with both local DT pattern extraction and global DT pattern characterization. The experimental results showed its performance gain over existing methods.

Supplementary Materials for the Paper "A Convergent Incoherent Dictionary Learning Algorithm for Sparse Coding, ECCV, Zurich, 2014"

Gradient-based Dictionary Optimization for Compressive Spectral Imaging

Supplementary Material to “ Convolutional Sparse Coding for Image Super-resolution ”

Dictionary Learning for Sparse Coding: Algorithms and Convergence Analysis

K-CPD: Learning of Overcomplete Dictionaries for Tensor Sparse Coding.

Sparse Canonical Correlation Analysis: New Formulation and Algorithm.

Supplemental Document for “Matrix Completion with Covariate Information”

Self-Distillation for Few-Shot Image Captioning (Supplementary Materials)

Polarimetric Multi-View Stereo Supplementary Material

Self-Explanatory Convex Sparse Representation for Image Classification

Incoherent Dictionary Learning for Sparse Representation

Transfer Sparse Coding for Robust Image Representation

Understanding L4-Based Dictionary Learning: Interpretation, Stability, and Robustness.

Online Convolutional Sparse Coding with Sample-Dependent Dictionary

“Self-Paced Learning for Matrix Factorization”: Supplementary Material

A Short Note on Strongly Convex Programming for Exact Matrix Completion and Robust Principal Component Analysis

Constrained Sparse Concept Coding Algorithm with Application to Image Representation.

Designing Incoherent Dictionaries for Compressed Sensing: Algorithm Comparison

Corrigendum to "Random grid-based visual secret sharing with abilities of OR and XOR decryptions"[Journal of Visual Communication and Image Representation. 24(2013) 48-62]

Supplementary material : C 3 AE : Exploring the Limits of Compact Model for Age Estimation

Equiangular Kernel Dictionary Learning with Applications to Dynamic Texture Analysis