Abstract:Collaborative filtering (CF) works have demonstrated the robust capabilities of Shallow Autoencoders on implicit feedback, showcasing highly competitive performance with other reasonable approaches (e.g., iALS and VAE-CF). However, despite their dual advantages of high performance and simple construction, EASE still exhibits several major shortcomings that must be addressed. To be more precise, the scalability of EASE is limited by the number of items, which determines the storage and inversion cost of a large dense matrix; the square-loss optimization objective does not consistently meet the recommendation task’s requirement for predicting personalized rankings, resulting in sub-optimal outcomes; the regularization coefficients are sensitive and require re-calibration with different datasets, leading to an exhaustive and time-consuming fine-tuning process. In order to address these obstacles, we propose a novel approach called Similarity-Structure Aware Shallow Autoencoder (AutoS 2 AE) that aims to enhance both recommendation accuracy and model efficiency. Our method introduces three similarity structures: Co-Occurrence, KNN, and NSW graphs, which replace the large dense matrix in EASE with a sparse structure, thus facilitating model compression. Additionally, we optimize the model by incorporating a low-rank training component into the matrix and applying a weighted square loss for improved ranking-oriented approximations. To automatically tune the hyperparameters, we further design two validation losses on the validation set for guidance and update the hyperparameters using the gradients of these validation losses. Both theoretical analyses regarding the introduction of similarity structures and empirical evaluations on multiple real-world datasets demonstrate the effectiveness of our proposed method, which significantly outperforms competing baselines.

HSAE: A Hessian Regularized Sparse Auto-Encoders.

Sparse Representation Learning of Data by Autoencoders with L_1/2 Regularization

Robust and structural sparsity auto-encoder with L21-norm minimization

Deep Marginalized Sparse Denoising Auto-Encoder for Image Denoising

Hybrid Embedded Deep Stacked Sparse Autoencoder with w_LPPD SVM Ensemble

An Empirical Analysis of Different Sparse Penalties for Autoencoder in Unsupervised Feature Learning

LMAE: A Large Margin Auto-Encoders for Classification

Study on Sparse De-noising Auto-Encoder Neural Network

Comparison of auto-encoders with different sparsity regularizers

Learning Autoencoders with Relational Regularization

k-Sparse Autoencoders

Single Sample Face Recognition Via Learning Deep Supervised Autoencoders

Sparse Auto-encoder with Smoothed l_1 Regularization

Automated Sparse and Low-Rank Shallow Autoencoders for Recommendation

Stacked Convolutional Sparse Auto-Encoders for Representation Learning

Efficient Dictionary Learning with Switch Sparse Autoencoders

Functional Autoencoder for Smoothing and Representation Learning

Sparse Auto-encoder with Smoothed $$l_1$$l1 Regularization

Locality-Constrained Sparse Auto-Encoder for Image Classification

An Enhanced Hierarchical Extreme Learning Machine With Random Sparse Matrix Based Autoencoder

Facial Expression Recognition Via Learning Deep Sparse Autoencoders.