Abstract:<p>How to design deep neural networks (DNNs) for the representation and analysis of high dimensional but small sample size data is still a big challenge. One solution is to construct a sparse network. At present, there exist many approaches to achieve sparsity for DNNs by regularization, but most of them are carried out only in the pre-training process due to the difficulty in the derivation of explicit formulae in the fine-tuning process. In this paper, a log-sum function is used as the regularization terms for both the responses of hidden neurons and the network connections in the loss function of the fine-tuning process. It provides a better approximation to the L0-norm than several often used norms. Based on the gradient formula of the loss function, the fine-tuning process can be executed more efficiently. Specifically, the commonly used gradient calculation in many deep learning research platforms, such as PyTorch or TensorFlow, can be accelerated. Given the analytic formula for calculating gradients used in any layer of DNN, the error accumulated from successive numerical approximations in the differentiation process can be avoided. With the proposed log-sum enhanced sparse deep neural network (LSES-DNN), the sparsity of the responses and the connections can be well controlled to improve the adaptivity of DNNs. The proposed model is applied to MRI data for both the diagnosis of schizophrenia and the study of brain developments. Numerical experiments demonstrate its superior performance among several classical classifiers tested.</p>

LogSumExp for Unlabeled Data Processing

Generalized entropy based semi-supervised learning

Exploiting Unlabeled Data via Partial Label Assignment for Multi-Class Semi-Supervised Learning

Locally Weighted Prediction Methods for Latent Factor Analysis with Supervised and Semisupervised Process Data

Noise-injected Consistency Training and Entropy-constrained Pseudo Labeling for Semi-supervised Extractive Summarization.

The Use of Unlabeled Data in Predictive Modeling

Semi-supervised Sparse Metric Learning Using Alternating Linearization Optimization

Similar Data Points Identification with LLM: A Human-in-the-loop Strategy Using Summarization and Hidden State Insights

Semi-Supervised Process Data Regression and Application Based on Latent Factor Analysis Model

Unlabeled Data and Multiple Views

Unsupervised Domain Expansion from Multiple Sources

Estimate Unlabeled-Data-Distribution for Semi-supervised PU Learning

Unsupervised Linear Discriminant Analysis for Jointly Clustering and Subspace Learning

LogSummary: Unstructured Log Summarization for Software Systems

Lagrangian supervised and semi-supervised extreme learning machine

Log-sum enhanced sparse deep neural network

Unsupervised learning with contrastive latent variable models

LogAvgExp Provides a Principled and Performant Global Pooling Operator

Dlsa: Semi-supervised Partial Label Learning Via Dependence-Maximized Label Set Assignment

Semi-supervised Label Enhancement Via Structured Semantic Extraction

Latent Semantics Encoding for Label Distribution Learning.