Abstract:In this paper, we propose two manifold regularization (MR) based distributed semi-supervised learning (DSSL) algorithms using the random vector functional link (RVFL) network and alternating direction method of multipliers (ADMM) strategy. In DSSL problems, training data consisting of labeled and unlabeled samples are often large-scale or high-dimension and split across samples or features. These distributed data separately stored over a communication network where each node has only access to its own data and can only communicate with its neighboring nodes. In many scenarios, centralized algorithms cannot be applied to solve DSSL problems. In our previous work, we proposed a MR based DSSL algorithm, denoted as the D-LapWNN algorithm, to solve DSSL problems with distributed samples. It has been proved that the D-LapWNN algorithm, combining the wavelet neural network (WNN) with the zero-gradient-sum (ZGS) strategy, is an efficient DSSL algorithm with distributed samples or horizontally partitioned data. The drawback of the D-LapWNN algorithm is that the loss function of each node or agent over the communication network must be twice continuously differentiable. In order to extend our previous work and settle the corresponding drawback, we propose a horizontally DSSL (HDSSL) algorithm to solve DSSL problems with distributed samples. Then, we novelly propose a vertically DSSL (VDSSL) algorithm to solve DSSL problems with distributed features or vertically partitioned data. As far as we know, the VDSSL algorithm is the first work focusing on DSSL problems with distributed features. During the learning process of the proposed algorithms, nodes over the communication network only exchange coefficients rather than raw data. It means that the proposed algorithms are privacy-preserving methods. Finally, some simulations are given to show the efficiency of the proposed algorithms.

The distributed L1/2 regularization

Distributed Logistic Regression for Separated Massive Data.

Unified algorithms for distributed regularized linear regression model

Distributed Linear Regression with Compositional Covariates

Model Selection with Distributed SCAD Penalty

Regularized Adaptive Huber Matrix Regression and Distributed Learning

Distributed Principal Component Analysis Based on Randomized Low-Rank Approximation.

Linear Convergence of ADMM under Metric Subregularity for Distributed Optimization.

A distributed block coordinate descent method for training $l_1$ regularized linear classifiers

Distributed Learning and Distribution Regression of Coefficient Regularization.

Sublinear and Linear Convergence of Modified ADMM for Distributed Nonconvex Optimization

A Two-Layer Distributed Algorithm Using Neurodynamic System for Solving L1-Minimization

Debiased distributed learning for sparse partial linear models in high dimensions

On Linear Convergence of ADMM for Decentralized Quantile Regression

Fast Classification Algorithms Via Distributed Accelerated Alternating Direction Method of Multipliers.

L₂,₁-Norm Regularized Robust and Sparse Linear Discriminant Analysis Via an Alternating Direction Method of Multipliers

Convergence Analysis of Distributed Multi-Penalty Regularized Pairwise Learning

Distributed non-convex regularization for generalized linear regression

Optimal Subsample Selection for Massive Logistic Regression with Distributed Data

Distributed semi-supervised learning algorithms for random vector functional-link networks with distributed data splitting across samples and features

Robust Regression for Non-Randomly Distributed Big Data with Application to Nonconvex Penalized Learning