Abstract:This paper aims to propose a distributed semi-supervised learning (D-SSL) algorithm to solve D-SSL problems, where training samples are often extremely large-scale and located on distributed nodes over communication networks. Training data of each node consists of labeled and unlabeled samples whose output values or labels are unknown. These nodes communicate in a distributed way, where each node has only access to its own data and can only exchange local information with its neighboring nodes. In some scenarios, these distributed data cannot be processed centrally. As a result, D-SSL problems cannot be centrally solved by using traditional semi-supervised learning (SSL) algorithms. The state-of-the-art D-SSL algorithm, denoted as Distributed Laplacian Regularization Least Square (D-LapRLS), is a kernel based algorithm. It is essential for the D-LapRLS algorithm to estimate the global Euclidian Distance Matrix (EDM) with respect to total samples, which is time-consuming especially when the scale of training data is large. In order to solve D-SSL problems and overcome the common drawback of kernel based D-SSL algorithms, we propose a novel Manifold Regularization (MR) based D-SSL algorithm using Wavelet Neural Network (WNN) and Zero-Gradient-Sum (ZGS) distributed optimization strategy. Accordingly, each node is assigned an individual WNN with the same basis functions. In order to initialize the proposed D-SSL algorithm, we propose a centralized MR based SSL algorithm using WNN. We denote the proposed SSL and D-SSL algorithms as Laplacian WNN (LapWNN) and distributed LapWNN (D-LapWNN), respectively. The D-LapWNN algorithm works in a fully distributed fashion by using ZGS strategy, whose convergence is guaranteed by the Lyapunov method. During the learning process, each node only exchanges local coefficients with its neighbors rather than raw data. It means that the D-LapWNN algorithm is a privacy preserving method. At last, several illustrative simulations are presented to show the efficiency and advantage of the proposed algorithm.

Coded Distributed Graph-Based Semi-Supervised Learning

A Distributed Algorithm for Graph Semi-Supervised Learning

Distributed Semi-Supervised Metric Learning.

Manifold Regularization Based Distributed Semi-Supervised Learning Algorithm Using Extreme Learning Machine over Time-Varying Network

Two-Stage Coded Distributed Learning: A Dynamic Partial Gradient Coding Perspective

Distributed Semi-Supervised Learning Algorithm Based on Extreme Learning Machine over Networks Using Event-Triggered Communication Scheme

Distributed Online Semi-Supervised Support Vector Machine

Simulation analysis of distance-aware graph-based semi-supervised learning

Pseudo-label based semi-supervised learning in the distributed machine learning framework

Semi-supervised Discriminant Analysis Based on Sparse-Coding Theory

A Framework for Distributed Semi-supervised Learning Using Single-layer Feedforward Networks

Distributed semi-supervised learning algorithms for random vector functional-link networks with distributed data splitting across samples and features

Efficient And Robust Semi-Supervised Learning Over A Sparse-Regularized Graph

Scalable Graph-Based Semi-Supervised Learning Through Sparse Bayesian Model.

Coded Parallelism for Distributed Deep Learning.

Distributionally Robust Semi-Supervised Learning Over Graphs

Approximated Coded Computing: Towards Fast, Private and Secure Distributed Machine Learning

A Distributed Semi-Supervised Learning Algorithm Based on Manifold Regularization Using Wavelet Neural Network.

Distributed Semi-supervised Regression Learning with Coefficient Regularization

Safety-aware Graph-based Semi-Supervised Learning

Mixture Distribution Modeling for Scalable Graph-Based Semi-Supervised Learning.