Abstract:Recently, distributed semi-supervised learning has attracted increasing research attention due to its tremendous practical value. A promising distributed semi-supervised learning method should not only achieve desirable classification performance but also protect data privacy in distributed scenarios. Existing approaches typically capture the similarities between data instances with privacy-preserving computations. This paradigm introduces extra computation and heuristic changes to the algorithm, resulting in sub-optimal solutions that are time-consuming. In current distributed semi-supervised learning, instance similarities are widely used to capture the underlying manifold or guide label propagation. This paper emphasizes that instance similarities are not necessary because the structure of data connections can be estimated using coarser-grained information. We propose a Privacy-preserving Mixture-distribution based Graph Smoothing (PPMGS) model for distributed privacy-preserving semi-supervised learning. Our motivation is to construct a graph based on a Gaussian mixture distribution instead of individual data instances, which better captures the underlying data distribution and improves model efficiency. PPMGS includes a privacy-preserving expectation-maximization (EM) phase to estimate the Gaussian mixture distribution depicting the input data and a mixture-distribution-based graph smoothing algorithm to learn a distribution-based classifier by fitting a few labeled samples. Experimental results show that the proposed PPMGS achieves 5%-10% higher accuracy and macro-F1 than state-of-the-art privacy-preserving semi-supervised learning methods. In terms of efficiency, it reduces time cost by 97% and communication cost by 96% in the most complex dataset. The numerical results demonstrate that our proposal outperforms state-of-the-art baselines in both efficiency and effectiveness.

A Multidata-Source Privacy-Preserving Approach: A Semisupervised Learning-Based Model for Migrating Data Annotation

Private Knowledge Transfer via Model Distillation with Generative Adversarial Networks

Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

SEDML: Securely and efficiently harnessing distributed knowledge in machine learning

Privacy‐enhancing machine learning framework with private aggregation of teacher ensembles

A Distributed Approach to Enabling Privacy-Preserving Model-Based Classifier Training

A Privacy-Preserving Computation Framework for Multisource Label Propagation Services

SoK: Training Machine Learning Models over Multiple Sources with Privacy Preservation

Distributed Intelligent Model for Privacy and Secrecy in Preschool Education

Privacy-Preserving Distributed Multi-Task Learning Against Inference Attack in Cloud Computing

Ppmgs: an Efficient and Effective Solution for Distributed Privacy-Preserving Semi-Supervised Learning

Securely Outsourcing Machine Learning with Multiple Users

Distributed Modelling Approaches for Data Privacy Preserving

Enhance PATE on Complex Tasks with Knowledge Transferred from Non-Private Data

Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation

FL-PATE: Differentially Private Federated Learning with Knowledge Transfer

Private Semi-supervised Knowledge Transfer for Deep Learning from Noisy Labels

Privacy preserving distributed machine learning with federated learning

Guaranteed Distributed Machine Learning: Privacy-preserving Empirical Risk Minimization