Abstract:Deep learning (DL) models are used in a variety of real-world applications but are often vulnerable to privacy attacks. Nevertheless, this DL model is attacked by membership inference attacks, model inversion attacks, reconstruction attacks, model extraction attacks, gradient leakage attacks, correlation attacks, and white box attacks (inference attacks). In order to mitigate this issue, various existing research has attempted to design an effective privacy mechanism. However, the existing schemes failed to obtain higher security in DL because of several limitations like computational complexity, lower efficiency, difficult-to-select parameters, cumulative privacy loss, etc. Recently, the auto-encoder-based deep learning model has become more popular due to its great ability, and its variants have achieved notable success in various fields such as medicine, healthcare and NLP (Natural Language Programming). However, the privacy of the auto-encoder model is affected because of the vulnerable attacks. Thus, to avoid this issue, the proposed study prefers the differential privacy (DP) method for securing the deep auto-encoder model. DP is a privacy-preserving technique that can be used to protect deep learning models from the aforementioned attacks. In this paper, a Differential Privacy-Improved Stochastic Gradient Descent (DP-ISGD) algorithm is proposed to improve the privacy and utility of the Deep Autoencoder method by adding Gaussian noise to the gradients before the clipping process. Thus, the convergence speed and accuracy of the proposed algorithm are enhanced. The experimentation is conducted in the Python platform, and metrics like convergence, accuracy and TPVD (Total Parameters Value Difference) are evaluated to measure the performance of the proposed study. The comparative analysis is performed for the no privacy, privacy with SGD, privacy with batch gradient descent (BGD) and mini-batch gradient descent (MBGD) models. The proposed approach is evaluated against six datasets, Pima Indians Diabetes (PID), Adult, MNIST, CIFAR-10, MovieLens 20 M and CD-FSL, with improved accuracy results of 98.6%, 98.6%, 98.3%, 98.14%, 97.92% and 98.17% for each dataset at the epsilon ( ) value of 0.2. The comparison analysis showed that the proposed algorithm achieves better accuracy than other privacy protection methods. Thus, the significant findings in the proposed work state that the proposed privacy model is suitable for several applications, including medical fields, algorithm development, education and awareness, by affording strong privacy guarantees.

DP-EPSO: Differential privacy protection algorithm based on differential evolution and particle swarm optimization

Adaptive differential privacy preserving based on multi‐objective optimization in deep neural networks

Differentially-Private Deep Learning from an Optimization Perspective.

A(DP)$^2$SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy

Differentially Private Deep Learning with Iterative Gradient Descent Optimization

A Stochastic Gradient Descent Algorithm Based on Adaptive Differential Privacy

A(DP)$^2$2SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy

Differential Privacy Protection over Deep Learning: an Investigation of Its Impacted Factors

Dynamic Differential-Privacy Preserving SGD

SA-DPSGD: Differentially Private Stochastic Gradient Descent based on Simulated Annealing

GReDP: A More Robust Approach for Differential Private Training with Gradient-Preserving Noise Reduction

An Improved Privacy-Preserving Stochastic Gradient Descent Algorithm

Adaptive Differential Privacy Mechanism Based on Entropy Theory for Preserving Deep Neural Networks

DPDR: Gradient Decomposition and Reconstruction for Differentially Private Deep Learning

Differential privacy may have a potential optimization effect on some swarm intelligence algorithms besides privacy-preserving

An Improved Stochastic Gradient Descent Algorithm Based on Rényi Differential Privacy

Data Heterogeneity Differential Privacy: From Theory to Algorithm.

Inverse optimization strategy for improved differential privacy in deep auto encoder

Rethinking Improved Privacy-Utility Trade-off with Pre-existing Knowledge for DP Training

Differentially-Private Deep Learning with Directional Noise