Abstract:In this manuscript, we extend our previous work on privacy-preserving regression to address multi-output regression problems using data encrypted under a fully homomorphic encryption scheme. We build upon the simplified fixed Hessian approach for linear and ridge regression and adapt our novel LFFR algorithm, initially designed for single-output logistic regression, to handle multiple outputs. We further refine the constant simplified Hessian method for the multi-output context, ensuring computational efficiency and robustness. Evaluations on multiple real-world datasets demonstrate the effectiveness of our multi-output LFFR algorithm, highlighting its capability to maintain privacy while achieving high predictive accuracy. Normalizing both data and target predictions remains essential for optimizing homomorphic encryption parameters, confirming the practicality of our approach for secure and efficient multi-output regression tasks.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is **to develop an effective algorithm for multi - output regression problems under the premise of protecting data privacy**. Specifically, the authors extended their previous work on privacy - protected regression to deal with multi - output regression problems under the fully homomorphic encryption (FHE) scheme. ### Problem Background 1. **Multi - output regression problems**: Multi - output regression refers to the problem of simultaneously predicting multiple continuous target variables. Such problems are widely used in many fields such as industry and environment, such as ecological modeling and energy prediction. 2. **Data privacy problems**: Traditional regression algorithms usually need to access users' private data to build accurate prediction models, which causes a serious risk of data privacy leakage and makes users reluctant to share sensitive information. ### Solutions To solve the above problems, the authors proposed the following methods: - **Extension of the LFFR algorithm**: Extend the LFFR algorithm originally used for single - output logistic regression to multi - output regression problems, and adopt the Simplified Fixed Hessian (SFH) method to ensure computational efficiency and robustness. - **Application of homomorphic encryption**: Through homomorphic encryption technology, calculations are directly performed on encrypted data without decryption, thereby protecting the privacy of user data. - **Normalization processing**: Emphasize the importance of normalizing input data and prediction results when implementing regression algorithms to optimize homomorphic encryption parameters and improve performance. ### Main Contributions 1. **Extension of the LFFR algorithm**: Extend from single - output to multi - output, and use the simplified fixed Hessian minimization method for privacy - protected regression training. 2. **Improvement of the SFH method**: Eliminate the need to calculate the sigmoid function, and use the same calculation circuit as linear regression to effectively model non - linear relationships. 3. **Importance of normalization**: Normalize not only the input data but also the prediction results to achieve the best performance. ### Technical Details 1. **Linear regression model**: Briefly review the multi - output linear regression model and its optimization problems. 2. **Simplified Fixed Hessian (SFH)**: Describe in detail how to construct the simplified fixed Hessian matrix for multi - output linear regression. 3. **Logistic Function for Regression**: Introduce the calculation methods of the cost function, gradient, and Hessian matrix of the multi - output LFFR algorithm. 4. **Improved LFFR**: Solve the limitations of the original LFFR algorithm by introducing a new normalization mapping and improve the prediction accuracy. ### Security and Application Scenarios 1. **Database encoding method**: Describe how to encrypt training data and labels, as well as the initial weight matrix. 2. **Usage scenarios**: Applicable to secure calculations between data owners (such as hospitals or individuals) and cloud service providers (such as Amazon, Google, or Microsoft). 3. **Complete process**: Explain in detail the entire process from data preparation to finally obtaining encrypted weights, ensuring that the data is always in an encrypted state throughout the process. In short, this paper aims to develop a new algorithm that can both protect data privacy and efficiently handle multi - output regression problems, which has important theoretical and practical significance.

LFFR: Logistic Function For (multi-output) Regression

LFFR: Logistic Function For (single-output) Regression

When Homomorphic Encryption Marries Secret Sharing: Secure Large-Scale Sparse Logistic Regression and Applications in Risk Control.

Privacy-Preserving Logistic Regression Training on Large Datasets

Lancelot: Towards Efficient and Privacy-Preserving Byzantine-Robust Federated Learning within Fully Homomorphic Encryption

Privacy-enhanced and non-interactive linear regression with dropout-resilience

Supporting Regularized Logistic Regression Privately and Efficiently

A flexible and privacy-preserving federated learning framework based on logistic regression

Privacy-preserving logistic regression with secret sharing

PrivLogit: Efficient Privacy-preserving Logistic Regression by Tailoring Numerical Optimizers

Privacy-preserving multi-party logistic regression in cloud computing

Logistic regression model training based on the approximate homomorphic encryption

Privacy-preserving semi-parallel logistic regression training with fully homomorphic encryption

Efficient Privacy Preserving Logistic Regression for Horizontally Distributed Data

Practical and Robust Federated Learning With Highly Scalable Regression Training

VFLR: An Efficient and Privacy-Preserving Vertical Federated Framework for Logistic Regression

Online Efficient Secure Logistic Regression based on Function Secret Sharing

Local differentially private federated learning with homomorphic encryption

High Performance Logistic Regression for Privacy-Preserving Genome Analysis

Efficient Sparse Least Absolute Deviation Regression with Differential Privacy