Abstract:Distributed sensors in the internet-of-things (IoT) generate vast amounts of sparse data. Analyzing this high-dimensional data and identifying relevant predictors pose substantial challenges, especially when data is preferred to remain on the device where it was collected for reasons such as data integrity, communication bandwidth, and privacy. This paper introduces a federated quantile regression algorithm to address these challenges. Quantile regression provides a more comprehensive view of the relationship between variables than mean regression models. However, traditional approaches face difficulties when dealing with nonconvex sparse penalties and the inherent non-smoothness of the loss function. For this purpose, we propose a federated smoothing proximal gradient (FSPG) algorithm that integrates a smoothing mechanism with the proximal gradient framework, thereby enhancing both precision and computational speed. This integration adeptly handles optimization over a network of devices, each holding local data samples, making it particularly effective in federated learning scenarios. The FSPG algorithm ensures steady progress and reliable convergence in each iteration by maintaining or reducing the value of the objective function. By leveraging nonconvex penalties, such as the minimax concave penalty (MCP) and smoothly clipped absolute deviation (SCAD), the proposed method can identify and preserve key predictors within sparse models. Comprehensive simulations validate the robust theoretical foundations of the proposed algorithm and demonstrate improved estimation precision and reliable convergence.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to conduct effective sparse quantized regression analysis in the Internet of Things (IoT) environment when data is distributed across multiple devices and the original data is not shared. Specifically, the paper addresses the following challenges: 1. **Non - convex and non - smooth penalty terms**: Traditional quantized regression methods have difficulties in dealing with non - convex and non - smooth penalty terms (such as Minimax Concave Penalty (MCP) and Smoothly Clipped Absolute Deviation (SCAD)). These penalty terms are helpful for reducing the bias of the model and improving sparsity, but the optimization process is complex and prone to getting trapped in local optima. 2. **Optimization problems in federated learning**: In the federated learning framework, data is distributed across multiple devices, and each device can only access local data. Therefore, an optimization algorithm that can work effectively in a distributed environment while maintaining data privacy and communication efficiency is required. 3. **Model accuracy and convergence speed**: Existing optimization methods often lead to a decline in model accuracy and slow convergence speed when dealing with non - convex and non - smooth problems. Therefore, a method that can improve model accuracy and accelerate the convergence speed is needed. To solve the above problems, the paper proposes the Federated Smoothed Proximal Gradient (FSPG) algorithm. The FSPG algorithm deals with non - smooth loss functions by introducing a smoothing mechanism and combines with the proximal gradient framework, thereby optimizing efficiently on distributed devices. The main features of this algorithm include: - **Smoothing mechanism**: Through smoothing techniques, the non - smooth quantized regression loss function is transformed into a series of smooth approximate functions, thus simplifying the optimization process. - **Proximal gradient framework**: Combined with the proximal gradient method, by adding a time - varying penalty parameter to control the optimization process, it is ensured that the value of the objective function can be monotonically improved in each iteration. - **Federated learning adaptability**: The algorithm is designed to be suitable for the federated learning environment and can achieve the optimization of the global model through local updates on distributed devices without sharing the original data. The paper verifies the effectiveness of the FSPG algorithm through theoretical analysis and a large number of experiments, demonstrating its significant advantages in terms of model accuracy and convergence speed.

Federated Smoothing Proximal Gradient for Quantile Regression with Non-Convex Penalties

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Decentralized Smoothing ADMM for Quantile Regression with Non-Convex Sparse Penalties

Lazily Aggregated Quantized Gradient Innovation for Communication-Efficient Federated Learning.

A Non-parametric View of FedAvg and FedProx: Beyond Stationary Points

Distributed Quantile Regression over Sensor Networks

FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization

Federated Optimization in Heterogeneous Networks

Communication-Efficient Federated Learning via Quantized Compressed Sensing

Adaptive Fixed-Time Proximal Gradient Method for Non-Smooth Optimization: the Fractional Approach

Tighter Performance Theory of FedExProx

A Smoothing Proximal Gradient Algorithm for Nonsmooth Convex Regression with Cardinality Penalty.

Exact Penalty Method for Federated Learning

Rate distortion optimization for adaptive gradient quantization in federated learning

A Federated Learning Framework for Nonconvex-PL Minimax Problems

FedSQ: Sparse-Quantized Federated Learning for Communication Efficiency

Achieving Statistical Optimality of Federated Learning: Beyond Stationary Points

Optimizing Quantum Federated Learning Based on Federated Quantum Natural Gradient Descent

Flexible Model Aggregation for Quantile Regression

Residual projection for quantile regression in vertically partitioned big data

Communication Efficient Federated Learning with Adaptive Quantization