Abstract:Multioutput regression, referring to simultaneously predicting multiple continuous output variables with a single model, has drawn increasing attention in the machine learning community due to its strong ability to capture the correlations among multioutput variables. The methodology of output space embedding, built upon the low-rank assumption, is now the mainstream for multioutput regression since it can effectively reduce the parameter numbers while achieving effective performance. The existing low-rank methods, however, are sensitive to the noises of both inputs and outputs, referring to the noise problem. In this article, we develop a novel multioutput regression method by simultaneously alleviating input and output noises, namely, robust multioutput regression by alleviating input and output noises (RMoR-Aion), where both the noises of the input and output are exploited by leveraging auxiliary matrices. Furthermore, we propose a prediction output manifold constraint with the correlation information regarding the output variables to further reduce the adversarial effects of the noise. Our empirical studies demonstrate the effectiveness of RMoR-Aion compared with the state-of-the-art baseline methods, and RMoR-Aion is more stable in the settings with artificial noise.

What problem does this paper attempt to address?

This paper attempts to address the noise problem in multi-output regression tasks, particularly the noise present in both input and output data. Existing low-rank methods, while effective in reducing the number of parameters and achieving good performance, are highly sensitive to input and output noise, which is a significant challenge in practical applications. To solve this problem, the authors propose a new multi-output regression method—Robust Multi-output Regression with Alleviating Input and Output Noise (RMoR-A ION). Specifically, RMoR-A ION utilizes auxiliary matrices to capture noise in both input and output data and introduces output manifold regularization to further reduce the impact of noise. ### Main Contributions: 1. **Alleviating Input and Output Noise Simultaneously**: A new method RMoR-A ION is proposed, which captures noise in both input and output data by utilizing auxiliary matrices. 2. **Output Manifold Regularization**: Output manifold regularization is introduced to constrain the correlation between predicted output variables, thereby indirectly filtering out noise. 3. **Experimental Validation**: The effectiveness and stability of RMoR-A ION are validated through experiments on synthetic and real-world datasets, especially in the presence of artificial noise. ### Method Overview: - **Input Noise**: An auxiliary matrix \( F \) is introduced to capture noise in the input data, describing the unobservable latent features of the training inputs. - **Output Noise**: Another auxiliary matrix \( R \) is introduced to capture noise in the output data, used to model the noise response of the outputs. - **Output Manifold Regularization**: A regularization term \( R_m(U, V, F) \) is introduced to constrain the correlation between predicted output variables, making them closer in cases of high correlation. ### Experimental Results: - Experimental results on synthetic and real-world datasets show that RMoR-A ION outperforms existing baseline methods in prediction performance and is more stable in the presence of artificial noise. In summary, this paper effectively addresses the problem of input and output noise in multi-output regression tasks by proposing the RMoR-A ION method, improving the robustness and prediction performance of the model.

RMoR-Aion: Robust Multioutput Regression by Simultaneously Alleviating Input and Output Noises

Multi-Target Regression Via Robust Low-Rank Learning.

Multiple Output Regression with Latent Noise

Multi-output Regression on the Output Manifold

MORStreaming: A Multioutput Regression System for Streaming Data

Low-Rank Matrix Factorization under General Mixture Noise Distributions

Robust-MSA: Understanding the Impact of Modality Noise on Multimodal Sentiment Analysis

Low-rank Hypergraph Feature Selection for Multi-Output Regression

Robust sparse regression by modeling noise as a mixture of gaussians

Solving Sparse \& High-Dimensional-Output Regression via Compression

U2++ MoE: Scaling 4.7x parameters with minimal impact on RTF

A Survey on Multi-output Learning

Multi-Output Bayesian Support Vector Regression Considering Dependent Outputs

Robust noise-aware algorithm for randomized neural network and its convergence properties

Multi-Objective Interpolation Training for Robustness to Label Noise

MoE-AMC: Enhancing Automatic Modulation Classification Performance Using Mixture-of-Experts

RMIO: A Model-Based MARL Framework for Scenarios with Observation Loss in Some Agents

Low-Rank Feature Reduction and Sample Selection for Multi-output Regression.

Modal Regression based Structured Low-rank Matrix Recovery for Multi-view Learning

Regularized Modal Regression on Markov-dependent Observations: A Theoretical Assessment

Double-noise-dual-problem approach to the augmented Lagrange multiplier method for robust principal component analysis