Multivariate Gaussian and Student$-t$ Process Regression for Multi-output Prediction

Zexun Chen,Bo Wang,Alexander N. Gorban

DOI: https://doi.org/10.1007/s00521-019-04687-8

2019-01-07

Abstract:Gaussian process model for vector-valued function has been shown to be useful for multi-output prediction. The existing method for this model is to re-formulate the matrix-variate Gaussian distribution as a multivariate normal distribution. Although it is effective in many cases, re-formulation is not always workable and is difficult to apply to other distributions because not all matrix-variate distributions can be transformed to respective multivariate distributions, such as the case for matrix-variate Student$-t$ distribution. In this paper, we propose a unified framework which is used not only to introduce a novel multivariate Student$-t$ process regression model (MV-TPR) for multi-output prediction, but also to reformulate the multivariate Gaussian process regression (MV-GPR) that overcomes some limitations of the existing methods. Both MV-GPR and MV-TPR have closed-form expressions for the marginal likelihoods and predictive distributions under this unified framework and thus can adopt the same optimization approaches as used in the conventional GPR. The usefulness of the proposed methods is illustrated through several simulated and real data examples. In particular, we verify empirically that MV-TPR has superiority for the datasets considered, including air quality prediction and bike rent prediction. At last, the proposed methods are shown to produce profitable investment strategies in the stock markets.

Machine Learning

What problem does this paper attempt to address?

This paper attempts to address some limitations of existing Gaussian Process Regression (GPR) methods in multi-output prediction. Specifically:

1. **Limitations of existing methods**:
- Existing multi-output GPR methods typically require vectorizing the multi-output matrix to utilize the GPR model. This approach becomes complex and difficult to implement when the number of outputs and observations is large.
- Existing methods rely on the equivalence between the vectorized matrix-variate Gaussian distribution and the multivariate Gaussian distribution, but this equivalence does not exist in other elliptical distributions (such as the matrix-variate Student-t distribution).

2. **Proposed new framework**:
- The paper proposes a unified framework for introducing a new multivariate Student-t process regression model (MV-TPR) and reformulating the multivariate Gaussian process regression model (MV-GPR).
- This framework not only overcomes the limitations of existing methods but also allows for the derivation of more general elliptical process regression models.

3. **Objectives**:
- Provide a more direct and efficient approach to handling multi-output prediction problems, especially when dealing with non-Gaussian noise and small sample data.
- Demonstrate through experiments that the newly proposed MV-TPR model outperforms existing multi-output GPR models and other independent modeling methods in terms of prediction performance.

Overall, this paper aims to improve Gaussian process regression methods in multi-output prediction by proposing a new unified framework and introducing a more flexible multivariate Student-t process regression model to enhance prediction accuracy and robustness.