Epoch-evolving Gaussian Process Guided Learning

Jiabao Cui,Xuewei Li,Bin Li,Hanbin Zhao,Bourahla Omar,Xi Li

2024-03-12

Abstract:In this paper, we propose a novel learning scheme called epoch-evolving Gaussian Process Guided Learning (GPGL), which aims at characterizing the correlation information between the batch-level distribution and the global data distribution. Such correlation information is encoded as context labels and needs renewal every epoch. With the guidance of the context label and ground truth label, GPGL scheme provides a more efficient optimization through updating the model parameters with a triangle consistency loss. Furthermore, our GPGL scheme can be further generalized and naturally applied to the current deep models, outperforming the existing batch-based state-of-the-art models on mainstream datasets (CIFAR-10, CIFAR-100, and Tiny-ImageNet) remarkably.

Machine Learning,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper proposes a novel learning method called Epoch-evolving Gaussian Process Guided Learning (GPGL), aiming to address the characterization problem of correlation information between batch data distribution and global data distribution in deep learning. Traditional deep learning relies on mini-batch stochastic gradient descent algorithm, which leads to the "sawtooth" effect in the optimization process and requires a large number of iterations to fully learn the model. GPGL encodes the global data distribution information into contextual labels by non-parametrically modeling the learning model, and updates these labels with each epoch. It uses Gaussian processes to build a class distribution regression model and dynamically propagates class distribution information from anchor samples to given samples. A triangular consistency loss function is proposed, which combines deep model predictions, contextual labels, and ground truth labels to optimize model parameters in each epoch. Through this approach, GPGL can be applied to existing deep models and demonstrates superior performance on mainstream datasets. In short, the paper aims to solve the balance problem between local batch data distribution and global data distribution in the optimization process of deep learning to improve model learning efficiency and convergence speed.

Epoch-evolving Gaussian Process Guided Learning

Epoch-Evolving Gaussian Process Guided Learning for Classification

UniGrad-FS: Unified Gradient Projection with Flatter Sharpness for Continual Learning

Gaussian processes in inverse reinforcement learning

Large Linear Multi-output Gaussian Process Learning

Modulating Scalable Gaussian Processes for Expressive Statistical Learning

A New Reliable & Parsimonious Learning Strategy Comprising Two Layers of Gaussian Processes, to Address Inhomogeneous Empirical Correlation Structures

Efficient Learning Algorithms for Gaussian Processes

Thoughts on Massively Scalable Gaussian Processes

Image Matting With Deep Gaussian Process

Understanding and comparing scalable Gaussian process regression for big data

Non-stationary and Sparsely-correlated Multi-output Gaussian Process with Spike-and-Slab Prior

Gaussian Process Domain Experts for Model Adaptation in Facial Behavior Analysis

Sparsity-Aware Distributed Learning for Gaussian Processes with Linear Multiple Kernel

Learning in Deep Factor Graphs with Gaussian Belief Propagation

Regularized Multi-Output Gaussian Convolution Process With Domain Adaptation

Memory-Based Dual Gaussian Processes for Sequential Learning

Whom to Trust? Elective Learning for Distributed Gaussian Process Regression

Thin and Deep Gaussian Processes

Iterative Gaussian–Laplacian Pyramid Network for Hyperspectral Image Classification

PixelGaussian: Generalizable 3D Gaussian Reconstruction from Arbitrary Views