Abstract:Expansive Matching of Experts (EMOE) is a novel method that utilizes support-expanding, extrapolatory pseudo-labeling to improve prediction and uncertainty based rejection on out-of-distribution (OOD) points. We propose an expansive data augmentation technique that generates OOD instances in a latent space, and an empirical trial based approach to filter out augmented expansive points for pseudo-labeling. EMOE utilizes a diverse set of multiple base experts as pseudo-labelers on the augmented data to improve OOD performance through a shared MLP with multiple heads (one per expert). We demonstrate that EMOE achieves superior performance compared to state-of-the-art methods on tabular data.

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is to improve the performance and reliability of machine - learning models when dealing with **out - of - distribution (OOD) data**. Specifically, the author proposes a new method - **EMOE (Expansive Matching of Experts)**, aiming to improve the model's prediction of OOD data and the ability to reject based on uncertainty in the following ways: 1. **Generate out - of - distribution instances**: Generate more OOD instances by expanding the support range of the training data in the latent space. 2. **Pseudo - label filtering**: Use the method of empirical experiments to screen out the amplification points for pseudo - labels to ensure the quality of these points. 3. **Multi - expert matching network**: Use multiple basic expert models to pseudo - label the amplified data and match the labels of these experts through a shared multi - head neural network (MLP), thereby improving the performance on OOD data. ### Specific problem description In real - world applications, especially in fields such as drug discovery, the model may encounter new data with a different distribution from the training data. Traditional novelty - based rejection strategies are too conservative and will reject all predictions that are outside the support range of the training data, even if the model may be capable of making reliable predictions. This strategy limits the application of the model in tasks such as exploring new molecular structures, resulting in waste of resources and inefficiency. To solve this problem, EMOE proposes a more flexible and effective solution, that is, allowing the model to predict external distribution data while maintaining a certain level of confidence. Specifically, EMOE achieves this through the following steps: - **Data amplification**: Generate new instances that are outside the original data distribution range in the latent space. - **Pseudo - label generation and filtering**: Use multiple basic expert models to pseudo - label the amplified data and screen out high - quality pseudo - labels through experiments. - **Multi - head network training**: Match these pseudo - labels through a shared multi - head neural network, thereby improving the model's prediction ability for OOD data. ### Main contributions of the paper 1. **Propose a new data amplification technique**: Generate OOD instances in the latent space. 2. **Propose a pseudo - label screening method based on empirical experiments**: Ensure the quality of the amplification points. 3. **Develop a powerful and diverse set of basic experts**: For self - training. 4. **Design a novel EMOE method**: For training multi - head networks. 5. **Demonstrate performance superior to existing methods on OOD data**: Especially excellent performance on metrics such as AUPRC. Through these improvements, EMOE can better handle OOD data while maintaining high confidence, thereby improving the reliability and practicality of the model in practical applications.

EMOE: Expansive Matching of Experts for Robust Uncertainty Based Rejection

Uncertainty Voting Ensemble for Imbalanced Deep Regression

Training of Neural Networks with Uncertain Data: A Mixture of Experts Approach

Mitigating Overconfidence in Out-of-Distribution Detection by Capturing Extreme Activations

Split-Ensemble: Efficient OOD-aware Ensemble via Task and Model Splitting

ECMEE: Expert Constrained Multi-Expert Ensembles with Category Entropy Minimization for long-tailed visual recognition

Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detection

Learning Mixtures of Experts with EM

The Peril of Popular Deep Learning Uncertainty Estimation Methods

Boosting Out-of-Distribution Image Detection With Epistemic Uncertainty

Learning Multi-Manifold Embedding for Out-Of-Distribution Detection

DORO: Distributional and Outlier Robust Optimization

Merging Experts into One: Improving Computational Efficiency of Mixture of Experts

Logit-Based Ensemble Distribution Distillation for Robust Autoregressive Sequence Uncertainties

Dissecting Out-of-Distribution Detection and Open-Set Recognition: A Critical Analysis of Methods and Benchmarks

Self-Supervised Mixture-of-Experts by Uncertainty Estimation

Expert-Token Resonance: Redefining MoE Routing through Affinity-Driven Active Selection

A Likelihood Ratio-Based Approach to Segmenting Unknown Objects

Exploring Diverse Representations for Open Set Recognition

Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection