Abstract:Text generation commonly relies on greedy and beam decoding that limit the search space and degrade output quality. Minimum Bayes Risk (MBR) decoding can mitigate this problem by utilizing automatic evaluation metrics and model-generated pseudo-references. Previous studies have conducted empirical analyses to reveal the improvement by MBR decoding, and reported various observations. However, despite these observations, the theoretical relationship between them remains uncertain. To address this, we present a novel theoretical interpretation of MBR decoding from the perspective of bias-diversity decomposition. We decompose errors in the estimated quality of generated hypotheses in MBR decoding into two key factors: bias, which reflects the closeness between utility functions and human evaluations, and diversity, which represents the variation in the estimated quality of utility functions. Our theoretical analysis reveals the difficulty in simultaneously improving both bias and diversity, and highlights the effectiveness of increasing diversity to enhance MBR decoding performance. This analysis verifies the alignment between our theoretical insights and the empirical results reported in previous work. Furthermore, to support our theoretical findings, we propose a new metric, pseudo-bias, which approximates the bias term using gold references. We also introduce a new MBR approach, Metric-augmented MBR (MAMBR), which increases diversity by adjusting the behavior of utility functions without altering the pseudo-references. Experimental results across multiple NLP tasks show that the decomposed terms in the bias-diversity decomposition correlate well with performance, and that MAMBR improves text generation quality by modifying utility function behavior. Our code will be available at <a class="link-external link-https" href="https://github.com/naist-nlp/mbr-bias-diversity" rel="external noopener nofollow">this https URL</a>.

DC-MBR: Distributional Cooling for Minimum Bayesian Risk Decoding

Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translation

RMBR: A Regularized Minimum Bayes Risk Reranking Framework for Machine Translation

Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation

Unveiling the Power of Source: Source-based Minimum Bayes Risk Decoding for Neural Machine Translation

Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding

Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decoding

MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methods

It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk

Focus on the Target's Vocabulary: Masked Label Smoothing for Machine Translation.

On the True Distribution Approximation of Minimum Bayes-Risk Decoding

mbrs: A Library for Minimum Bayes Risk Decoding

Efficient Minimum Bayes Risk Decoding using Low-Rank Matrix Completion Algorithms

Theoretical Aspects of Bias and Diversity in Minimum Bayes Risk Decoding

Model-Based Minimum Bayes Risk Decoding for Text Generation

Rethinking Label Smoothing on Multi-Hop Question Answering

Chasing COMET: Leveraging Minimum Bayes Risk Decoding for Self-Improving Machine Translation

Centroid-Based Efficient Minimum Bayes Risk Decoding

Faster Minimum Bayes Risk Decoding with Confidence-based Pruning

Mitigating Metric Bias in Minimum Bayes Risk Decoding

Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decoding