Abstract:Text generation commonly relies on greedy and beam decoding that limit the search space and degrade output quality. Minimum Bayes Risk (MBR) decoding can mitigate this problem by utilizing automatic evaluation metrics and model-generated pseudo-references. Previous studies have conducted empirical analyses to reveal the improvement by MBR decoding, and reported various observations. However, despite these observations, the theoretical relationship between them remains uncertain. To address this, we present a novel theoretical interpretation of MBR decoding from the perspective of bias-diversity decomposition. We decompose errors in the estimated quality of generated hypotheses in MBR decoding into two key factors: bias, which reflects the closeness between utility functions and human evaluations, and diversity, which represents the variation in the estimated quality of utility functions. Our theoretical analysis reveals the difficulty in simultaneously improving both bias and diversity, and highlights the effectiveness of increasing diversity to enhance MBR decoding performance. This analysis verifies the alignment between our theoretical insights and the empirical results reported in previous work. Furthermore, to support our theoretical findings, we propose a new metric, pseudo-bias, which approximates the bias term using gold references. We also introduce a new MBR approach, Metric-augmented MBR (MAMBR), which increases diversity by adjusting the behavior of utility functions without altering the pseudo-references. Experimental results across multiple NLP tasks show that the decomposed terms in the bias-diversity decomposition correlate well with performance, and that MAMBR improves text generation quality by modifying utility function behavior. Our code will be available at <a class="link-external link-https" href="https://github.com/naist-nlp/mbr-bias-diversity" rel="external noopener nofollow">this https URL</a>.

Model-Based Minimum Bayes Risk Decoding for Text Generation

mbrs: A Library for Minimum Bayes Risk Decoding

Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decoding

Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decoding

Theoretical Aspects of Bias and Diversity in Minimum Bayes Risk Decoding

On the True Distribution Approximation of Minimum Bayes-Risk Decoding

Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translation

It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk

Faster Minimum Bayes Risk Decoding with Confidence-based Pruning

Linear-time Minimum Bayes Risk Decoding with Reference Aggregation

Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation

Unveiling the Power of Source: Source-based Minimum Bayes Risk Decoding for Neural Machine Translation

Centroid-Based Efficient Minimum Bayes Risk Decoding

High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metrics

Efficient Minimum Bayes Risk Decoding using Low-Rank Matrix Completion Algorithms

Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding

Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translation

RMBR: A Regularized Minimum Bayes Risk Reranking Framework for Machine Translation

Improving Minimum Bayes Risk Decoding with Multi-Prompt

Minimum Bayes' Risk Decoding for System Combination of Grammatical Error Correction Systems

A Simple, Fast Diverse Decoding Algorithm for Neural Generation