Abstract:While text-to-image models like DALLE-3 and Stable Diffusion are rapidly proliferating, they often encounter challenges such as hallucination, bias, and the production of unsafe, low-quality output. To effectively address these issues, it is crucial to align these models with desired behaviors based on feedback from a multimodal judge. Despite their significance, current multimodal judges frequently undergo inadequate evaluation of their capabilities and limitations, potentially leading to misalignment and unsafe fine-tuning outcomes. To address this issue, we introduce MJ-Bench, a novel benchmark which incorporates a comprehensive preference dataset to evaluate multimodal judges in providing feedback for image generation models across four key perspectives: alignment, safety, image quality, and bias. Specifically, we evaluate a large variety of multimodal judges including smaller-sized CLIP-based scoring models, open-source VLMs (e.g. LLaVA family), and close-source VLMs (e.g. GPT-4o, Claude 3) on each decomposed subcategory of our preference dataset. Experiments reveal that close-source VLMs generally provide better feedback, with GPT-4o outperforming other judges in average. Compared with open-source VLMs, smaller-sized scoring models can provide better feedback regarding text-image alignment and image quality, while VLMs provide more accurate feedback regarding safety and generation bias due to their stronger reasoning capabilities. Further studies in feedback scale reveal that VLM judges can generally provide more accurate and stable feedback in natural language (Likert-scale) than numerical scales. Notably, human evaluations on end-to-end fine-tuned models using separate feedback from these multimodal judges provide similar conclusions, further confirming the effectiveness of MJ-Bench. All data, code, models are available at <a class="link-external link-https" href="https://huggingface.co/MJ-Bench" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are the challenges encountered by existing text - to - image generation models (such as DALLE - 3 and Stable Diffusion) when generating images, including but not limited to: 1. **Text - image alignment problem**: The generated image is inconsistent with the text instruction, that is, although the content generated by the model seems reasonable, it does not match the given text description (for example, the hallucination phenomenon). 2. **Unsafe content**: The generated image contains harmful or inappropriate content, such as violence, pornography or other uncomfortable concepts. 3. **Low - quality generation**: The generated image has blurry or unnatural artifacts, which affect the quality of the image. 4. **Bias and stereotype**: The generated image shows bias towards certain groups of people and may be in favor of or against specific population groups. To solve these problems and ensure that these models can be adjusted according to the desired behavior, researchers introduced a new benchmark called MJ - BENCH. This benchmark aims to comprehensively evaluate the ability of multi - modal reward models as "judges" through four key dimensions (text - image alignment, safety, image quality, and generation bias). Specifically, MJ - BENCH contains a comprehensive preference dataset for evaluating the performance of different types of multi - modal reward models (such as CLIP - based scoring models, open - source visual - language models VLM, and closed - source VLM) in providing feedback. Through this benchmark, researchers hope to better understand the capabilities and limitations of multi - modal reward models and provide valuable insights for improving text - to - image generation models. In addition, the research also explored the effectiveness of different feedback scales (such as numerical scales and Likert scales), and the application effect of these feedbacks in end - to - end fine - tuning.

MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?

MMBench: Is Your Multi-modal Model an All-around Player?

A-Bench: Are LMMs Masters at Evaluating AI-generated Images?

VLRewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models

MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark

II-Bench: An Image Implication Understanding Benchmark for Multimodal Large Language Models

MMGenBench: Evaluating the Limits of LMMs from the Text-to-Image Generation Perspective

MIBench: Evaluating Multimodal Large Language Models over Multiple Images

MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models

Q-Bench+: A Benchmark for Multi-modal Foundation Models on Low-level Vision from Single Images to Pairs

MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria

MMEvalPro: Calibrating Multimodal Benchmarks Towards Trustworthy and Efficient Evaluation

MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs

MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMs

MM-BigBench: Evaluating Multimodal Models on Multimodal Content Comprehension Tasks

Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision

A Survey on Benchmarks of Multimodal Large Language Models

MMR: Evaluating Reading Ability of Large Multimodal Models

MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images

MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs