Abstract:While text-to-image models like DALLE-3 and Stable Diffusion are rapidly proliferating, they often encounter challenges such as hallucination, bias, and the production of unsafe, low-quality output. To effectively address these issues, it is crucial to align these models with desired behaviors based on feedback from a multimodal judge. Despite their significance, current multimodal judges frequently undergo inadequate evaluation of their capabilities and limitations, potentially leading to misalignment and unsafe fine-tuning outcomes. To address this issue, we introduce MJ-Bench, a novel benchmark which incorporates a comprehensive preference dataset to evaluate multimodal judges in providing feedback for image generation models across four key perspectives: alignment, safety, image quality, and bias. Specifically, we evaluate a large variety of multimodal judges including smaller-sized CLIP-based scoring models, open-source VLMs (e.g. LLaVA family), and close-source VLMs (e.g. GPT-4o, Claude 3) on each decomposed subcategory of our preference dataset. Experiments reveal that close-source VLMs generally provide better feedback, with GPT-4o outperforming other judges in average. Compared with open-source VLMs, smaller-sized scoring models can provide better feedback regarding text-image alignment and image quality, while VLMs provide more accurate feedback regarding safety and generation bias due to their stronger reasoning capabilities. Further studies in feedback scale reveal that VLM judges can generally provide more accurate and stable feedback in natural language (Likert-scale) than numerical scales. Notably, human evaluations on end-to-end fine-tuned models using separate feedback from these multimodal judges provide similar conclusions, further confirming the effectiveness of MJ-Bench. All data, code, models are available at <a class="link-external link-https" href="https://huggingface.co/MJ-Bench" rel="external noopener nofollow">this https URL</a>.

DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation

FaceChain: A Playground for Identity-Preserving Portrait Generation

Multi-Dimensional Insights: Benchmarking Real-World Personalization in Large Multimodal Models

IDEA-Bench: How Far are Generative Models from Professional Designing?

DEsignBench: Exploring and Benchmarking DALL-E 3 for Imagining Visual Design

DreamEdit: Subject-driven Image Editing

A-Bench: Are LMMs Masters at Evaluating AI-generated Images?

Imagine yourself: Tuning-Free Personalized Image Generation

DreamBlend: Advancing Personalized Fine-tuning of Text-to-Image Diffusion Models

MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?

GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation

F-Bench: Rethinking Human Preference Evaluation Metrics for Benchmarking Face Generation, Customization, and Restoration

DreamArtist: Towards Controllable One-Shot Text-to-Image Generation via Positive-Negative Prompt-Tuning

CharacterBench: Benchmarking Character Customization of Large Language Models

Imaginique Expressions: Tailoring Personalized Short-Text-to-Image Generation Through Aesthetic Assessment and Human Insights

BIGbench: A Unified Benchmark for Social Bias in Text-to-Image Generative Models Based on Multi-modal LLM

DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation

AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generation

HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models

VBench: Comprehensive Benchmark Suite for Video Generative Models

Personalized Image Generation with Large Multimodal Models