Abstract:High-quality training triplets (instruction, original image, edited image) are essential for instruction-based image editing. Predominant training datasets (e.g., InsPix2Pix) are created using text-to-image generative models (e.g., Stable Diffusion, DALL-E) which are not trained for image editing. Accordingly, these datasets suffer from inaccurate instruction following, poor detail preserving, and generation artifacts. In this paper, we propose to address the training data quality issue with multi-perspective reward data instead of refining the ground-truth image quality. 1) we first design a quantitative metric system based on best-in-class LVLM (Large Vision Language Model), i.e., GPT-4o in our case, to evaluate the generation quality from 3 perspectives, namely, instruction following, detail preserving, and generation quality. For each perspective, we collected quantitative score in $0\sim 5$ and text descriptive feedback on the specific failure points in ground-truth edited images, resulting in a high-quality editing reward dataset, i.e., RewardEdit20K. 2) We further proposed a novel training framework to seamlessly integrate the metric output, regarded as multi-reward, into editing models to learn from the imperfect training triplets. During training, the reward scores and text descriptions are encoded as embeddings and fed into both the latent space and the U-Net of the editing models as auxiliary conditions. During inference, we set these additional conditions to the highest score with no text description for failure points, to aim at the best generation outcome. Experiments indicate that our multi-reward conditioned model outperforms its no-reward counterpart on two popular editing pipelines, i.e., InsPix2Pix and SmartEdit. The code and dataset will be released.

InstructRL4Pix: Training Diffusion for Image Editing by Reinforcement Learning

InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructions

Instruct Pix-to-3D: Instructional 3D Object Generation from a Single Image

InstructPix2NeRF: Instructed 3D Portrait Editing from a Single Image

Multi-Reward as Condition for Instruction-based Image Editing

Learning to Follow Object-Centric Image Editing Instructions Faithfully

Learning from Mistakes: Iterative Prompt Relabeling for Text-to-Image Diffusion Model Training

Large-scale Reinforcement Learning for Diffusion Models

Diffusion-Based Conditional Image Editing through Optimized Inference with Guidance

InsightEdit: Towards Better Instruction Following for Image Editing

PRedItOR: Text Guided Image Editing with Diffusion Prior

ColorEdit: Training-free Image-Guided Color editing with diffusion model

Focus on Your Instruction: Fine-grained and Multi-instruction Image Editing by Attention Modulation

Image Inpainting Models are Effective Tools for Instruction-guided Image Editing

Paint by Example: Exemplar-based Image Editing with Diffusion Models

LayerDiffusion: Layered Controlled Image Editing with Diffusion Models

Training Diffusion Models with Reinforcement Learning

EditWorld: Simulating World Dynamics for Instruction-Following Image Editing

PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preference

DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models

Diffusion Model-Based Image Editing: A Survey