Abstract:Text-guided diffusion models have revolutionized image generation and editing, offering exceptional realism and diversity. Specifically, in the context of diffusion-based editing, where a source image is edited according to a target prompt, the process commences by acquiring a noisy latent vector corresponding to the source image via the diffusion model. This vector is subsequently fed into separate source and target diffusion branches for editing. The accuracy of this inversion process significantly impacts the final editing outcome, influencing both essential content preservation of the source image and edit fidelity according to the target prompt. Prior inversion techniques aimed at finding a unified solution in both the source and target diffusion branches. However, our theoretical and empirical analyses reveal that disentangling these branches leads to a distinct separation of responsibilities for preserving essential content and ensuring edit fidelity. Building on this insight, we introduce "Direct Inversion," a novel technique achieving optimal performance of both branches with just three lines of code. To assess image editing performance, we present PIE-Bench, an editing benchmark with 700 images showcasing diverse scenes and editing types, accompanied by versatile annotations and comprehensive evaluation metrics. Compared to state-of-the-art optimization-based inversion techniques, our solution not only yields superior performance across 8 editing methods but also achieves nearly an order of speed-up.

A Novel Edit Propagation Algorithm via $$ L_0 $$ Gradient Minimization

Image Inpainting Based on Gradient Weighted Patch Propagation and Color Rectification

Image Copy-and-paste with Optimized Gradient

L0 Gradient-Preserving Color Transfer

L0 image smoothing via iterating truncated L1 gradient regularization

Edit propagation via color palettes

Fast Edit Propagation for 360 Degree Panoramas Using Function Interpolation

Instant Propagation of Sparse Edits on Images and Videos.

A Novel Variational Image Model: Towards a Unified Approach to Image Editing

Robust Edit Propagation Based On Hessian Local Linear Embedding

Edit Propagation for Natural Image Using Matting Affinity

Instant Edit Propagation on Images Based on Bilateral Grid

Image Smoothing Via L 0 Gradient Minimization

A Novel Unified Variational Image Editing Model

ProxEdit: Improving Tuning-Free Real Image Editing with Proximal Guidance

Efficient Manifold Preserving Edit Propagation with Adaptive Neighborhood Size

Efficient Antialiased Edit Propagation for Images and Videos

Image Smoothing Via Gradient Sparsity and Surface Area Minimization

Efficient Affinity-Based Edit Propagation Using K-D Tree

PostEdit: Posterior Sampling for Efficient Zero-Shot Image Editing

Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Code