Abstract:Recently, how to achieve precise image editing has attracted increasing attention, especially given the remarkable success of text-to-image generation models. To unify various spatial-aware image editing abilities into one framework, we adopt the concept of layers from the design domain to manipulate objects flexibly with various operations. The key insight is to transform the spatial-aware image editing task into a combination of two sub-tasks: multi-layered latent decomposition and multi-layered latent fusion. First, we segment the latent representations of the source images into multiple layers, which include several object layers and one incomplete background layer that necessitates reliable inpainting. To avoid extra tuning, we further explore the inner inpainting ability within the self-attention mechanism. We introduce a key-masking self-attention scheme that can propagate the surrounding context information into the masked region while mitigating its impact on the regions outside the mask. Second, we propose an instruction-guided latent fusion that pastes the multi-layered latent representations onto a canvas latent. We also introduce an artifact suppression scheme in the latent space to enhance the inpainting quality. Due to the inherent modular advantages of such multi-layered representations, we can achieve accurate image editing, and we demonstrate that our approach consistently surpasses the latest spatial editing methods, including Self-Guidance and DiffEditor. Last, we show that our approach is a unified framework that supports various accurate image editing tasks on more than six different editing tasks.

Edit Propagation for Natural Image Using Matting Affinity

Easy Matting - A Stroke Based Approach for Continuous Image Matting

Fast Edit Propagation for 360 Degree Panoramas Using Function Interpolation

Efficient Affinity-Based Edit Propagation Using K-D Tree

Edit propagation via color palettes

Instant Propagation of Sparse Edits on Images and Videos.

Efficient Antialiased Edit Propagation for Images and Videos

Efficient Manifold Preserving Edit Propagation with Adaptive Neighborhood Size

DesignEdit: Multi-Layered Latent Decomposition and Fusion for Unified & Accurate Image Editing

Energy Matting

A Survey on Natural Image Matting with Closed-Form Solutions.

Inversion-Free Image Editing with Natural Language

MagicProp: Diffusion-based Video Editing via Motion-aware Appearance Propagation

Move and Act: Enhanced Object Manipulation and Background Integrity for Image Editing

Pathways on the Image Manifold: Image Editing via Video Generation

MAG-Edit: Localized Image Editing in Complex Scenarios via Mask-Based Attention-Adjusted Guidance

A Visual Representation for Editing Face Images

Streamlining Image Editing with Layered Diffusion Brushes

E4C: Enhance Editability for Text-Based Image Editing by Harnessing Efficient CLIP Guidance

ProxEdit: Improving Tuning-Free Real Image Editing with Proximal Guidance

Vision-guided and Mask-enhanced Adaptive Denoising for Prompt-based Image Editing