Abstract:Image style transfer aims to imbue digital imagery with the distinctive attributes of style targets, such as colors, brushstrokes, shapes, whilst concurrently preserving the semantic integrity of the content. Despite the advancements in arbitrary style transfer methods, a prevalent challenge remains the delicate equilibrium between content semantics and style attributes. Recent developments in large-scale text-to-image diffusion models have heralded unprecedented synthesis capabilities, albeit at the expense of relying on extensive and often imprecise textual descriptions to delineate artistic styles. Addressing these limitations, this paper introduces DiffStyler, a novel approach that facilitates efficient and precise arbitrary image style transfer. DiffStyler lies the utilization of a text-to-image Stable Diffusion model-based LoRA to encapsulate the essence of style targets. This approach, coupled with strategic cross-LoRA feature and attention injection, guides the style transfer process. The foundation of our methodology is rooted in the observation that LoRA maintains the spatial feature consistency of UNet, a discovery that further inspired the development of a mask-wise style transfer technique. This technique employs masks extracted through a pre-trained FastSAM model, utilizing mask prompts to facilitate feature fusion during the denoising process, thereby enabling localized style transfer that preserves the original image's unaffected regions. Moreover, our approach accommodates multiple style targets through the use of corresponding masks. Through extensive experimentation, we demonstrate that DiffStyler surpasses previous methods in achieving a more harmonious balance between content preservation and style integration.

Image Stylization by Non-linear Filtering and Texture Transfer

Learning Structure-Aware Transformations for Arbitrary Image Style Transfer

Diverse Image Style Transfer Via Invertible Cross-Space Mapping

Improved method of image color transfer.

Artistic Style Transfer with Internal-external Learning and Contrastive Learning

Homogenous Color Transfer Using Texture Retrieval and Matching.

L0 Gradient-Preserving Color Transfer

Optimal Transport of Deep Feature for Image Style Transfer

PTGCF: Printing Texture Guided Color Fusion for Impressionism Oil Painting Style Rendering

Color and Texture Dual Pipeline Lightweight Style Transfer

A Common Framework for Interactive Texture Transfer

DiffStyler: Diffusion-based Localized Image Style Transfer

Name Your Style: An Arbitrary Artist-aware Image Style Transfer

Controllable style transfer of ink painting textures via contrastive learning

Text Style Transfer with Decorative Elements

Style Transfer by Relaxed Optimal Transport and Self-Similarity

Real-time Photo Style Transfer

Aesthetic-Aware Image Style Transfer.

Region-controlled Style Transfer

Style Transfer Based on Style Primitive Discovery.

Lightweight texture transfer based on texture feature preset