Abstract:Adjusting the photo color to associate with some design elements is an essential way for a graphic design to effectively deliver its message and make it aesthetically pleasing. However, existing tools and previous works face a dilemma between the ease of use and level of expressiveness. To this end, we introduce an interactive language-based approach for photo recoloring, which provides an intuitive system that can assist both experts and novices on graphic design. Given a graphic design containing a photo that needs to be recolored, our model can predict the source colors and the target regions, and then recolor the target regions with the source colors based on the given language-based instruction. The multi-granularity of the instruction allows diverse user intentions. The proposed novel task faces several unique challenges, including: 1) color accuracy for recoloring with exactly the same color from the target design element as specified by the user; 2) multi-granularity instructions for parsing instructions correctly to generate a specific result or multiple plausible ones; and 3) locality for recoloring in semantically meaningful local regions to preserve original image semantics. To address these challenges, we propose a model called LangRecol with two main components: the language-based source color prediction module and the semantic-palette-based photo recoloring module. We also introduce an approach for generating a synthetic graphic design dataset with instructions to enable model training. We evaluate our model via extensive experiments and user studies. We also discuss several practical applications, showing the effectiveness and practicality of our approach. Code and data for this paper are at: <a class="link-external link-https" href="https://zhenwwang.github.io/langrecol" rel="external noopener nofollow">this https URL</a>.

Color2Vec: Web-based Modeling of Word-color Association with Sociocultural Contexts

Colors $-$Messengers of Concepts: Visual Design Mining for Learning Color Semantics

ColorNetVis: an Interactive Color Network Analysis System for Exploring the Color Composition of Traditional Chinese Painting

Language-based colorization of scene sketches

Culture-inspired Multi-modal Color Palette Generation and Colorization: A Chinese Youth Subculture Case

Automatic Advertising Image Color Design Incorporating a Visual Color Analyzer

Colourful Language: Measuring Word-Colour Associations

Image Search by modality analysis: A study of color semantics

comp-syn: Perceptually Grounded Word Embeddings with Color

Color Maker: a Mixed-Initiative Approach to Creating Accessible Color Maps

Language-based Photo Color Adjustment for Graphic Designs

Image Colorization with an Affective Word.

Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation

Multimodal Color Recommendation in Vector Graphic Documents

A Colour Design Tool Based on Empirical Studies

A systematic investigation of conceptual color associations.

Interactive genetic color matching design of cultural and creative products considering color image and visual aesthetics

Emocolor: An assistant design method for emotional color matching based on semantics and images

Even the Abstract have Colour: Consensus in Word-Colour Associations

C2Ideas: Supporting Creative Interior Color Design Ideation with Large Language Model

Effect of Color Combination on Graphical Symbol Visual Search Efficiency