Learning to Color from Language

Varun Manjunatha,Mohit Iyyer,Jordan Boyd-Graber,Larry Davis
DOI: https://doi.org/10.48550/arXiv.1804.06026
2018-04-17
Abstract:Automatic colorization is the process of adding color to greyscale images. We condition this process on language, allowing end users to manipulate a colorized image by feeding in different captions. We present two different architectures for language-conditioned colorization, both of which produce more accurate and plausible colorizations than a language-agnostic version. Through this language-based framework, we can dramatically alter colorizations by manipulating descriptive color words in captions.
Computer Vision and Pattern Recognition,Computation and Language
What problem does this paper attempt to address?