Abstract:Architects are used to hand drawing sketches to express the architectural creation intention.To present these abstract sketches, architects and teams need to convert sketches into architectural scheme images, which requires a lot of time and labour.Deep learning may have the potential to improve the efficiency of this work.The common sketch-to-image generation is based on Generative Adversarial Network (GAN), and the research of edge-to-image has made a big progress.But these methods require strict alignment of data pairs, which is difficult to achieve.Zhu et al. proposed the loss of Cycle-Consistent, which solved the problem that pairs of data sets are difficult to collect.However, most of the image translation methods require strict alignment between image data pairs, which can be achieved only for the edge mapping extracted from the image; but the sketch is very different from the edge.Due to the abstractness and fuzziness of the sketch, any simple distortion cannot complete the task of providing pixel-level alignment between the sketch and image; And image translation is the transfer of image features such as colour and texture.The original image has a strong constraint on the generated image, which makes the original structure of the image impossible to be changed.By image inpainting, we address this topic using a joint image completion approach with Context-Encoder, where the architectural sketch provides the image context for generating the scheme images.This setting has two advantages: first, the joint images can avoid the complexity of cross modal problems and the strict alignment of the data pairs as image-to-image translation; second, because of the weak constraint, the outputs have greater freedom, which perhaps can generate more imaginative results.The Context-Encoder generates scheme images on the data sets of general architectural sketches.The results present that the applicability of the completion method is better than that of the method of image translation.And scheme images that is different from the original architectural sketch contours have been generated.

Research on Architectural Sketch to Scheme Image Based on Context Encoder

Learn, Imagine and Create: Text-to-Image Generation from Prior Knowledge.

Sketch to Building: Architecture Image Translation Based on GAN

Sketchformer++: A Hierarchical Transformer Architecture for Vector Sketch Representation

Mural Sketch Generation via Style-aware Convolutional Neural Network.

Sketch-Guided Scenery Image Outpainting

Coloring and fusing architectural sketches by combining a Y‐shaped generative adversarial network and a denoising diffusion implicit model

Image Generation from Sketch Constraint Using Contextual GAN

Picture that Sketch: Photorealistic Image Generation from Abstract Sketches

A self‐sparse generative adversarial network for autonomous early‐stage design of architectural sketches

DRAWING WITH BOTS: HUMAN-COMPUTER COL-LABORATIVE DRAWING EXPERIMENTS

Sketch-Guided Scene Image Generation

Enhancing Urban Landscape Design: A GAN-Based Approach for Rapid Color Rendering of Park Sketches

Transforming Sketches into Realistic Images: Leveraging Machine Learning and Image Processing for Enhanced Architectural Visualization

Learning to Sketch with Shortcut Cycle Consistency

Contour-enhanced CycleGAN framework for style transfer from scenery photos to Chinese landscape paintings

Staged Sketch-to-Image Synthesis via Semi-supervised Generative Adversarial Networks

DiffSketching: Sketch Control Image Synthesis with Diffusion Models

A Deep Learning Model-Based Feature Extraction Method for Architectural Space and Color Connection in Interior Design

SketchyCOCO: Image Generation from Freehand Scene Sketches

RSCAN: Residual Spatial Cross-Attention Network for High-Fidelity Architectural Image Editing by Fusing Multi-Latent Spaces