Code generation from a graphical user interface via attention-based encoder–decoder model

Wen-Yin Chen,Pavol Podstreleny,Wen-Huang Cheng,Yung-Yao Chen,Kai-Lung Hua
DOI: https://doi.org/10.1007/s00530-021-00804-7
IF: 3.9
2021-05-18
Multimedia Systems
Abstract:Code generation from graphical user interface images is a promising area of research. Recent progress on machine learning methods made it possible to transform user interface into the code using several methods. The encoder–decoder framework represents one of the possible ways to tackle code generation tasks. Our model implements the encoder–decoder framework with an attention mechanism that helps the decoder to focus on a subset of salient image features when needed. Our attention mechanism also helps the decoder to generate token sequences with higher accuracy. Experimental results show that our model outperforms previously proposed models on the pix2code benchmark dataset.
computer science, information systems, theory & methods
What problem does this paper attempt to address?