Real-time portrait image retouching extended from DualBLN
Xiang Zhang,Dawei Yan,Genqing Bian,Chengzhe Lu,Sifei Wang,Ghulam Mohi-ud-din,Qingsen Yan,Wei Dong
DOI: https://doi.org/10.1016/j.eswa.2023.121958
IF: 8.5
2024-03-01
Expert Systems with Applications
Abstract:Towards the target of image retouching tasks, the three-dimension LookUp Table (3D LUT) is a discrete 3D lattice that represents the color mapping to correct the inconsistent color of images produced from varying devices and environments. We design a network named Dual Branch LUT-aware Network (DualBLN), which innovatively introduces the information representing the pixel value mapping in 3D LUT into the real-time retouching procedure, learning the adaptive weights and the multiple 3D LUTs with strong representation capability. Specifically, we elaborate two branches to conduct feature extraction of the raw image and 3D LUTs, which consider the detail of the dual inputs, then generate the precise weights to fuse LUTs for graceful retouching. To integrate the information of the image and the 3D LUTs meticulously, we adopt bilinear pooling to alleviate the conflict between both features when fusing them from two branches of the network, which avoids the feature distortion caused by concatenation or summation directly. In addition, to meet the needs of portrait photo retouching (PPR) for portrait area priority (HRP), we propose Human Region Mask Generator (HRMG) module to solve the shortcomings of the current PPR task that requires the manual marking of portrait region, which achieve the PPR task more conveniently and efficiently. Extensive experiments verify the effectiveness of our method, e.g., the PSNR metric receives over 0.6 gains on high-resolution and 1.4 gains on the human-centered region at most compared with prior state-of-art works.
computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science