Two-stream deep sparse network for accurate and efficient image restoration
Shuhui Wang,Ling Hu,Liang Li,Weigang Zhang,Qingming Huang
DOI: https://doi.org/10.1016/j.cviu.2020.103029
IF: 4.886
2020-11-01
Computer Vision and Image Understanding
Abstract:<p>Deep convolutional neural network (CNN) has achieved great success in image restoration. However, previous methods ignore the complementarity between low-level and high-level features, thereby leading to limited image reconstruction quality. In this paper, we propose a two-stream sparse network (TSSN) to explicitly learn shallow and deep features to enforce their respective contribution to image restoration. The shallow stream learns shallow features (<em>e.g.</em>, texture edges), and the deep stream learns deep features (<em>e.g.</em>, salient semantics). In each stream, sparse residual block (SRB) is proposed to efficiently aggregate hierarchical features by constructing sparse connections among layers in the local block. Spatial-wise and channel-wise attention are used to fuse the shallow and deep stream which recalibrates features by weight assignment in both spatial and channel dimensions. A novel loss function called Softmax-<span class="math"><math>L1</math></span> loss is proposed to increase penalties of pixels that have large <span class="math"><math>L1</math></span> loss (<em>i.e.</em>, hard pixels). TSSN is evaluated with three representative IR applications, <em>i.e.</em>, single image super-resolution, image denoising and JPEG deblocking. Extensive experiments demonstrate that TSSN outperforms most of state-of-the-art methods on benchmark datasets on both quantitative metric and visual quality.</p>
computer science, artificial intelligence,engineering, electrical & electronic