Senet: Spatial Information Enhancement for Semantic Segmentation Neural Networks

Yifang Huang,Peng Shi,Haitao He,Hongdou He,Bowen Zhao
DOI: https://doi.org/10.1007/s00371-023-03043-1
2024-01-01
Abstract:Image semantic segmentation is a basic task of computer vision, and plays an important role in automatic driving, robot navigation and many other fields. However, the expensive computing cost limits its deployment on mobile devices. Therefore, the primary object of this study is to balance accuracy and inference speed in the semantic segmentation task. To this end, we propose a real-time semantic segmentation network with Spatial Enhancement (SENet). We propose to strengthen the information association between feature maps of different resolutions by attention mechanism. We design a spatial information branch to retain the high quality spatial features. The segmentation of object edges is improved by enhancing edge information, and the representation of features is improved by correlating high-level semantic information with low-level spatial information. The real-time performance of the model is achieved by using a lightweight feature enhancement module and a backbone network with low computational complexity. We have carried out several sets of experiments to test the validity of our SENet. The effectiveness and efficiency of SENet are evaluated on the PASCAL VOC2012 and the CityScapes dataset. The model achieves 76.37% and 77.23% mIoU segmentation accuracy, respectively, while the speed reaches 193.3 FPS and 30.8 FPS on a NVIDIA RTX 3080 GPU card. The research has resulted in a solution of balancing the accuracy and inference speed.
What problem does this paper attempt to address?