Chemical signalling in the nervous system.

L. Iversen

Progress in Brain Research

Abstract:

What problem does this paper attempt to address?

Deep Dual-Stream Network with Scale Context Selection Attention Module for Semantic Segmentation

Yifu Liu,Chenfeng Xu,Zhihong Chen,Chao Chen,Han Zhao,Xinyu Jin

DOI: https://doi.org/10.1007/s11063-019-10148-z

2020-01-01

Abstract:The fusion of multi-scale features has been an effective method to get state-of-the-art performance in semantic segmentation. In this work, we concentrate on two tricky problems—the intra-class inconsistency and the blur on the localization of object boundaries and tackle them by combining two separate multi-scale context features respectively. Specifically, we propose a dual-stream structure with the scale context selection attention module to enhance the capabilities for multi-scale processing, where one stream collects global-scale context and the other captures local-scale information. Meanwhile, the embedded scale context selection attention module in each stream can adaptively focus on different scale context information to get optimal scale features. Based on our dual-stream structure with attention modules, our network can efficiently make use of multi-scale context to generate more comprehensive and powerful features. Our experiments show that our dual-stream network with scale context selection attention module achieves promising performance on the PASCAL VOC 2012 and PASCAL-Person-Part datasets.
SAFPN: a Full Semantic Feature Pyramid Network for Object Detection

Gaihua Wang,Qi Li,Nengyuan Wang,Hong Liu

DOI: https://doi.org/10.1007/s10044-023-01200-9

IF: 2.307

2023-01-01

Pattern Analysis and Applications

Abstract:To enhance the performance of object detection algorithm, this paper proposes segmentation attention feature pyramid network (SAFPN) to address the issue of semantic information loss. Compared to prior works, SAFPN discards the original 1 × 1 convolutions and achieves feature dimension reduction through a segmentation and accumulation architecture, thereby preserving the semantic information of high-dimensional features completely. To capture fine-grained semantic details, it integrates channel attention and spatial attention mechanisms to enhance the network’s focus on important information. Extensive experimental validation demonstrates that SAFPN achieves favorable results on multiple public datasets, and can better complete the target detection task.
Enhanced Multi-Scale Feature Adaptive Fusion Sparse Convolutional Network for Large-Scale Scenes Semantic Segmentation

Lingfeng Shen,Yanlong Cao,Wenbin Zhu,Kai Ren,Yejun Shou,Haocheng Wang,Zhijie Xu

DOI: https://doi.org/10.1016/j.cag.2024.104105

IF: 1.821

2024-01-01

Computers & Graphics

Abstract:Semantic segmentation has made notable strides in analysing homogeneous large-scale 3D scenes, yet its application to varied scenes with diverse characteristics poses considerable challenges. Traditional methods have been hampered by the dependence on resource-intensive neighborhood search algorithms, leading to elevated computational demands. To overcome these limitations, we introduce the MFAF-SCNet, a novel and computationally streamlined approach for voxel-based sparse convolutional. Our key innovation is the multi-scale feature adaptive fusion (MFAF) module, which intelligently applies a spectrum of convolution kernel sizes at the network’s entry point, enabling the extraction of multi-scale features. It adaptively calibrates the feature weighting to achieve optimal scale representation for different objects. Further augmenting our methodology is the LKSNet, an original sparse convolutional backbone designed to tackle the inherent inconsistencies in point cloud distribution. This is achieved by integrating inverted bottleneck structures with large kernel convolutions, significantly bolstering the network’s feature extraction and spatial correlation proficiency. The efficacy of MFAF-SCNet was rigorously tested against three large-scale benchmark datasets—ScanNet and S3DIS for indoor scenes, and SemanticKITTI for outdoor scenes. The experimental results underscore our method’s competitive edge, achieving high-performance benchmarks while ensuring computational efficiency.
Adaptive multi-scale dual attention network for semantic segmentation

Weizhen Wang,Suyu Wang,Yue Li,Yishu Jin

DOI: https://doi.org/10.1016/j.neucom.2021.06.068

IF: 6

2021-10-01

Neurocomputing

Abstract:<p>Semantic segmentation in complex traffic scenes is a challenging research topic in the field of computer vision. Algorithms based on convolutional neural network has achieved more outstanding results than traditional algorithms, but their segmentation performance needs to be further improved when faced with real scenes with complex backgrounds and variable scales. In response to this issue, this study proposes a fully convolutional network architecture based on an multi-scale attention pyramid to improve the performance of the semantic segmentation algorithm from several perspectives. Firstly, a lightweight dual attention module based on depth separable convolution is designed. This module uses depth separable convolution to simplify the modeling of semantic correlation between the spatial dimension and the channel dimension, and reduces the parameter quantity of the original dual attention module. Secondly, we constructed a multi-scale attention pyramid module, which uses feature maps of different receptive fields or different scales to output multiple prediction results. Finally, an adaptive multi-scale prediction fusion module is designed. This module adaptively fuses the prediction results of multiple different receptive fields or different scales. It further enhances the network's predictive capabilities and generates detailed high-resolution predictive maps. Compared to the baseline DANet, we have achieved better results on the Cityscapes, PASCAL VOC 2012, and COCO Stuff datasets. We make the code publicly available at https://github.com/Exception-star/AMDANet.</p>

computer science, artificial intelligence
MAPPING NEUTRAL HYDROGEN IN EXTERNAL GALAXIES

M. Wright

DOI: https://doi.org/10.1007/978-3-642-96178-6_11

Abstract:
DARSegNet: A Real-Time Semantic Segmentation Method Based on Dual Attention Fusion Module and Encoder-Decoder Network

Yongfeng Xing,Luo Zhong,Xian Zhong

DOI: https://doi.org/10.1155/2022/6195148

IF: 1.43

2022-06-08

Mathematical Problems in Engineering

Abstract:The convolutional neural network achieves excellent semantic segmentation results in artificially annotated datasets with complex scenes. However, semantic segmentation methods still suffer from several problems such as low use rate of the features, high computational complexity, and being far from practical real-time application, which bring about challenges for the image semantic segmentation. Two factors are very critical to semantic segmentation task: global context and multilevel semantics. However, generating these two factors will always lead to high complexity. In order to solve this, we propose a novel structure, dual attention fusion module (DAFM), by eliminating structural redundancy. Unlike most of the existing algorithms, we combine the attention mechanism with the depth pyramid pool module (DPPM) to extract accurate dense features for pixel labeling rather than complex expansion convolution. Specifically, we introduce a DPPM to execute the spatial pyramid structure in output and combine the global pool method. The DAFM is introduced in each decoder layer. Finally, the low-level features and high-level features are fused to obtain semantic segmentation result. The experiments and visualization results on Cityscapes and CamVid datasets show that, in real-time semantic segmentation, we have achieved a satisfactory balance between accuracy and speed, which proves the effectiveness of the proposed algorithm. In particular, on a single 1080ti GPU computer, ResNet-18 produces 75.53% MIoU at 70 FPS on Cityscapes and 73.96% MIoU at 109 FPS on CamVid.

engineering, multidisciplinary,mathematics, interdisciplinary applications
An Attention-Fused Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery

Xuan Yang,Shanshan Li,Zhengchao Chen,Jocelyn Chanussot,Xiuping Jia,Bing Zhang,Baipeng Li,Pan Chen

DOI: https://doi.org/10.1016/j.isprsjprs.2021.05.004

2021-05-28

Abstract:Semantic segmentation is an essential part of deep learning. In recent years, with the development of remote sensing big data, semantic segmentation has been increasingly used in remote sensing. Deep convolutional neural networks (DCNNs) face the challenge of feature fusion: very-high-resolution remote sensing image multisource data fusion can increase the network's learnable information, which is conducive to correctly classifying target objects by DCNNs; simultaneously, the fusion of high-level abstract features and low-level spatial features can improve the classification accuracy at the border between target objects. In this paper, we propose a multipath encoder structure to extract features of multipath inputs, a multipath attention-fused block module to fuse multipath features, and a refinement attention-fused block module to fuse high-level abstract features and low-level spatial features. Furthermore, we propose a novel convolutional neural network architecture, named attention-fused network (AFNet). Based on our AFNet, we achieve state-of-the-art performance with an overall accuracy of 91.7% and a mean F1 score of 90.96% on the ISPRS Vaihingen 2D dataset and an overall accuracy of 92.1% and a mean F1 score of 93.44% on the ISPRS Potsdam 2D dataset.

Computer Vision and Pattern Recognition
Multilevel feature fusion dilated convolutional network for semantic segmentation

Tao Ku,Qirui Yang,Hao Zhang

DOI: https://doi.org/10.1177/17298814211007665

IF: 1.714

2021-03-01

International Journal of Advanced Robotic Systems

Abstract:Recently, convolutional neural network (CNN) has led to significant improvement in the field of computer vision, especially the improvement of the accuracy and speed of semantic segmentation tasks, which greatly improved robot scene perception. In this article, we propose a multilevel feature fusion dilated convolution network (Refine-DeepLab). By improving the space pyramid pooling structure, we propose a multiscale hybrid dilated convolution module, which captures the rich context information and effectively alleviates the contradiction between the receptive field size and the dilated convolution operation. At the same time, the high-level semantic information and low-level semantic information obtained through multi-level and multi-scale feature extraction can effectively improve the capture of global information and improve the performance of large-scale target segmentation. The encoder–decoder gradually recovers spatial information while capturing high-level semantic information, resulting in sharper object boundaries. Extensive experiments verify the effectiveness of our proposed Refine-DeepLab model, evaluate our approaches thoroughly on the PASCAL VOC 2012 data set without MS COCO data set pretraining, and achieve a state-of-art result of 81.73% mean interaction-over-union in the validate set.

robotics
Semantic Image Segmentation with Improved Position Attention and Feature Fusion

Hegui Zhu,Yan Miao,Xiangde Zhang

DOI: https://doi.org/10.1007/s11063-020-10240-9

IF: 2.565

2020-05-12

Neural Processing Letters

Abstract:Encoder–decoder structure is an universal method for semantic image segmentation. However, some important information of images will lost with the increasing depth of convolutional neural network (CNN), and the correlation between arbitrary pixels will get worse. This paper designs a novel image segmentation model to obtain dense feature maps and promote segmentation effects. In encoder stage, we employ ResNet-50 to extract features, and then add a spatial pooling pyramid (SPP) to achieve multi-scale feature fusion. In decoder stage, we provide an improved position attention module to integrate contextual information effectively and remove the trivial information through changing the construction way of attention matrix. Furthermore, we also propose the feature fusion structure to generate dense feature maps by preforming element–wise sum operation on the upsampling features and corresponding encoder features. The simulation results illustrate that the average accuracy and mIOU on CamVid dataset can reach 90.7% and 63.1% respectively. It verifies the effectiveness and reliability of the proposed method.

computer science, artificial intelligence
MFEAFN: Multi-scale feature enhanced adaptive fusion network for image semantic segmentation

Shusheng Li,Liang Wan,Lu Tang,Zhining Zhang

DOI: https://doi.org/10.1371/journal.pone.0274249

IF: 3.7

2022-09-30

PLoS ONE

Abstract:Low-level features contain spatial detail information, and high-level features contain rich semantic information. Semantic segmentation research focuses on fully acquiring and effectively fusing spatial detail with semantic information. This paper proposes a multiscale feature-enhanced adaptive fusion network named MFEAFN to improve semantic segmentation performance. First, we designed a Double Spatial Pyramid Module named DSPM to extract more high-level semantic information. Second, we designed a Focusing Selective Fusion Module named FSFM to fuse different scales and levels of feature maps. Specifically, the feature maps are enhanced to adaptively fuse these features by generating attention weights through a spatial attention mechanism and a two-dimensional discrete cosine transform, respectively. To validate the effectiveness of FSFM, we designed different fusion modules for comparison and ablation experiments. MFEAFN achieved 82.64% and 78.46% mIoU on the PASCAL VOC2012 and Cityscapes datasets. In addition, our method has better segmentation results than state-of-the-art methods.

multidisciplinary sciences
FCPFNet: Feature Complementation Network with Pyramid Fusion for Semantic Segmentation

Jingsheng Lei,Chente Shu,Qiang Xu,Yunxiang Yu,Shengying Yang

DOI: https://doi.org/10.1007/s11063-024-11464-9

IF: 2.565

2024-02-21

Neural Processing Letters

Abstract:Traditional pyramid pooling modules have shown effective improvements in semantic segmentation tasks by capturing multi-scale feature information. However, their limitations arise from the shallow structure, which fails to fully extract contextual information, and the fused multi-scale feature information lacks distinctiveness, resulting in issues with the final segmentation discriminability. To address these issues, we proposes an effective solution called FCPFNet, which is based on global contextual prior for deep feature extraction of detailed information. Specifically, we introduce a novel deep feature aggregation module to extract semantic information from the output feature map of each layer through a deep aggregation of context information module, and expands the effective perception range. Additionally, we propose an Efficient Pyramid Pooling Module (EPPM) to capture distinctive features through communicating information between different sub-features and performs multi-scale fusion, which is integrated as a branch within the network to complement the information loss resulting from downsampling operations. Furthermore, in order to ensure the richness of image detail feature information and maintain a large receptive field to obtain more contextual information, EPPM concatenates the input feature map and the output feature map of the pyramid pooling module to acquire more comprehensive global contextual information. It has been demonstrated by experiment that the method described in this article achieves competitive performance on the challenging scene segmentation datasets Pascal VOC 2012, Cityscapes and Coco-Stuff, with MIOU of 81.0%, 78.8% and 40.1%, respectively.

computer science, artificial intelligence
Semantic segmentation based on double pyramid network with improved global attention mechanism

Xianfeng Ou,Hanpu Wang,Guoyun Zhang,Wujing Li,Shuixiang Yu

DOI: https://doi.org/10.1007/s10489-023-04463-1

IF: 5.3

2023-02-16

Applied Intelligence

Abstract:Scene semantic segmentation is an important and challenging task, which requires labeling the category of each pixel in the image accurately. The encoder-decoder framework represented by fully convolutional network(FCN) has unique advantages in semantic segmentation. However, it is still hard to segment the small target and object boundary in the FCN framework. So, this paper proposes a global attention double pyramid network(GADPNet) based on an improved global attention mechanism to improve the performance of semantic segmentation. It is composed of deep convolutional neural networks Resnet-101, atrous spatial pyramid pooling(ASPP) module, proposed pyramid decoder structure and improved global attention module. Resnet-101 is the backbone which is used to extract different stages' features. ASPP module is used to capture multi-scale features from a high-level feature branch. Pyramid decoder structure can take advantage of multi-scale features from ASPP module and different stages' low-level multi-scale feature maps guided by improved global attention module. The proposed decoder enhances the ability to capture multi-scale features. GADPNet is an end-to-end network. The experimental results of the value of mIoU on Pascal VOC 2012 test dataset and cityscapes val dataset are 80.5% and 72.9%, which indicate that the proposed GADPNet obtains higher semantic segmentation accuracy compared with the current methods.

computer science, artificial intelligence
Dual attention deep fusion semantic segmentation networks of large-scale satellite remote-sensing images

Xin Li,Feng Xu,Xin Lyu,Hongmin Gao,Yao Tong,Sujin Cai,Shengyang Li,Daofang Liu

DOI: https://doi.org/10.1080/01431161.2021.1876272

IF: 3.531

2021-02-11

International Journal of Remote Sensing

Abstract:<span>Since DCNNs (deep convolutional neural networks) have been successfully applied to various academic and industrial fields, semantic segmentation methods, based on DCNNs, are increasingly explored for remote-sensing image interpreting and information extracting. It is still highly challenging due to the presence of irregular target shapes, and similarities of inter – and intra-class objects in large-scale high-resolution satellite images. A majority of existing methods fuse the multi-scale features that always fail to provide satisfactory results. In this paper, a dual attention deep fusion semantic segmentation network of large-scale satellite remote-sensing images is proposed (DASSN_RSI). The framework consists of novel encoder-decoder architecture, and a weight-adaptive loss function based on focal loss. To refine high-level semantic and low-level spatial feature maps, the deep layer channel attention module (DLCAM) and shallow layer spatial attention module (SLSAM) are designed and appended with specific blocks. Then the DUpsampling is incorporated to fuse feature maps in a lossless way. Peculiarly, the weight-adaptive focal loss (W-AFL) is inferred and embedded successfully, alleviating the class-imbalanced issue as much as possible. The extensive experiments are conducted on Gaofen image dataset (GID) datasets (Gaofen-2 satellite images, coarse set with five categories and refined set with fifteen categories). And the results show that our approach achieves state-of-the-art performance compared to other typical variants of encoder-decoder networks in the numerical evaluation and visual inspection. Besides, the necessary ablation studies are carried out for a comprehensive evaluation.</span>

imaging science & photographic technology,remote sensing
A Deep Fully Convolution Neural Network for Semantic Segmentation Based on Adaptive Feature Fusion

Anbang Liu,Yiqin Yang,Qingyu Sun,Qingyang Xu

DOI: https://doi.org/10.1109/ICISCE.2018.00013

2018-01-01

Abstract:Fully convolutional neural network is a special deep neural networks based on convolutional neural networks and are often used for semantic segmentation. This paper proposes an improved fully convolutional neural network which fuses the feature maps of deeper layers and shallower layers to improve the performance of image segmentation. In the process of feature fusion, adaptive parameters are introduced to enable different layers to participate in feature fusion as different proportion. The deep layers of neural network mainly extract the abstract information of the object, and the shallow layers of neural network extracts the refined features of objects, such as edge information and precise shape. Adaptive parameters can speed up the training speed and improve the prediction accuracy. In the early stages of training, the feature maps of shallow layers have a larger fusion coefficient, which allows the neural network to learn the feature of object's location and shape quickly. As the training progresses, gradually weakening the fusion coefficient of shallow layers and increasing the fusion coefficient of deep layers which can enhance the network's ability of predicting the details of the objects. This paper uses Scene Parsing Challenge 2016 dataset presented by MIT for training. Experiments show that the method proposed in this paper speeds up the training and improves the pixel prediction accuracy.
DPNet: Dual-Pyramid Semantic Segmentation Network Based on Improved Deeplabv3 Plus

Jun Wang,Xiaolin Zhang,Tianhong Yan,Aihong Tan

DOI: https://doi.org/10.3390/electronics12143161

IF: 2.9

2023-07-22

Electronics

Abstract:Semantic segmentation finds wide-ranging applications and stands as a crucial task in the realm of computer vision. It holds significant implications for scene comprehension and decision-making in unmanned systems, including domains such as autonomous driving, unmanned aerial vehicles, robotics, and healthcare. Consequently, there is a growing demand for high precision in semantic segmentation, particularly for these contents. This paper introduces DPNet, a novel image semantic segmentation method based on the Deeplabv3 plus architecture. (1) DPNet utilizes ResNet-50 as the backbone network to extract feature maps at various scales. (2) Our proposed method employs the BiFPN (Bi-directional Feature Pyramid Network) structure to fuse multi-scale information, in conjunction with the ASPP (Atrous Spatial Pyramid Pooling) module, to handle information at different scales, forming a dual pyramid structure that fully leverages the effective features obtained from the backbone network. (3) The Shuffle Attention module is employed in our approach to suppress the propagation of irrelevant information and enhance the representation of relevant features. Experimental evaluations on the Cityscapes dataset and the PASCAL VOC 2012 dataset demonstrate that our method outperforms current approaches, showcasing superior semantic segmentation accuracy.

engineering, electrical & electronic,computer science, information systems,physics, applied
MAFNet: dual-branch fusion network with multiscale atrous pyramid pooling aggregate contextual features for real-time semantic segmentation

Shan Zhao,Yunlei Wang,Xuan Wu,Fukai Zhang

DOI: https://doi.org/10.1007/s40747-024-01428-w

IF: 6.7

2024-04-17

Complex & Intelligent Systems

Abstract:Abstract Currently, many real-time semantic segmentation networks aim for heightened accuracy, inevitably leading to increased computational complexity and reduced inference speed. Therefore, striking a balance between accuracy and speed has emerged as a crucial concern in this domain. To address these challenges, this study proposes a dual-branch fusion network with multiscale atrous pyramid pooling aggregate contextual features for real-time semantic segmentation (MAFNet). The first key component, the semantics guide spatial-details module (SGSDM) not only facilitates precise boundary extraction and fine-grained classification, but also provides semantic-based feature representation, thereby enhancing support for spatial analysis and decision boundaries. The second component, the multiscale atrous pyramid pooling module (MSAPPM), is designed by combining dilation convolution with feature pyramid pooling operations at various dilation rates. This design not only expands the receptive field, but also aggregates rich contextual information more effectively. To further improve the fusion of feature information generated by the dual-branch, a bilateral fusion module (BFM) is introduced. This module employs cross-fusion by calculating weights generated by the dual-branch to balance the weight relationship between the dual branches, thereby achieving effective feature information fusion. To validate the effectiveness of the proposed network, experiments are conducted on a single A100 GPU. MAFNet achieves a mean intersection over union (mIoU) of 77.4% at 70.9 FPS on the Cityscapes test dataset and 77.6% mIoU at 192.5 FPS on the CamVid test dataset. The experimental results conclusively demonstrated that MAFNet effectively strikes a balance between accuracy and speed.

computer science, artificial intelligence
Discriminative Features Reconstruction Network For Semantic Segmentation

Qiuhao Zhou,Yanbo Ma,Haihua Lu,Xuesong Chen,Yong Zhao

DOI: https://doi.org/10.1109/icassp.2019.8683459

2019-01-01

Abstract:Thanks to the development of convolutional neural networks (CNNs), researchers have proposed lots of effective semantic segmentation models. However, there are still two problems disturbing researchers, one of which is objects misidentification on the image level and another one is poor performance on details, especially the boundary of objects. 'lb tackle these two problems, we propose Discriminative Features Reconstruction Network (DER) containing two modules: Second-order Pyramid Features Reconstruction Module (SPFR) and Second-order Boundary Attention Module (SBA). Specifically, SPFR fuses different scales features to gain pyramid receptive field. Besides, SPFR extracts second order statistics data to retrieve more discriminative features. Furthermore, we put forward SBA that is helpful to refine the segmentation results. On SBA, low-level features recover localization details under the high-level feature guidance. Our DFR achieves state-of-the-art performance on PASCAL VOC 2012 dataset with mloU accuracy 81.1% without pre-training COCO dataset and post-processing.
Research on Image Semantic Segmentation Based on Hybrid Cascade Feature Fusion and Detailed Attention Mechanism

Zuoqiang Du,Yuan Liang

DOI: https://doi.org/10.1109/access.2024.3395541

IF: 3.9

2024-05-07

IEEE Access

Abstract:In view of the low segmentation accuracy for small-scale object and insufficient segmentation of local boundary for semantic segmentation methods based on Deep Learning, this paper proposes an image semantic segmentation approach based on attention mechanism and feature fusion. On the basis of ensuring the overall accuracy, the segmentation accuracy of small-scale object and local boundary is improved, and it meets the requirement of accurately segmenting the object in the complex background. Firstly, an image semantic segmentation model based on hybrid cascade and feature fusion is proposed, and the hybrid concatenation and multi-cores pooling methods are used to extract deeper semantic information. Then, a cross-stages fusion approach is designed to divide the backbone network of the encoder stage in the network and the improved Atrous Spatial Pyramid Pooling module into three stages to fully utilize the different semantic information of the shallow and deep layers. Thirdly, the attention mechanism is introduced into the hybrid cascade and feature fusion image semantic segmentation network model, and image semantic segmentation model based on cross-stages and attention mechanisms is explained. Self attention is added to channel attention enhances the connection between feature maps, and one-dimensional convolution in the spatial attention mechanism is used to increase the spatial receptive field. The final results on the public dataset PASCAL VOC2012 and SUIM show that MIoUs have reached 86.68% and 61.55% respectively, and it proved that the overall accuracy of the approach proposed in this paper is higher than other ones.

computer science, information systems,telecommunications,engineering, electrical & electronic
Attention Guided Global Enhancement and Local Refinement Network for Semantic Segmentation

Jiangyun Li,Sen Zha,Chen Chen,Meng Ding,Tianxiang Zhang,Hong Yu

DOI: https://doi.org/10.1109/tip.2022.3166673

IF: 10.6

2022-01-01

IEEE Transactions on Image Processing

Abstract:The encoder-decoder architecture is widely used as a lightweight semantic segmentation network. However, it struggles with a limited performance compared to a well-designed Dilated-FCN model for two major problems. First, commonly used upsampling methods in the decoder such as interpolation and deconvolution suffer from a local receptive field, unable to encode global contexts. Second, low-level features may bring noises to the network decoder through skip connections for the inadequacy of semantic concepts in early encoder layers. To tackle these challenges, a Global Enhancement Method is proposed to aggregate global information from high-level feature maps and adaptively distribute them to different decoder layers, alleviating the shortage of global contexts in the upsampling process. Besides, aLocal Refinement Module is developed by utilizing the decoder features as the semantic guidance to refine the noisy encoder features before the fusion of these two (the decoder features and the encoder features). Then, the two methods are integrated into a Context Fusion Block, and based on that, a novel Attention guided Global enhancement and Local refinement Network (AGLN) is elaborately designed. Extensive experiments on PASCAL Context, ADE20K, and PASCAL VOC 2012 datasets have demonstrated the effectiveness of the proposed approach. In particular, with a vanilla ResNet-101 backbone, AGLN achieves the state-of-the-art result (56.23% mean IOU) on the PASCAL Context dataset. The code is available at https://github.com/zhasen1996/AGLN.

computer science, artificial intelligence,engineering, electrical & electronic
A Deep Semantic Segmentation Network with Semantic and Contextual Refinements

Zhiyan Wang,Deyin Liu,Lin Yuanbo Wu,Song Wang,Xin Guo,Lin Qi

2024-12-11

Abstract:Semantic segmentation is a fundamental task in multimedia processing, which can be used for analyzing, understanding, editing contents of images and videos, among others. To accelerate the analysis of multimedia data, existing segmentation researches tend to extract semantic information by progressively reducing the spatial resolutions of feature maps. However, this approach introduces a misalignment problem when restoring the resolution of high-level feature maps. In this paper, we design a Semantic Refinement Module (SRM) to address this issue within the segmentation network. Specifically, SRM is designed to learn a transformation offset for each pixel in the upsampled feature maps, guided by high-resolution feature maps and neighboring offsets. By applying these offsets to the upsampled feature maps, SRM enhances the semantic representation of the segmentation network, particularly for pixels around object boundaries. Furthermore, a Contextual Refinement Module (CRM) is presented to capture global context information across both spatial and channel dimensions. To balance dimensions between channel and space, we aggregate the semantic maps from all four stages of the backbone to enrich channel context information. The efficacy of these proposed modules is validated on three widely used datasets-Cityscapes, Bdd100K, and ADE20K-demonstrating superior performance compared to state-of-the-art methods. Additionally, this paper extends these modules to a lightweight segmentation network, achieving an mIoU of 82.5% on the Cityscapes validation set with only 137.9 GFLOPs.

Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing

Chemical signalling in the nervous system.

Deep Dual-Stream Network with Scale Context Selection Attention Module for Semantic Segmentation

SAFPN: a Full Semantic Feature Pyramid Network for Object Detection

Enhanced Multi-Scale Feature Adaptive Fusion Sparse Convolutional Network for Large-Scale Scenes Semantic Segmentation

Adaptive multi-scale dual attention network for semantic segmentation

MAPPING NEUTRAL HYDROGEN IN EXTERNAL GALAXIES

DARSegNet: A Real-Time Semantic Segmentation Method Based on Dual Attention Fusion Module and Encoder-Decoder Network

An Attention-Fused Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery

Multilevel feature fusion dilated convolutional network for semantic segmentation

Semantic Image Segmentation with Improved Position Attention and Feature Fusion

MFEAFN: Multi-scale feature enhanced adaptive fusion network for image semantic segmentation

FCPFNet: Feature Complementation Network with Pyramid Fusion for Semantic Segmentation

Semantic segmentation based on double pyramid network with improved global attention mechanism

Dual attention deep fusion semantic segmentation networks of large-scale satellite remote-sensing images

A Deep Fully Convolution Neural Network for Semantic Segmentation Based on Adaptive Feature Fusion

DPNet: Dual-Pyramid Semantic Segmentation Network Based on Improved Deeplabv3 Plus

MAFNet: dual-branch fusion network with multiscale atrous pyramid pooling aggregate contextual features for real-time semantic segmentation

Discriminative Features Reconstruction Network For Semantic Segmentation

Research on Image Semantic Segmentation Based on Hybrid Cascade Feature Fusion and Detailed Attention Mechanism

Attention Guided Global Enhancement and Local Refinement Network for Semantic Segmentation

A Deep Semantic Segmentation Network with Semantic and Contextual Refinements