The preparation and pharmacological properties of 3-acetyltyrosine and 3-acetyltyramine.

L. Florvall,S. Ross,S. Ogren,N. Stjernström

Abstract:

What problem does this paper attempt to address?

A Scalable Real-time Semantic Segmentation Network for Autonomous Driving

Jun Liu,Chao Wu,Geng Yuan,Wei Niu,Wenbin Zhang,Houbing Herbert Song

DOI: https://doi.org/10.1145/3606042.3616451

2023-01-01

Abstract:Computational resources are limited on real-time embedded devices, so the available computing cost of deployment on the target platform must be considered. We develop a feature extraction module based on the MobileNet backbone that can be adjusted in terms of computational complexity and capacity using the depth multiplier parameter, classifier depth, and kernel depth. These three parameters allow us to control the count of channels within the network, effectively managing the model's capacity and computational requirements. To achieve semantic segmentation, we incorporate additional components, such as an extension module. This extension module typically includes 1x1 pointwise convolutional layers for pixel-level classification and a transposed convolutional layer for upsampling the output to the original input image size. By combining the feature extraction module with this extension module, we can create a complete architecture capable of performing semantic segmentation tasks. The feature extraction module provides the initial feature extraction and the extension module adds the necessary components for accurate pixel-wise classification and upsampling. Compared to Hardware-aware Neural Architecture Search (NAS), pruning, runtime pruning, and knowledge distillation methods, our model has several advantages in terms of modular design, structural controllability, ease of implementation, and cost-effectiveness. Our computational efficiency, as measured by FLOPS, is highly competitive. Our method is distinguished by solving the problem of MobileNet's inability to adjust the size and number of convolution kernels. It achieves this through adaptable parameter tuning, including MobileNet's depth multiplier, the kernel size in the FCN head's Separable Convolution layer, and the depth of the first Point-wise Convolution layer. These adjustments are customized to match hardware's max multiply-accumulates (MACs), optimizing network capacity and maximizing resource utilization.
Real-time Semantic Segmentation with Weighted Factorized-Depthwise Convolution

Xiaochen Hao,Xingjun Hao,Yaru Zhang,Yuanyuan Li,Chao Wu

DOI: https://doi.org/10.1016/j.imavis.2021.104269

IF: 3.86

2021-01-01

Image and Vision Computing

Abstract:Semantic segmentation has achieved great success with the popularity of convolutional neural networks (CNNs). However, the huge computational burden restricts the application of most existing networks on edge devices with strict inference time constraints. To solve this problem, a weighted factorized-depthwise convolution network (WFDCNet) is presented in this paper, which contains full- dimensional continuous separation convolution (FCS) modules and a lateral asymmetric pyramid fusion (LAPF) module, aiming to obtain high accuracy without damaging inference speed. Specifically, the FCS module enables the calculation of each dimension to be completed independently in a continuous separation process and uses simplified SE (SSE) attention layer to adjust the channels, achieving the extensive extraction of feature information. The LAPF module is able to eliminate semantic divergence and fuse feature maps of three different scales to realize the combination of multiple information from the front-end and the back-end network. WFDCNet shows superior performance on Cityscapes, Camvid, Mapillary Vistas and COCO-Stuff datasets. Especially, the experimental results demonstrate that our network achieves 73.7% mIoU on Cityscapes dataset, with the inference speed of 102.6FPS on a single RTX 2080 Ti GPU, and 17.2FPS on Jetson TX2.
Lightweight and Progressively-Scalable Networks for Semantic Segmentation

Yiheng Zhang,Ting Yao,Zhaofan Qiu,Tao Mei

DOI: https://doi.org/10.48550/arXiv.2207.13600

2022-07-28

Abstract:Multi-scale learning frameworks have been regarded as a capable class of models to boost semantic segmentation. The problem nevertheless is not trivial especially for the real-world deployments, which often demand high efficiency in inference latency. In this paper, we thoroughly analyze the design of convolutional blocks (the type of convolutions and the number of channels in convolutions), and the ways of interactions across multiple scales, all from lightweight standpoint for semantic segmentation. With such in-depth comparisons, we conclude three principles, and accordingly devise Lightweight and Progressively-Scalable Networks (LPS-Net) that novelly expands the network complexity in a greedy manner. Technically, LPS-Net first capitalizes on the principles to build a tiny network. Then, LPS-Net progressively scales the tiny network to larger ones by expanding a single dimension (the number of convolutional blocks, the number of channels, or the input resolution) at one time to meet the best speed/accuracy tradeoff. Extensive experiments conducted on three datasets consistently demonstrate the superiority of LPS-Net over several efficient semantic segmentation methods. More remarkably, our LPS-Net achieves 73.4% mIoU on Cityscapes test set, with the speed of 413.5FPS on an NVIDIA GTX 1080Ti, leading to a performance improvement by 1.5% and a 65% speed-up against the state-of-the-art STDC. Code is available at \url{<a class="link-external link-https" href="https://github.com/YihengZhang-CV/LPS-Net" rel="external noopener nofollow">this https URL</a>}.

Computer Vision and Pattern Recognition
LMANet: A Lightweight Asymmetric Semantic Segmentation Network Based on Multi-Scale Feature Extraction

Hui Chen,Zhexuan Xiao,Bin Ge,Xuedi Li

DOI: https://doi.org/10.3390/electronics13173361

IF: 2.9

2024-08-25

Electronics

Abstract:With the swift progress of deep learning and its wide application in semantic segmentation, the effect of semantic segmentation has been significantly improved. However, how to achieve a reasonable compromise between accuracy, model size, and inference speed is crucial. In this paper, we propose a lightweight multi-scale asymmetric encoder–decoder network (LMANet) that is designed on the basis of an encoder–decoder structure. First, an optimized bottleneck module is used to extract features from different levels, and different receptive fields are applied to obtain effective information on different scales. Then, a channel-attention module and a feature-extraction module are introduced to constitute the residual structure, and different feature maps are connected by a feature-fusion module to effectively improve segmentation accuracy. Finally, a lightweight multi-scale decoder is designed to recover the image, and a spatial attention module is added to recover the spatial details effectively. This paper has verified the proposed method on the Cityscapes dataset and CamVid dataset and achieved mean intersection over union (mIoU) of 73.9% and 71.3% with the inference speeds of 111 FPS and 118 FPS, respectively, and the number of parameters is only 0.85 M.

engineering, electrical & electronic,computer science, information systems,physics, applied
Lightweight Semantic Segmentation Network for Semantic Scene Understanding on Low-Compute Devices

James Weiland,H. Son

DOI: https://doi.org/10.1109/IROS55552.2023.10342110

2023-10-01

Abstract:Semantic scene understanding is beneficial for mobile robots. Semantic information obtained through onboard cameras can improve robots' navigation performance. However, obtaining semantic information on small mobile robots with constrained power and computation resources is challenging. We propose a new lightweight convolution neural network comparable to previous semantic segmentation algorithms for mobile applications. Our network achieved 73.06% on the Cityscapes validation set and 71.8% on the Cityscapes test set. Our model runs at 116 fps with $\mathbf{1024\mathrm{x}2048}$, 172 fps with $1024\mathrm{x}1024$, and 175 fps with $720\mathrm{x}960$ on NVIDIA GTX 1080. We analyze a model size, which is defined as the summation of the number of floating operations and the number of parameters. The smaller model size enables tiny mobile robot systems that should operate multiple tasks simultaneously to work efficiently. Our model has the smallest model size compared to the real-time semantic segmentation convolution neural networks ranked on Cityscapes real-time benchmark and other high performing, lightweight convolution neural networks. On the Camvid test set, our model achieved a mIoU of 73.29% with Cityscapes pre-training, which outperformed the accuracy of other lightweight convolution neural networks. For mobile applicability, we measured frame-per-second on different low-compute devices. Our model operates 35 fps on Jetson Xavier AGX, 21 FPS on Jetson Xavier NX, and 14 FPS on a ROS ASUS gaming phone. $1024\mathrm{x}2048$ resolution is used for the Jetson devices, and $512\mathrm{x}512$ size is utilized for the measurement on the phone. Our network did not use extra datasets such as ImageNet, Coarse Cityscapes, and Mapillary. Additionally, we did not use TensorRT to achieve fast inference speed. Compared to other real-time and lightweight CNNs, our model achieved significantly more efficiency while balancing accuracy, inference speed, and model size.

Computer Science,Engineering
Enhanced Multi-Scale Feature Adaptive Fusion Sparse Convolutional Network for Large-Scale Scenes Semantic Segmentation

Lingfeng Shen,Yanlong Cao,Wenbin Zhu,Kai Ren,Yejun Shou,Haocheng Wang,Zhijie Xu

DOI: https://doi.org/10.1016/j.cag.2024.104105

IF: 1.821

2024-01-01

Computers & Graphics

Abstract:Semantic segmentation has made notable strides in analysing homogeneous large-scale 3D scenes, yet its application to varied scenes with diverse characteristics poses considerable challenges. Traditional methods have been hampered by the dependence on resource-intensive neighborhood search algorithms, leading to elevated computational demands. To overcome these limitations, we introduce the MFAF-SCNet, a novel and computationally streamlined approach for voxel-based sparse convolutional. Our key innovation is the multi-scale feature adaptive fusion (MFAF) module, which intelligently applies a spectrum of convolution kernel sizes at the network’s entry point, enabling the extraction of multi-scale features. It adaptively calibrates the feature weighting to achieve optimal scale representation for different objects. Further augmenting our methodology is the LKSNet, an original sparse convolutional backbone designed to tackle the inherent inconsistencies in point cloud distribution. This is achieved by integrating inverted bottleneck structures with large kernel convolutions, significantly bolstering the network’s feature extraction and spatial correlation proficiency. The efficacy of MFAF-SCNet was rigorously tested against three large-scale benchmark datasets—ScanNet and S3DIS for indoor scenes, and SemanticKITTI for outdoor scenes. The experimental results underscore our method’s competitive edge, achieving high-performance benchmarks while ensuring computational efficiency.
LiteSeg: A Novel Lightweight ConvNet for Semantic Segmentation

Taha Emara,Hossam E. Abd El Munim,Hazem M. Abbas

DOI: https://doi.org/10.1109/DICTA47822.2019.8945975

2019-12-14

Abstract:Semantic image segmentation plays a pivotal role in many vision applications including autonomous driving and medical image analysis. Most of the former approaches move towards enhancing the performance in terms of accuracy with a little awareness of computational efficiency. In this paper, we introduce LiteSeg, a lightweight architecture for semantic image segmentation. In this work, we explore a new deeper version of Atrous Spatial Pyramid Pooling module (ASPP) and apply short and long residual connections, and depthwise separable convolution, resulting in a faster and efficient model. LiteSeg architecture is introduced and tested with multiple backbone networks as Darknet19, MobileNet, and ShuffleNet to provide multiple trade-offs between accuracy and computational cost. The proposed model LiteSeg, with MobileNetV2 as a backbone network, achieves an accuracy of 67.81% mean intersection over union at 161 frames per second with $640 \times 360$ resolution on the Cityscapes dataset.

Computer Vision and Pattern Recognition
LRNNet: A Light-Weighted Network with Efficient Reduced Non-Local Operation for Real-Time Semantic Segmentation

Weihao Jiang,Zhaozhi Xie,Yaoyi Li,Chang Liu,Hongtao Lu

DOI: https://doi.org/10.48550/arXiv.2006.02706

2020-06-04

Abstract:The recent development of light-weighted neural networks has promoted the applications of deep learning under resource constraints and mobile applications. Many of these applications need to perform a real-time and efficient prediction for semantic segmentation with a light-weighted network. This paper introduces a light-weighted network with an efficient reduced non-local module (LRNNet) for efficient and realtime semantic segmentation. We proposed a factorized convolutional block in ResNet-Style encoder to achieve more lightweighted, efficient and powerful feature extraction. Meanwhile, our proposed reduced non-local module utilizes spatial regional dominant singular vectors to achieve reduced and more representative non-local feature integration with much lower computation and memory cost. Experiments demonstrate our superior trade-off among light-weight, speed, computation and accuracy. Without additional processing and pretraining, LRNNet achieves 72.2% mIoU on Cityscapes test dataset only using the fine annotation data for training with only 0.68M parameters and with 71 FPS on a GTX 1080Ti card.

Computer Vision and Pattern Recognition
LEDNet: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation

Yu Wang,Quan Zhou,Jia Liu,Jian Xiong,Guangwei Gao,Xiaofu Wu,Longin Jan Latecki

DOI: https://doi.org/10.48550/arXiv.1905.02423

2019-05-13

Abstract:The extensive computational burden limits the usage of CNNs in mobile devices for dense estimation tasks. In this paper, we present a lightweight network to address this problem,namely LEDNet, which employs an asymmetric encoder-decoder architecture for the task of real-time semantic <a class="link-external link-http" href="http://segmentation.More" rel="external noopener nofollow">this http URL</a> specifically, the encoder adopts a ResNet as backbone network, where two new operations, channel split and shuffle, are utilized in each residual block to greatly reduce computation cost while maintaining higher segmentation accuracy. On the other hand, an attention pyramid network (APN) is employed in the decoder to further lighten the entire network complexity. Our model has less than 1M parameters,and is able to run at over 71 FPS in a single GTX 1080Ti GPU. The comprehensive experiments demonstrate that our approach achieves state-of-the-art results in terms of speed and accuracy trade-off on CityScapes dataset.

Computer Vision and Pattern Recognition
Lightweight Real-time Semantic Segmentation Network with Efficient Transformer and CNN

Guoan Xu,Juncheng Li,Guangwei Gao,Huimin Lu,Jian Yang,Dong Yue

DOI: https://doi.org/10.48550/arXiv.2302.10484

2023-02-21

Computer Vision and Pattern Recognition

Abstract:In the past decade, convolutional neural networks (CNNs) have shown prominence for semantic segmentation. Although CNN models have very impressive performance, the ability to capture global representation is still insufficient, which results in suboptimal results. Recently, Transformer achieved huge success in NLP tasks, demonstrating its advantages in modeling long-range dependency. Recently, Transformer has also attracted tremendous attention from computer vision researchers who reformulate the image processing tasks as a sequence-to-sequence prediction but resulted in deteriorating local feature details. In this work, we propose a lightweight real-time semantic segmentation network called LETNet. LETNet combines a U-shaped CNN with Transformer effectively in a capsule embedding style to compensate for respective deficiencies. Meanwhile, the elaborately designed Lightweight Dilated Bottleneck (LDB) module and Feature Enhancement (FE) module cultivate a positive impact on training from scratch simultaneously. Extensive experiments performed on challenging datasets demonstrate that LETNet achieves superior performances in accuracy and efficiency balance. Specifically, It only contains 0.95M parameters and 13.6G FLOPs but yields 72.8\% mIoU at 120 FPS on the Cityscapes test set and 70.5\% mIoU at 250 FPS on the CamVid test dataset using a single RTX 3090 GPU. The source code will be available at https://github.com/IVIPLab/LETNet.
ELANet: an efficiently lightweight asymmetrical network for real-time semantic segmentation

Jiafei Chen,Junyang Yu,Yingqi Wang,Xin He

DOI: https://doi.org/10.1117/1.jei.33.1.013008

IF: 0.829

2024-01-08

Journal of Electronic Imaging

Abstract:Semantic segmentation plays a crucial role in practical applications, such as autonomous driving and robot navigation. However, prevalent semantic segmentation networks suffer from two primary challenges: oversized networks with redundant parameters that hinder network inference speed and excessively lightweight network structures that sacrifice semantic segmentation accuracy. Therefore, it is essential to design a semantic segmentation network that strikes a balance between accuracy and inference speed. We propose the asymmetric residual bottleneck module, which incorporates dilated convolution, depth-wise separable asymmetric convolution, channel attention mechanism, and a channel shuffle unit. By utilizing these components, model parameters are effectively reduced, and inference speed is accelerated. Furthermore, a feature aggregation module is designed to integrate features from feature maps with various resolutions, thereby enhancing segmentation accuracy. Based on these advancements, an efficient and lightweight real-time semantic segmentation network called efficiently lightweight asymmetrical network (ELANet) is proposed. Experimental results of the Cityscapes and CamVid datasets demonstrate that ELANet strikes a favorable balance between speed and accuracy. Notably, without any pretrained model and postprocessing scheme, ELANet achieves an impressive mean intersection over union of 72.5% on the Cityscapes test dataset with only 0.82 million parameters, operating at an inference speed of 173.5 frames per second on a single NVIDIA GTX 3090 GPU, with a 512×1024 input image. These findings underscore ELANet’s tremendous potential for real-time applications.

engineering, electrical & electronic,optics,imaging science & photographic technology
Lightweight semantic segmentation network with configurable context and small object attention

Chunyu Zhang,Fang Xu,Chengdong Wu,Jinzhao Li

DOI: https://doi.org/10.3389/fncom.2023.1280640

IF: 3.387

2023-10-23

Frontiers in Computational Neuroscience

Abstract:The current semantic segmentation algorithms suffer from encoding feature distortion and small object feature loss. Context information exchange can effectively address the feature distortion problem, but it has the issue of fixed spatial range. Maintaining the input feature resolution can reduce the loss of small object information but would slow down the network's operation speed. To tackle these problems, we propose a lightweight semantic segmentation network with configurable context and small object attention (CCSONet). CCSONet includes a long-short distance configurable context feature enhancement module (LSCFEM) and a small object attention decoding module (SOADM). The LSCFEM differs from the regular context exchange module by configuring long and short-range relevant features for the current feature, providing a broader and more flexible spatial range. The SOADM enhances the features of small objects by establishing correlations among objects of the same category, avoiding the introduction of redundancy issues caused by high-resolution features. On the Cityscapes and Camvid datasets, our network achieves the accuracy of 76.9 mIoU and 73.1 mIoU, respectively, while maintaining speeds of 87 FPS and 138 FPS. It outperforms other lightweight semantic segmentation algorithms in terms of accuracy.

neurosciences,mathematical & computational biology
FDDWNet: A Lightweight Convolutional Neural Network for Real-time Sementic Segmentation

Jia Liu,Quan Zhou,Yong Qiang,Bin Kang,Xiaofu Wu,Baoyu Zheng

DOI: https://doi.org/10.48550/arXiv.1911.00632

2019-11-08

Abstract:This paper introduces a lightweight convolutional neural network, called FDDWNet, for real-time accurate semantic segmentation. In contrast to recent advances of lightweight networks that prefer to utilize shallow structure, FDDWNet makes an effort to design more deeper network architecture, while maintains faster inference speed and higher segmentation accuracy. Our network uses factorized dilated depth-wise separable convolutions (FDDWC) to learn feature representations from different scale receptive fields with fewer model parameters. Additionally, FDDWNet has multiple branches of skipped connections to gather context cues from intermediate convolution layers. The experiments show that FDDWNet only has 0.8M model size, while achieves 60 FPS running speed on a single RTX 2080Ti GPU with a 1024x512 input image. The comprehensive experiments demonstrate that our model achieves state-of-the-art results in terms of available speed and accuracy trade-off on CityScapes and CamVid datasets.

Computer Vision and Pattern Recognition
Lightweight Attention Network for Very High-Resolution Image Semantic Segmentation

Renchu Guan,Mingming Wang,Lorenzo Bruzzone,Haishi Zhao,Chen Yang

DOI: https://doi.org/10.1109/tgrs.2023.3272614

IF: 8.2

2023-05-16

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Semantic segmentation is one of the most challenging tasks for very high-resolution (VHR) remote sensing applications. Deep convolutional neural networks (DCNNs) based on the attention mechanism have shown outstanding performance in VHR remote sensing images semantic segmentation. However, the existing attention-guided methods require the estimation of a large number of parameters that are affected by the limited number of available labeled samples that results in underperforming segmentation results. In this article, we propose a multistage feature fusion lightweight (MSFFL) model to greatly reduce the number of parameters and improve the accuracy of semantic segmentation. In this model, two parallel enhanced attention modules, i.e., the spatial attention module (SAM) and the channel attention module (CAM), are designed by introducing encoding position information. Then, a covariance calculation strategy is adopted to recalibrate the generated attention maps. The integration of enhanced attention modules into the proposed lightweight module results in an efficient lightweight attention network (LiANet). The performance of the proposed LiANet is assessed on two benchmark datasets. Experimental results demonstrate that LiANet can achieve promising performance with a small number of parameters.

imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics
EADNet: Efficient Asymmetric Dilated Network for Semantic Segmentation

Qihang Yang,Tao Chen,Jiayuan Fan,Ye Lu,Chongyan Zuo,Qinghua Chi

DOI: https://doi.org/10.48550/arXiv.2103.08914

2021-03-16

Abstract:Due to real-time image semantic segmentation needs on power constrained edge devices, there has been an increasing desire to design lightweight semantic segmentation neural network, to simultaneously reduce computational cost and increase inference speed. In this paper, we propose an efficient asymmetric dilated semantic segmentation network, named EADNet, which consists of multiple developed asymmetric convolution branches with different dilation rates to capture the variable shapes and scales information of an image. Specially, a multi-scale multi-shape receptive field convolution (MMRFC) block with only a few parameters is designed to capture such information. Experimental results on the Cityscapes dataset demonstrate that our proposed EADNet achieves segmentation mIoU of 67.1 with smallest number of parameters (only 0.35M) among mainstream lightweight semantic segmentation networks.

Computer Vision and Pattern Recognition
Efficient Semantic Segmentation via Lightweight Multiple-Information Interaction Network

Yangyang Qiu,Guoan Xu,Guangwei Gao,Zhenhua Guo,Yi Yu,Chia-Wen Lin

2024-10-03

Abstract:Recently, the integration of the local modeling capabilities of Convolutional Neural Networks (CNNs) with the global dependency strengths of Transformers has created a sensation in the semantic segmentation community. However, substantial computational workloads and high hardware memory demands remain major obstacles to their further application in real-time scenarios. In this work, we propose a lightweight multiple-information interaction network for real-time semantic segmentation, called LMIINet, which effectively combines CNNs and Transformers while reducing redundant computations and memory footprint. It features Lightweight Feature Interaction Bottleneck (LFIB) modules comprising efficient convolutions that enhance context integration. Additionally, improvements are made to the Flatten Transformer by enhancing local and global feature interaction to capture detailed semantic information. The incorporation of a combination coefficient learning scheme in both LFIB and Transformer blocks facilitates improved feature interaction. Extensive experiments demonstrate that LMIINet excels in balancing accuracy and efficiency. With only 0.72M parameters and 11.74G FLOPs, LMIINet achieves 72.0% mIoU at 100 FPS on the Cityscapes test set and 69.94% mIoU at 160 FPS on the CamVid test dataset using a single RTX2080Ti GPU.

Computer Vision and Pattern Recognition
Efficient Context Integration through Factorized Pyramidal Learning for Ultra-Lightweight Semantic Segmentation

Nadeem Atif,Saquib Mazhar,Debajit Sarma,M. K. Bhuyan,Shaik Rafi Ahamed

DOI: https://doi.org/10.48550/arXiv.2302.11785

2023-02-23

Abstract:Semantic segmentation is a pixel-level prediction task to classify each pixel of the input image. Deep learning models, such as convolutional neural networks (CNNs), have been extremely successful in achieving excellent performances in this domain. However, mobile application, such as autonomous driving, demand real-time processing of incoming stream of images. Hence, achieving efficient architectures along with enhanced accuracy is of paramount importance. Since, accuracy and model size of CNNs are intrinsically contentious in nature, the challenge is to achieve a decent trade-off between accuracy and model size. To address this, we propose a novel Factorized Pyramidal Learning (FPL) module to aggregate rich contextual information in an efficient manner. On one hand, it uses a bank of convolutional filters with multiple dilation rates which leads to multi-scale context aggregation; crucial in achieving better accuracy. On the other hand, parameters are reduced by a careful factorization of the employed filters; crucial in achieving lightweight models. Moreover, we decompose the spatial pyramid into two stages which enables a simple and efficient feature fusion within the module to solve the notorious checkerboard effect. We also design a dedicated Feature-Image Reinforcement (FIR) unit to carry out the fusion operation of shallow and deep features with the downsampled versions of the input image. This gives an accuracy enhancement without increasing model parameters. Based on the FPL module and FIR unit, we propose an ultra-lightweight real-time network, called FPLNet, which achieves state-of-the-art accuracy-efficiency trade-off. More specifically, with only less than 0.5 million parameters, the proposed network achieves 66.93\% and 66.28\% mIoU on Cityscapes validation and test set, respectively. Moreover, FPLNet has a processing speed of 95.5 frames per second (FPS).

Computer Vision and Pattern Recognition
LACTNet: A Lightweight Real-Time Semantic Segmentation Network Based on an Aggregated Convolutional Neural Network and Transformer

Xiangyue Zhang,Hexiao Li,Jingyu Ru,Peng Ji,Chengdong Wu

DOI: https://doi.org/10.3390/electronics13122406

IF: 2.9

2024-06-20

Electronics

Abstract:Transformers have demonstrated a significant advantage over CNNs in modeling long-range dependencies, leading to increasing attention being paid towards their application in semantic segmentation tasks. In the present work, a novel semantic segmentation model, LACTNet, is introduced, which synergistically combines Transformer and CNN architectures for the real-time processing of local and global contextual features. LACTNet is designed with a lightweight Transformer, which integrates a specially designed gated convolutional feedforward network, to establish feature dependencies across distant regions. A Lightweight Average Feature Bottleneck (LAFB) module is designed to effectively capture spatial detail information within the features, thereby enhancing segmentation accuracy. To address the issue of spatial feature loss in the decoder, a long skip-connection approach is employed through the designed Feature Fusion Enhancement Module (FFEM), which enhances the integrity of spatial features and the feature interaction capability in the decoder. LACTNet is evaluated on two datasets, achieving a segmentation accuracy of 74.8% mIoU and a frame rate of 90 FPS on the Cityscapes dataset, and a segmentation accuracy of 71.8% mIoU with a frame rate of 126 FPS on the CamVid dataset.

engineering, electrical & electronic,computer science, information systems,physics, applied
A Lightweight CNN–Transformer Network With Laplacian Loss for Low-Altitude UAV Imagery Semantic Segmentation

Wen Lu,Zhiqi Zhang,Minh Nguyen

DOI: https://doi.org/10.1109/tgrs.2024.3385318

IF: 8.2

2024-04-16

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Semantic segmentation is crucial for enabling autonomous flight and landing of low-altitude unmanned aerial vehicles (UAVs) and is indispensable for various intelligent applications. However, real-time semantic segmentation is a computationally intensive task because it involves pixel-wise classification, which renders conventional semantic segmentation networks impractical for deployment on embedded systems of limited hardware resources. Moreover, variations in flight height and object appearance increase the likelihood of misjudgment in segmentation results. To address these challenges, we propose an efficient approach consisting of a convolutional neural network (CNN)–Transformer network and an auxiliary loss. The encoder of the network integrates a newly designed module, which equally handles objects with varying scales. The decoder is composed of the innovative query–value squeeze axial transformer attention (QVSATA), which reduces computational complexity from quadratic in terms of image size to , linear in terms of image size. By incorporating Laplacian operator convolution, the novel network-agnostic loss effectively captures intricate patterns, boundaries, and small objects. This enables extra penalization of misjudgments in these areas and compels the network to focus on objects that are challenging to distinguish. Our approach attains impressive accuracy when processing 4K resolution images in real time (15 FPS) on a mobile GPU. It demonstrates over faster speed compared to representative lightweight networks, underscoring its suitability for onboard deployment.

imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics
EFRNet: A Lightweight Network with Efficient Feature Fusion and Refinement for Real-Time Semantic Segmentation

Kuayue Zhang,Qingmin Liao,Juncheng Zhang,Shaojun Liu,Haoyu Ma,Jing-Hao Xue

DOI: https://doi.org/10.1109/icme51207.2021.9428371

2021-01-01

Abstract:To pursue high accuracy, most image semantic segmentation methods are computationally costly and thus not suitable to real-time applications. Existing lightweight methods either adopt a single branch without feature fusion, which dam-ages accuracy, or introduce extra branches for feature fusion, which harms efficiency. In this paper, we propose a lightweight network named EFRNet, with feature fusion and refinement in a single branch to achieve better balance between accuracy and efficiency in real-time semantic segmentation. Specifically, in EFRNet, we design a novel Feature Fusion Module to fuse multi-stage features in a single CNN efficiently, and we propose a lightweight Channel Attention Refinement Module to refine features with few extra parameters. Extensive experiments show that our EFRNet achieves decent accuracy with an extremely small model size and high inference speed. It achieves the best accuracy of 70.02% mIoU compared with state-of-the-art lightweight methods on CamVid with only 0.48M parameters.

The preparation and pharmacological properties of 3-acetyltyrosine and 3-acetyltyramine.

A Scalable Real-time Semantic Segmentation Network for Autonomous Driving

Real-time Semantic Segmentation with Weighted Factorized-Depthwise Convolution

Lightweight and Progressively-Scalable Networks for Semantic Segmentation

LMANet: A Lightweight Asymmetric Semantic Segmentation Network Based on Multi-Scale Feature Extraction

Lightweight Semantic Segmentation Network for Semantic Scene Understanding on Low-Compute Devices

Enhanced Multi-Scale Feature Adaptive Fusion Sparse Convolutional Network for Large-Scale Scenes Semantic Segmentation

LiteSeg: A Novel Lightweight ConvNet for Semantic Segmentation

LRNNet: A Light-Weighted Network with Efficient Reduced Non-Local Operation for Real-Time Semantic Segmentation

LEDNet: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation

Lightweight Real-time Semantic Segmentation Network with Efficient Transformer and CNN

ELANet: an efficiently lightweight asymmetrical network for real-time semantic segmentation

Lightweight semantic segmentation network with configurable context and small object attention

FDDWNet: A Lightweight Convolutional Neural Network for Real-time Sementic Segmentation

Lightweight Attention Network for Very High-Resolution Image Semantic Segmentation

EADNet: Efficient Asymmetric Dilated Network for Semantic Segmentation

Efficient Semantic Segmentation via Lightweight Multiple-Information Interaction Network

Efficient Context Integration through Factorized Pyramidal Learning for Ultra-Lightweight Semantic Segmentation

LACTNet: A Lightweight Real-Time Semantic Segmentation Network Based on an Aggregated Convolutional Neural Network and Transformer

A Lightweight CNN–Transformer Network With Laplacian Loss for Low-Altitude UAV Imagery Semantic Segmentation

EFRNet: A Lightweight Network with Efficient Feature Fusion and Refinement for Real-Time Semantic Segmentation