[Harmful effects of common drugs on the visual apparatus. Anthelmintics].

L. Dralands

Abstract:

What problem does this paper attempt to address?

Hyperspectral Image Super-resolution via Deep Spatio-spectral Convolutional Neural Networks

Jin-Fan Hu,Ting-Zhu Huang,Liang-Jian Deng,Tai-Xiang Jiang,Gemine Vivone,Jocelyn Chanussot

DOI: https://doi.org/10.48550/arXiv.2005.14400

2020-05-29

Abstract:Hyperspectral images are of crucial importance in order to better understand features of different materials. To reach this goal, they leverage on a high number of spectral bands. However, this interesting characteristic is often paid by a reduced spatial resolution compared with traditional multispectral image systems. In order to alleviate this issue, in this work, we propose a simple and efficient architecture for deep convolutional neural networks to fuse a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI), yielding a high-resolution hyperspectral image (HR-HSI). The network is designed to preserve both spatial and spectral information thanks to an architecture from two folds: one is to utilize the HR-HSI at a different scale to get an output with a satisfied spectral preservation; another one is to apply concepts of multi-resolution analysis to extract high-frequency information, aiming to output high quality spatial details. Finally, a plain mean squared error loss function is used to measure the performance during the training. Extensive experiments demonstrate that the proposed network architecture achieves best performance (both qualitatively and quantitatively) compared with recent state-of-the-art hyperspectral image super-resolution approaches. Moreover, other significant advantages can be pointed out by the use of the proposed approach, such as, a better network generalization ability, a limited computational burden, and a robustness with respect to the number of training samples.

Image and Video Processing,Computer Vision and Pattern Recognition
Progressive split-merge super resolution for hyperspectral imagery with group attention and gradient guidance

Yadong Li,Zhenhong Du,Sensen Wu,Yuanyuan Wang,Zhongyi Wang,Xianwei Zhao,Feng Zhang

DOI: https://doi.org/10.1016/j.isprsjprs.2021.09.023

IF: 12.7

2021-12-01

ISPRS Journal of Photogrammetry and Remote Sensing

Abstract:Hyperspectral image (HSI) has been an important valuable information source for many remote sensing applications due to its rich spectral information. However, limited by existing imaging system, the spatial resolution of HSI greatly limits its practical applications. In this paper, in order to deal with the challenge of feature extraction and super-resolution of HSI, a progressive split-merge super-resolution (PSMSR) framework is proposed to overcome the inherent resolution limitations, which apply a multi-level split strategy from the task level, spectral level and feature level. Within this framework, a gradient-guided group-attention network (GGAN) is designed to extract the spatial-spectral features and reconstruct realistic texture, where group attention block (GAB) aims to increase the distinguishing ability of spectral channels, and gradient information is fully fused in the reconstruction process to promote sharp edges and realistic texture. Compared with current state-of-the-art CNN-based SR methods, the proposed method achieves significant improvement in six evaluation metrics and visual quality on multiple and diverse scenes, and it obtains a higher classification accuracy in classification experiment and gets better SR results on real data, which proves that our method can effectively improve the spatial resolution while preserving the spectral correlation. In addition, a series of ablation experiments prove the effectiveness of each component of our method.

imaging science & photographic technology,remote sensing,geography, physical,geosciences, multidisciplinary
Remote Sensing Hyperspectral Image Super-Resolution via Multidomain Spatial Information and Multiscale Spectral Information Fusion

Chi Chen,Yongcheng Wang,Yuxi Zhang,Zhikang Zhao,Hao Feng

DOI: https://doi.org/10.1109/tgrs.2024.3388531

IF: 8.2

2024-04-27

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Hyperspectral image (HSI) super-resolution (SR) technology has made remarkable progress due to the development of deep learning (DL). However, the technique still faces two challenges, i.e., the imbalance between spectral and spatial information extraction, and the parameter deviation and high computational effort associated with 3-D convolution. In this article, we propose an SR method for remote sensing HSIs based on multidomain spatial information and multiscale spectral information fusion (MSSR). Specifically, inspired by the high degree of self-similarity of remote sensing HSIs, a spatial–spectral attention module based on dilated convolution (DSSA) for capturing global spatial information is proposed. The extraction of local spatial information is then accomplished by residual blocks using small-size convolution kernels. Meanwhile, we propose the 3-D inception module to efficiently mine multiscale spectral information. The module only retains the scale of the 3-D convolution kernel in spectral dimension, which greatly reduces the high computational cost caused by 3-D convolution. Comparative experimental results on four benchmark datasets demonstrate that compared with the current cutting-edge models, our method achieves state-of-the-art (SOTA) results and the model computation is greatly reduced.

imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics
Spectral Response Function-Guided Deep Optimization-Driven Network for Spectral Super-Resolution

Jiang He,Jie Li,Qiangqiang Yuan,Huanfeng Shen,Liangpei Zhang

DOI: https://doi.org/10.1109/tnnls.2021.3056181

IF: 14.255

2021-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Hyperspectral images (HSIs) are crucial for many research works. Spectral super-resolution (SSR) is a method used to obtain high-spatial-resolution (HR) HSIs from HR multispectral images. Traditional SSR methods include model-driven algorithms and deep learning. By unfolding a variational method, this article proposes an optimization-driven convolutional neural network (CNN) with a deep spatial–spectral prior, resulting in physically interpretable networks. Unlike the fully data-driven CNN, auxiliary spectral response function (SRF) is utilized to guide CNNs to group the bands with spectral relevance. In addition, the channel attention module (CAM) and the reformulated spectral angle mapper loss function are applied to achieve an effective reconstruction model. Finally, experiments on two types of data sets, including natural and remote sensing images, demonstrate the spectral enhancement effect of the proposed method, and also, the classification results on the remote sensing data set verified the validity of the information enhanced by the proposed method.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
Hyperspectral Image Super-Resolution via Deep Spatiospectral Attention Convolutional Neural Networks

Jin-Fan Hu,Ting-Zhu Huang,Liang-Jian Deng,Tai-Xiang Jiang,Gemine Vivone,Jocelyn Chanussot

DOI: https://doi.org/10.1109/tnnls.2021.3084682

IF: 14.255

2021-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Hyperspectral images (HSIs) are of crucial importance in order to better understand features from a large number of spectral channels. Restricted by its inner imaging mechanism, the spatial resolution is often limited for HSIs. To alleviate this issue, in this work, we propose a simple and efficient architecture of deep convolutional neural networks to fuse a low-resolution HSI (LR-HSI) and a high-resolution multispectral image (HR-MSI), yielding a high-resolution HSI (HR-HSI). The network is designed to preserve both spatial and spectral information thanks to a new architecture based on: 1) the use of the LR-HSI at the HR-MSI's scale to get an output with satisfied spectral preservation and 2) the application of the attention and pixelShuffle modules to extract information, aiming to output high-quality spatial details. Finally, a plain mean squared error loss function is used to measure the performance during the training. Extensive experiments demonstrate that the proposed network architecture achieves the best performance (both qualitatively and quantitatively) compared with recent state-of-the-art HSI super-resolution approaches. Moreover, other significant advantages can be pointed out by the use of the proposed approach, such as a better network generalization ability, a limited computational burden, and the robustness with respect to the number of training samples. Please find the source code and pretrained models from https://liangjiandeng.github.io/Projects_Res/HSRnet_2021tnnls.html.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
Spectral Super-Resolution via Model-Guided Cross-Fusion Network

Renwei Dian,Tianci Shan,Wei He,Haibo Liu

DOI: https://doi.org/10.1109/TNNLS.2023.3238506

Abstract:Spectral super-resolution, which reconstructs a hyperspectral image (HSI) from a single red-green-blue (RGB) image, has acquired more and more attention. Recently, convolution neural networks (CNNs) have achieved promising performance. However, they often fail to simultaneously exploit the imaging model of the spectral super-resolution and complex spatial and spectral characteristics of the HSI. To tackle the above problems, we build a novel cross fusion (CF)-based model-guided network (called SSRNet) for spectral super-resolution. In specific, based on the imaging model, we unfold the spectral super-resolution into the HSI prior learning (HPL) module and imaging model guiding (IMG) module. Instead of just modeling one kind of image prior, the HPL module is composed of two subnetworks with different structures, which can effectively learn the complex spatial and spectral priors of the HSI, respectively. Furthermore, a CF strategy is used to establish the connection between the two subnetworks, which further improves the learning performance of the CNN. The IMG module results in solving a strong convex optimization problem, which adaptively optimizes and merges the two features learned by the HPL module by exploiting the imaging model. The two modules are alternately connected to achieve optimal HSI reconstruction performance. Experiments on both the simulated and real data demonstrate that the proposed method can achieve superior spectral reconstruction results with relatively small model size. The code will be available at https://github.com/renweidian.
Hyperspectral Image Super-Resolution via Dual-domain Network Based on Hybrid Convolution

Tingting Liu,Yuan Liu,Chuncheng Zhang,Yuan Liyin,Xiubao Sui,Qian Chen

2023-07-14

Abstract:Since the number of incident energies is limited, it is difficult to directly acquire hyperspectral images (HSI) with high spatial resolution. Considering the high dimensionality and correlation of HSI, super-resolution (SR) of HSI remains a challenge in the absence of auxiliary high-resolution images. Furthermore, it is very important to extract the spatial features effectively and make full use of the spectral information. This paper proposes a novel HSI super-resolution algorithm, termed dual-domain network based on hybrid convolution (SRDNet). Specifically, a dual-domain network is designed to fully exploit the spatial-spectral and frequency information among the hyper-spectral data. To capture inter-spectral self-similarity, a self-attention learning mechanism (HSL) is devised in the spatial domain. Meanwhile the pyramid structure is applied to increase the acceptance field of attention, which further reinforces the feature representation ability of the network. Moreover, to further improve the perceptual quality of HSI, a frequency loss(HFL) is introduced to optimize the model in the frequency domain. The dynamic weighting mechanism drives the network to gradually refine the generated frequency and excessive smoothing caused by spatial loss. Finally, In order to better fully obtain the mapping relationship between high-resolution space and low-resolution space, a hybrid module of 2D and 3D units with progressive upsampling strategy is utilized in our method. Experiments on a widely used benchmark dataset illustrate that the proposed SRDNet method enhances the texture information of HSI and is superior to state-of-the-art methods.

Computer Vision and Pattern Recognition,Image and Video Processing
Hyperspectral Image Super-Resolution via Intrafusion Network

Jing Hu,Xiuping Jia,Yunsong Li,Gang He,Minghua Zhao

DOI: https://doi.org/10.1109/tgrs.2020.2982940

IF: 8.2

2020-10-01

IEEE Transactions on Geoscience and Remote Sensing

Abstract:This article presents an intrafusion network (IFN) for hyperspectral image (HSI) super-resolution (SR). Given that the HSI is a 3-D data cube with both the spatial information and the spectral information, the key challenge to construct HSI SR is how to efficiently exploit the spectral information among consecutive low-resolution (LR) bands, besides the spatial information. The proposed IFN consists of three modules, including the spectral difference module, the parallel convolution module, and the intrafusion module, which directly utilizes both the spatial information and the spectral information for reconstructing the high-resolution HSI. Different from most of the existed methods that tackle the spatial and spectral information separately, the proposed spatial–spectral utilization is achieved in one integrated network, which opens up a new way for HSI SR. Meanwhile, applications of this three modules strategy (first spectral difference, then parallel convolution, and finally, intrafusion) on both the conventional convolutional neural network and the residual network with deeper depth have shown the generalization capacity of this proposal. Experimental results and data analysis demonstrate the effectiveness of the proposed method using three hyperspectral data sets.

imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics
Deep Unsupervised Fusion Learning for Hyperspectral Image Super Resolution

Zhe Liu,Yinqiang Zheng,Xian-Hua Han

DOI: https://doi.org/10.3390/s21072348

IF: 3.9

2021-03-28

Sensors

Abstract:Hyperspectral image (HSI) super-resolution (SR) is a challenging task due to its ill-posed nature, and has attracted extensive attention by the research community. Previous methods concentrated on leveraging various hand-crafted image priors of a latent high-resolution hyperspectral (HR-HS) image to regularize the degradation model of the observed low-resolution hyperspectral (LR-HS) and HR-RGB images. Different optimization strategies for searching a plausible solution, which usually leads to a limited reconstruction performance, were also exploited. Recently, deep-learning-based methods evolved for automatically learning the abundant image priors in a latent HR-HS image. These methods have made great progress for HS image super resolution. Current deep-learning methods have faced difficulties in designing more complicated and deeper neural network architectures for boosting the performance. They also require large-scale training triplets, such as the LR-HS, HR-RGB, and their corresponding HR-HS images for neural network training. These training triplets significantly limit their applicability to real scenarios. In this work, a deep unsupervised fusion-learning framework for generating a latent HR-HS image using only the observed LR-HS and HR-RGB images without previous preparation of any other training triplets is proposed. Based on the fact that a convolutional neural network architecture is capable of capturing a large number of low-level statistics (priors) of images, the automatic learning of underlying priors of spatial structures and spectral attributes in a latent HR-HS image using only its corresponding degraded observations is promoted. Specifically, the parameter space of a generative neural network used for learning the required HR-HS image to minimize the reconstruction errors of the observations using mathematical relations between data is investigated. Moreover, special convolutional layers for approximating the degradation operations between observations and the latent HR-HS image are specifically to construct an end-to-end unsupervised learning framework for HS image super-resolution. Experiments on two benchmark HS datasets, including the CAVE and Harvard, demonstrate that the proposed method can is capable of producing very promising results, even under a large upscaling factor. Furthermore, it can outperform other unsupervised state-of-the-art methods by a large margin, and manifests its superiority and efficiency.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Diffused Convolutional Neural Network for Hyperspectral Image Super-Resolution

Sen Jia,Shuangzhao Zhu,Zhihao Wang,Meng Xu,Weixi Wang,Yujuan Guo

DOI: https://doi.org/10.1109/tgrs.2023.3250640

IF: 8.2

2023-03-15

IEEE Transactions on Geoscience and Remote Sensing

Abstract:With the rapid development of deep convolutional neural networks (CNNs), super-resolution (SR) in hyperspectral image (HSI) has achieved good results. Current methods generally use 2-D convolution for feature extraction, but they cannot effectively extract spectral information. Although 3-D convolution can better characterize feature structure of HSI, it will lead to parameter redundancy, model complexity, and severe memory shortage. To address the above problems, we propose a new HSI SR method, named diffused CNN (DCNN). Specifically, spectral convolutions have been added into the enhanced convolutional neural (ECN) block, and a series of spectral convolutions are introduced in the residual network to learn features in the channel direction of different depths. Furthermore, histogram of oriented gradient (HOG) and local binary pattern (LBP) are used to retain the shape and texture information of the image, respectively, which can well represent the spatial structure of the object. To effectively make use of the extracted shallow and deep features, a feature fusion strategy is used to reinforce the reconstruction efficiency. Besides, an image enhancement module has been developed to diffuse the SR image into the image space. Extensive evaluations and comparisons show that our DCNN approach can not only recover the HSI data with richer details but also achieve superiority over several state-of-the-art methods.

imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics
Unsupervised Blind Spectral-spatial Cross-super-resolution Network for HSI and MSI Fusion

Huajing Wu,Kefei Zhang,Suqin Wu,Xuexi Liu,Longjiang Li,Shuangshuang Shi

DOI: https://doi.org/10.1109/tgrs.2024.3362862

IF: 8.2

2024-01-01

IEEE Transactions on Geoscience and Remote Sensing

Abstract:A high-spatial-resolution hyperspectral image (HR-HSI) can be obtained by fusing a hyperspectral image and a multispectral image (HSI-MSI) since it takes the advantage of combing the low-spatial-resolution hyperspectral image (LR-HSI) and the high-spatial-resolution multispectral image (HR-MSI). Currently, most HSI-MSI fusion methods decompose HSIs in both spatial and spectral domains, which destroys the correlation of the two domains and results in poor fusion results. To further explore the correlation of spatial and spectral information, an unsupervised blind spectral-spatial cross-super-resolution network (CSRNet) is proposed for HSI-MSI fusion. The network transforms the fusion into two super-resolution reconstructions for a HR-HSI to avoid the decomposition of HSIs and achieves interaction with the two reconstructions to provide strong constraints for each other. The network uses the following procedure. First, the HSI-MSI fusion is transformed into a spectral-spatial cross-super-resolution model, which consists two branches: one for spatial and the other for spectral super-resolution reconstructions. As the basic blocks of the two branches, spatial super-resolution (SpaSR) and the spectral super-resolution (SpeSR) blocks in the new network are designed via iterations unfolding from half quadratic splitting (HQS). Then their corresponding spatial constraint (SpaC) and spectral constraint (SpeC) modules are established for the mutual constraints and interactions between the two branches. The SpaSR/SpeSR and SpaC/SpeC modules are alternately connected to form the above spatial/spectral super-resolution branches for reconstructing a HR-HSI. Both visual and quantitative results of experiments based on both simulated and real observation datasets showed that the proposed method outperformed seven commonly used methods, suggesting that the new method is effective for preserving the correlation of spatial and spectral information in HR-HSIs.

imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics
A Spectral Grouping and Attention-Driven Residual Dense Network for Hyperspectral Image Super-Resolution

Denghong Liu,Jie Li,Qiangqiang Yuan

DOI: https://doi.org/10.1109/tgrs.2021.3049875

IF: 8.2

2021-09-01

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Although unprecedented success has been achieved in convolutional neural network (CNN)-based super-resolution (SR) for natural images, hyperspectral image (HSI) SR without auxiliary high-resolution images remains a challenging task due to the high spectral dimensionality, where learning effective spatial and spectral representations is of great importance. In this article, we introduce a novel CNN-based HSI SR method, termed spectral grouping and attention-driven residual dense network (SGARDN) to facilitate the modeling of all spectral bands and focus on the exploration of spatial-spectral features. Considering the block characteristic of HSI, we employ group convolutions in and between groups composed of highly similar spectral bands at early stages to extract informative spatial features and avoid spectral disorder caused by normal convolution. To exploit spectral prior, a new spectral attention mechanism constructed by covariance statistics of features is designed to adaptively recalibrate features. We adapt the spectral attention for group convolutions to rescale grouping features with holistic spectral information. These two sequential operations called spectral grouping and integration module aim to extract effective shallow spatial-spectral features that are reused in the following layers. On the other hand, the residual dense block can better deal with spatial–spectral features by experimental comparison and hence is combined with the spectral attention to form a new basic building block for powerful feature expression and spectral correlation learning. The experimental results on synthesized and real-scenario HSIs demonstrate the feasibility and superiority of the proposed method over other state-of-the-art methods.

imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics
Spatial-spectral dual path hyperspectral image super-resolution reconstruction network based on spectral response functions

Yinghao Xu,Xi Jiang,Junyi Hou,Yuanyuan Sun,Xijun Zhu

DOI: https://doi.org/10.1080/10106049.2022.2157497

IF: 3.45

2022-12-22

Geocarto International

Abstract:Recently, hyperspectral image (HSI) super-resolution (SR) techniques based on deep learning have been actively developed. However, most hyperspectral image super-resolution reconstruction methods usually use all spectral bands simultaneously, leading to a mismatch of spectral properties between reconstructed HSI bands. Therefore, we proposed a new method of spatial-spectral dual path residual network (SGDPRN) based on spectral response function (SRF) to address the above problem. The SGDPRN is composed of the SRF guided grouping part, the spatial-spectral feature extraction part, and the final reconstruction part. Firstly, the reconstructed features for different spectral ranges are identified separately using SRF as a guide. Then, based on the grouping results, a spatial-spectral dual-path residual block is used to explore the spatial and spectral features by the designed parallel structure simultaneously. The spatial path is designed to extract sharp edges and realistic textures, and the spectral path is designed to model inter-spectral correlations to refine spectral features. At last, the third block of SGDPRN concatenates features of all groups and finishes the reconstruction of HSISR. QUST-1 satellite images have been applied in experiments, and the results showed that SGDPRN produced a higher peak signal to noise ratio, structural similarity metric, correlation coefficient, and lower spectral angle mapper, root mean square error than the other methods. This demonstrates that our method can effectively maintain the correlation of spectral bands while improving the spatial resolution.

geosciences, multidisciplinary,environmental sciences,remote sensing,imaging science & photographic technology
SSR-NET: Spatial–Spectral Reconstruction Network for Hyperspectral and Multispectral Image Fusion

Wei Huang,Xuelong Li,Xueting Zhang,Qi Wang

DOI: https://doi.org/10.1109/TGRS.2020.3018732

IF: 8.2

2021-07-01

IEEE Transactions on Geoscience and Remote Sensing

Abstract:The fusion of a low-spatial-resolution hyperspectral image (HSI) (LR-HSI) with its corresponding high-spatial-resolution multispectral image (MSI) (HR-MSI) to reconstruct a high-spatial-resolution HSI (HR-HSI) has been a significant subject in recent years. Nevertheless, it is still difficult to achieve the cross-mode information fusion of spatial mode and spectral mode when reconstructing HR-HSI for the existing methods. In this article, based on a convolutional neural network (CNN), an interpretable spatial–spectral reconstruction network (SSR-NET) is proposed for more efficient HSI and MSI fusion. More specifically, the proposed SSR-NET is a physical straightforward model that consists of three components: 1) cross-mode message inserting (CMMI); this operation can produce the preliminary fused HR-HSI, preserving the most valuable information of LR-HSI and HR-MSI; 2) spatial reconstruction network (SpatRN); the SpatRN concentrates on reconstructing the lost spatial information of LR-HSI with the guidance of spatial edge loss ( $\mathcal {L}_{\mathrm{ spat}}$ ); and 3) spectral reconstruction network (SpecRN); the SpecRN pays attention to reconstruct the lost spectral information of HR-MSI under the constraint of spatial edge loss ( $\mathcal {L}_{\mathrm{ spec}}$ ). Comparative experiments are conducted on six HSI data sets of Urban, Pavia University (PU), Pavia Center (PC), Botswana, Indian Pines (IP), and Washington DC Mall (WDCM), and the proposed SSR-NET achieves the superior or competitive results in comparison with seven state-of-the-art methods. The code of SSR-NET is available at https://github.com/hw2hwei/SSRNET.

Computer Science,Engineering,Environmental Science
Hyperspectral Reconstruction From RGB Images via Physically Guided Graph Deep Prior Learning

Haifeng Xu,Fang Liu,Jia Liu,Liang Xiao,Tian Lin,Jingxiang Yang

DOI: https://doi.org/10.1109/TGRS.2024.3471616

IF: 8.2

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Recovering the latent hyperspectral image (HSI) from RGB or multispectral image (MSI), which is dubbed spectral super-resolution (SSR), has demonstrated outstanding performance owing to the advancements in convolutional neural networks (CNNs). However, most of the current algorithms concentrate on the pursuit of networks with more expensive or complex structures, while ignoring the significant role of physical degradation models in SSR. In addition, the inherent defects of CNN make these networks focus more on the local correlation, while their ability to model the long-range correlations in the spectral and spatial domains still has room to improve. To overcome this shortcoming, we propose a physical degradation-guided deep prior learning network (PGDL-Net) for SSR via unfolding the optimization process of the blind SSR model, in which the priors of unknown spectral response function (SRF) and latent HSI are learned explicitly and represented by proximal operators. To jointly extract the local and non-local information, we design a hybrid graph Transformer as the proximal operator to solve the latent HSI. Furthermore, to ensure efficient learning of SRF and HSI, we also propose a novel loss function constraining the reconstruction error, degradation consistency, and observation fidelity for the learned SRF and HSI. Experimental results on multiple datasets illustrate the improved performance and stability of our method in SSR.

Environmental Science,Engineering,Computer Science
Remote Sensing Image Super-Resolution Using Enriched Spatial-Channel Feature Aggregation Networks

Shuai Hao,Yunzhi Zhuge,Jia Xu,Huchuan Lu,You He

DOI: https://doi.org/10.1109/docs63458.2024.10704496

2024-01-01

Abstract:Single Image Super-Resolution (SISR) pertains to the intricate task of recovering high-fidelity, high-resolution images, a process highly germane to remote sensing applications. Conventional Convolutional Neural Network (CNN) paradigms have achieved commendable results through the incorporation of residual architectures and attention mechanisms. However, these approaches are mired by excessive localization and information attrition due to simplistic stacking of residual structures and naive embedding of attention modules. This work introduces a sophisticated Transformer-based architecture, specifically tai-lored for super-resolution in remote sensing imagery, adept at harnessing high-dimensional spatial and channel features. The crux of this methodology is the integration of Local Enhanced Transformer layers and Global Transformer layers, adept at cap-turing multiscale features across spatial domains. Additionally, the Residual Channel Attention Block (RCAB) is employed for channel-wise feature calibration. Central to our approach is the introduction of a Spatial and Channel Attention fusion(SCF) module, ingeniously designed to exploit the synergies between spatial and channel information. Comprehensive experimentation substantiates the efficacy of the proposed SCANet, outstripping the performance of contemporary state-of-the-art alternatives in super-resolution tasks.
Local–Global Feature Fusion Network for Efficient Hyperspectral Image Super-Resolution

Jingran Xu,Jiankang Zhao,Chao Cui

DOI: https://doi.org/10.1109/lgrs.2024.3358368

IF: 5.343

2024-02-10

IEEE Geoscience and Remote Sensing Letters

Abstract:Numerous hyperspectral image (HSI) super-resolution (SR) approaches have been proposed and attained remarkable performance recently. However, the enormous computational and memory costs of the deeper and heavier networks make the application challenging in the actual scene. Therefore, a lightweight but efficient local–global feature fusion module network (LGFFMN) is proposed to reduce the computational burden while achieving superior reconstruction performance. In particular, a module local feature fusion module (LFFM) is proposed to use convolutional neural network (CNN) for effective local feature extraction. Meanwhile, to exploit global information and avoid the computational burden of traditional transformers, a multiscale representation-based feature modulation mechanism is adopted to build a vision transformer (ViT)-like block global feature fusion module (GFFM). To reduce the high dimension of HSI while using the higher similarity among adjacent bands, the overall processing manner is group by group. Experiments demonstrate that our LGFFMN achieves a satisfactory balance in SR reconstruction performance and efficiency metrics and is highly competitive compared with other state-of-the-art HSI SR methods.

imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics
Hyperspectral Image Super-Resolution with Self-Supervised Spectral-Spatial Residual Network

Wenjing Chen,Xiangtao Zheng,Xiaoqiang Lu

DOI: https://doi.org/10.3390/rs13071260

IF: 5

2021-03-26

Remote Sensing

Abstract:Recently, many convolutional networks have been built to fuse a low spatial resolution (LR) hyperspectral image (HSI) and a high spatial resolution (HR) multispectral image (MSI) to obtain HR HSIs. However, most deep learning-based methods are supervised methods, which require sufficient HR HSIs for supervised training. Collecting plenty of HR HSIs is laborious and time-consuming. In this paper, a self-supervised spectral-spatial residual network (SSRN) is proposed to alleviate dependence on a mass of HR HSIs. In SSRN, the fusion of HR MSIs and LR HSIs is considered a pixel-wise spectral mapping problem. Firstly, this paper assumes that the spectral mapping between HR MSIs and HR HSIs can be approximated by the spectral mapping between LR MSIs (derived from HR MSIs) and LR HSIs. Secondly, the spectral mapping between LR MSIs and LR HSIs is explored by SSRN. Finally, a self-supervised fine-tuning strategy is proposed to transfer the learned spectral mapping to generate HR HSIs. SSRN does not require HR HSIs as the supervised information in training. Simulated and real hyperspectral databases are utilized to verify the performance of SSRN.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary
GuidedNet: A General CNN Fusion Framework via High-Resolution Guidance for Hyperspectral Image Super-Resolution

Ran Ran,Liang-Jian Deng,Tai-Xiang Jiang,Jin-Fan Hu,Jocelyn Chanussot,Gemine Vivone

DOI: https://doi.org/10.1109/tcyb.2023.3238200

IF: 11.8

2023-01-01

IEEE Transactions on Cybernetics

Abstract:Hyperspectral image super-resolution (HISR) is about fusing a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI) to generate a high-resolution hyperspectral image (HR-HSI). Recently, convolutional neural network (CNN)-based techniques have been extensively investigated for HISR yielding competitive outcomes. However, existing CNN-based methods often require a huge amount of network parameters leading to a heavy computational burden, thus, limiting the generalization ability. In this article, we fully consider the characteristic of the HISR, proposing a general CNN fusion framework with high-resolution guidance, called GuidedNet. This framework consists of two branches, including 1) the high-resolution guidance branch (HGB) that can decompose the high-resolution guidance image into several scales and 2) the feature reconstruction branch (FRB) that takes the low-resolution image and the multiscaled high-resolution guidance images from the HGB to reconstruct the high-resolution fused image. GuidedNet can effectively predict the high-resolution residual details that are added to the upsampled HSI to simultaneously improve spatial quality and preserve spectral information. The proposed framework is implemented using recursive and progressive strategies, which can promote high performance with a significant network parameter reduction, even ensuring network stability by supervising several intermediate outputs. Additionally, the proposed approach is also suitable for other resolution enhancement tasks, such as remote sensing pansharpening and single-image super-resolution (SISR). Extensive experiments on simulated and real datasets demonstrate that the proposed framework generates state-of-the-art outcomes for several applications (i.e., HISR, pansharpening, and SISR). Finally, an ablation study and more discussions assessing, for example, the network generalization, the low computational cost, and the fewer network parameters, are provided to the readers. The code link is: https://github.com/Evangelion09/GuidedNet.

automation & control systems,computer science, cybernetics, artificial intelligence
Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery

Junjun Jiang,He Sun,Xianming Liu,Jiayi Ma

DOI: https://doi.org/10.1109/TCI.2020.2996075

2020-05-24

Abstract:Recently, single gray/RGB image super-resolution reconstruction task has been extensively studied and made significant progress by leveraging the advanced machine learning techniques based on deep convolutional neural networks (DCNNs). However, there has been limited technical development focusing on single hyperspectral image super-resolution due to the high-dimensional and complex spectral patterns in hyperspectral image. In this paper, we make a step forward by investigating how to adapt state-of-the-art residual learning based single gray/RGB image super-resolution approaches for computationally efficient single hyperspectral image super-resolution, referred as SSPSR. Specifically, we introduce a spatial-spectral prior network (SSPN) to fully exploit the spatial information and the correlation between the spectra of the hyperspectral data. Considering that the hyperspectral training samples are scarce and the spectral dimension of hyperspectral image data is very high, it is nontrivial to train a stable and effective deep network. Therefore, a group convolution (with shared network parameters) and progressive upsampling framework is proposed. This will not only alleviate the difficulty in feature extraction due to high-dimension of the hyperspectral data, but also make the training process more stable. To exploit the spatial and spectral prior, we design a spatial-spectral block (SSB), which consists of a spatial residual module and a spectral attention residual module. Experimental results on some hyperspectral images demonstrate that the proposed SSPSR method enhances the details of the recovered high-resolution hyperspectral images, and outperforms state-of-the-arts. The source code is available at \url{<a class="link-external link-https" href="https://github.com/junjun-jiang/SSPSR" rel="external noopener nofollow">this https URL</a>

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning,Multimedia

[Harmful effects of common drugs on the visual apparatus. Anthelmintics].

Hyperspectral Image Super-resolution via Deep Spatio-spectral Convolutional Neural Networks

Progressive split-merge super resolution for hyperspectral imagery with group attention and gradient guidance

Remote Sensing Hyperspectral Image Super-Resolution via Multidomain Spatial Information and Multiscale Spectral Information Fusion

Spectral Response Function-Guided Deep Optimization-Driven Network for Spectral Super-Resolution

Hyperspectral Image Super-Resolution via Deep Spatiospectral Attention Convolutional Neural Networks

Spectral Super-Resolution via Model-Guided Cross-Fusion Network

Hyperspectral Image Super-Resolution via Dual-domain Network Based on Hybrid Convolution

Hyperspectral Image Super-Resolution via Intrafusion Network

Deep Unsupervised Fusion Learning for Hyperspectral Image Super Resolution

Diffused Convolutional Neural Network for Hyperspectral Image Super-Resolution

Unsupervised Blind Spectral-spatial Cross-super-resolution Network for HSI and MSI Fusion

A Spectral Grouping and Attention-Driven Residual Dense Network for Hyperspectral Image Super-Resolution

Spatial-spectral dual path hyperspectral image super-resolution reconstruction network based on spectral response functions

SSR-NET: Spatial–Spectral Reconstruction Network for Hyperspectral and Multispectral Image Fusion

Hyperspectral Reconstruction From RGB Images via Physically Guided Graph Deep Prior Learning

Remote Sensing Image Super-Resolution Using Enriched Spatial-Channel Feature Aggregation Networks

Local–Global Feature Fusion Network for Efficient Hyperspectral Image Super-Resolution

Hyperspectral Image Super-Resolution with Self-Supervised Spectral-Spatial Residual Network

GuidedNet: A General CNN Fusion Framework via High-Resolution Guidance for Hyperspectral Image Super-Resolution

Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery