Abstract:Requirements of compressed sensing techniques targeted at miniaturized hyperspectral satellite applications include lightweight onboard hardware, high-speed sensing, low sampling rate for compressing the massive volume of typical hyperspectral data, and noise robustness for reliable data transmission to the ground station. We achieve all these aims via deep learning, and neural networks resulted from which can be implemented on-chip, thereby allowing light hardware implementation. Our neural networks were trained from small-scaled data, but, even so, the resulting encoder achieves a very low sampling rate and very high speed. Unlike typical network training, the input–output pairs are not square but stripe-like images, partly because compressed acquisition does not allow performing compression after obtaining complete data cube and partly because stripe-like acquisition well matches the popular pushbroom hyperspectral sensing schemes. Even with such hard restriction caused by nontraditional training, the resulting decoder still reconstructs the image with high accuracy. To match the requirement of pushbroom sensing, a lightweight encoder is proposed to compress the stripe-like images immediately. Meanwhile, multiscale feature fusion block (MFB) and aggregation (MFA) modules are proposed to form our decoder for enhancing the feature representation of the compressed acquisitions. Furthermore, we achieve joint spatial/spectral super-resolution (SR) progressively, ensuring accurate hyperspectral reconstruction via a low-rank-driven decoder. The encoder and decoder are trained in an end-to-end manner, where noise robustness is forced during the training stage. Comprehensive experiments demonstrate the superiority of the proposed hyperspectral compressed sensing method, as well as its one-shot transfer learning (OTL)-based extension, both quantitatively and qualitatively.

TinyPredNet: A Lightweight Framework for Satellite Image Sequence Prediction

Deep Neural Network Acceleration with Sparse Prediction Layers

Meteorological Satellite Images Prediction Based on Deep Multi-scales Extrapolation Fusion

PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning

PredFormer: Transformers Are Effective Spatial-Temporal Predictive Learners

PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs

SAM-Net: Spatio-Temporal Sequence Typhoon Cloud Image Prediction Net with Self-Attention Memory

Cloud Cover Prediction Model Using Multichannel Geostationary Satellite Images

MiniNet: An extremely lightweight convolutional neural network for real-time unsupervised monocular depth estimation

Satellite Image Prediction Relying on GAN and LSTM Neural Networks

Exploring and Exploiting High-Order Spatial-Temporal Dynamics for Long-Term Frame Prediction

Enhancing spatiotemporal predictive learning: an approach with nested attention module

DCSN: Deep Compressed Sensing Network for Efficient Hyperspectral Data Transmission of Miniaturized Satellite

PSRUNet: a recurrent neural network for spatiotemporal sequence forecasting based on parallel simple recurrent unit

FlatteNet: A Simple Versatile Framework for Dense Pixelwise Prediction

Deep-learning-based solution for data deficient satellite image segmentation

PredCNN: Predictive Learning with Cascade Convolutions

Exploring Spatial Frequency Information for Enhanced Video Prediction Quality

Learning Robust Precipitation Forecaster by Temporal Frame Interpolation

PredRANN: the Spatiotemporal Attention Convolution Recurrent Neural Network for Precipitation Nowcasting

PrevPredMap: Exploring Temporal Modeling with Previous Predictions for Online Vectorized HD Map Construction