Global-optimal semi-supervised learning for single-pixel image-free sensing

Xinrui Zhan,Hui Lu,Rong Yan,Liheng Bian

DOI: https://doi.org/10.1364/ol.511448

IF: 3.6

2024-01-30

Optics Letters

Abstract:Xinrui Zhan, Hui Lu, Rong Yan, Liheng Bian Single-pixel sensing offers low-cost detection and reliable perception, and the image-free sensing technique enhances its efficiency by ... [Opt. Lett. 49, 682-685 (2024)]

optics

What problem does this paper attempt to address?

Single-pixel Multispectral Imaging Based on Macropixel Segmentation Method

Qin Xie,Chenning Tao,Xinyu Liu,Yan Sun,Chang Wang,Jinlei Zhang,Zhenrong Zheng

DOI: https://doi.org/10.1117/12.2601360

2021-01-01

Abstract:Spectral imaging can capture both spatial and spectral data of a scene, providing an efficient technique for analysis and identification. To improve the efficiency of data acquisition, compressive sensing (CS) methods have been introduced into spectral imaging systems. In this work, we propose a novel macropixel segmentation method to realize effective and non-mechanical single-pixel multispectral imaging. A series of macropixel-based patterns are designed to modulate data cube of target object. Spatial light modulator (SLM) and multispectral filter array are utilized to generate such patterns. CS algorithm is used to recover data cube from 1-D signal acquired by a single-pixel detector. Alignment of binary patterns with the subareas of macropixel filter array is conducted in the experimental set-up. Without mechanical or dispersive structure, the proposed method holds great potential in miniaturization and integration of spectral imaging devices.
Compressive Single-Pixel Hyperspectral Imaging Using RGB Sensors

Chenning Tao,Huanzheng Zhu,Xucheng Wang,Shuhang Zheng,Qin Xie,Chang Wang,Rengmao Wu,Zhenrong Zheng

DOI: https://doi.org/10.1364/oe.416388

IF: 3.8

2021-01-01

Optics Express

Abstract:Hyperspectral imaging that obtains the spatial-spectral information of a scene has been extensively applied in various fields but usually requires a complex and costly system. A single-pixel detector based hyperspectral system mitigates the complexity problem but simultaneously brings new difficulties on the spectral dispersion device. In this work, we propose a low-cost compressive single-pixel hyperspectral imaging system with RGB sensors. Based on the structured illumination single-pixel imaging configuration, the lens-free system directly captures data by the RGB sensors without dispersion in the spectral dimension. The reconstruction is performed with a pre-trained spatial-spectral dictionary, and the hyperspectral images are obtained through compressive sensing. In addition, the spatial patterns for the structured illumination and the dictionary for the sparse representation are optimized by coherence minimization, which further improve the reconstruction quality. In both spatial and spectral dimensions, the intrinsic sparse properties of the hyperspectral images are made full use of for high sampling efficiency and low reconstruction cost. This work may introduce opportunities for optimization of computational imaging systems and reconstruction algorithms towards high speed, high resolution, and low cost future.
Spatial-spectral Encoding and Dictionary Optimization in Compressive Single-Pixel Hyperspectral Imaging Based on Mutual Coherence Minimization

Yang Zhang,Xinyu Liu,Zhou Xu,Qiangbo Zhang,Chang Wang,Zhenrong Zheng

DOI: https://doi.org/10.1364/oe.531917

IF: 3.8

2024-01-01

Optics Express

Abstract:A single-pixel detector based hyperspectral system provides an effective way to obtain the spatial-spectral information of target scenes. However, complex spectral dispersion and the substantial number of measurements not only increase the complexity of the system but also decrease the sampling efficiency and the reconstruction accuracy. In this paper, we propose a compressive sensing (CS) theory based single-pixel hyperspectral imaging system. Based on structured illumination, the spatial information is modulated by binary spatial patterns displayed on a liquid crystal on silicon (LCoS), while polarizing elements at specific angles, acting as a serious of filters, modulate the spectral dimension, effectively avoiding spectral dispersion. In terms of sampling efficiency, the application of CS significantly decreases the number of measurements required compared to the Nyquist-Shannon sampling theorem. Besides, to improve the reconstruction accuracy, mutual coherence minimization is employed to optimize the pre-trained dictionary, spatial patterns and filters. Furthermore, a two-step encoding method based on macro-pixel segmentation is proposed to address the issue of low resolution constrained by the size of the dictionary. Compared to the unoptimized system and dictionary, the proposed method achieves more accurate reconstruction results in both spectral and spatial dimensions. This work may provide opportunities for high-resolution single-pixel hyperspectral imaging systems based on CS. (c) 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
Low Light Enhancement by Unsupervised Network.

Yangyang Qu,Yongsheng Ou,Rong Xiong

DOI: https://doi.org/10.1109/rcar49640.2020.9303267

2020-01-01

Abstract:This paper focuses on unsupervised low-light enhancement methods for practical applications such as automated driving, surveillance cameras, and so on. This paper proposes an unsupervised method that is independent of paired samples and works well to solve the image darkness problem. Furthermore, to increase the brightness of the image and restore the information hidden in the dark, an auxiliary attention module is incorporated into our model. This proposed module can help generate images with good edges and make the image colors and details richer. The experimental results show that the proposed method works well for both image enhancement and noise control.
Low-Quality Sensor Data-Based Semi-Supervised Learning for Medical Image Segmentation

Hengfan Li,Xuanbo Xu,Ziheng Liu,Qingfeng Xia,Min Xia

DOI: https://doi.org/10.3390/s24237799

IF: 3.9

2024-12-06

Sensors

Abstract:Traditional medical image sensors face multiple challenges. First, these sensors typically rely on large amounts of labeled data, which are time-consuming and costly to obtain. Second, when the data volume and image size are large, traditional sensors have limited computational power, making it difficult to effectively train and infer models. Additionally, traditional sensors have poor generalization ability and struggle to adapt to datasets with different modalities. This paper devises a novel framework, named LSDSL, and deploys it in the sensor. LSDSL utilizes low-quality sensor data for semi-supervised learning in medical image segmentation. in supervised learning, we devise the hard region exploration (hre) module to enhance the model's comprehension of low-quality pixels in hard regions. in unsupervised learning, we introduce a pseudo-label sharing (ps) module, which allows low-quality pixels in one network to learn from the high-quality pixels in the other networks. our model outperforms other semi-supervised methods on the datasets of two different modalities (CT and MRI) in medical image sensors, achieving superior inference speed and segmentation accuracy.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Non-imaging single-pixel sensing with optimized binary modulation

Hao Fu,Liheng Bian,Jun Zhang

DOI: https://doi.org/10.1364/OL.395150

2019-09-27

Abstract:The conventional high-level sensing techniques require high-fidelity images as input to extract target features, which are produced by either complex imaging hardware or high-complexity reconstruction algorithms. In this letter, we propose single-pixel sensing (SPS) that performs high-level sensing directly from coupled measurements of a single-pixel detector, without the conventional image acquisition and reconstruction process. The technique consists of three steps including binary light modulation that can be physically implemented at $\sim$22kHz, single-pixel coupled detection owning wide working spectrum and high signal-to-noise ratio, and end-to-end deep-learning based sensing that reduces both hardware and software complexity. Besides, the binary modulation is trained and optimized together with the sensing network, which ensures least required measurements and optimal sensing accuracy. The effectiveness of SPS is demonstrated on the classification task of handwritten MNIST dataset, and 96.68% classification accuracy at $\sim$1kHz is achieved. The reported single-pixel sensing technique is a novel framework for highly efficient machine intelligence.

Image and Video Processing,Computer Vision and Pattern Recognition
Weighted sampling-adaptive single-pixel sensing

Xinrui Zhan,Chunli Zhu,Jinli Suo,Liheng Bian

DOI: https://doi.org/10.1364/OL.458311

IF: 3.6

2022-01-01

Optics Letters

Abstract:The novel single-pixel sensing technique that uses an end-to-end neural network for joint optimization achieves high-level semantic sensing, which is effective but computation-consuming for varied sampling rates. In this Letter, we report a weighted optimization technique for sampling-adaptive single-pixel sensing, which only needs to train the network once for any dynamic sampling rate. Specifically, we innovatively introduce a weighting scheme in the encoding process to characterize different patterns' modulation efficiencies, in which the modulation patterns and their corresponding weights are updated iteratively. The optimal pattern series with the highest weights is employed for light modulation in the experimental implementation, thus achieving highly efficient sensing. Experiments validated that once the network is trained with a sampling rate of 1, the single-target classification accuracy reaches up to 95.00% at a sampling rate of 0.03 on the MNIST dataset and 90.20% at a sampling rate of 0.07 on the CCPD dataset for multi-target sensing. (C) 2022 Optica Publishing Group
Optical Remote Sensing Image Understanding with Weak Supervision: Concepts, Methods, and Perspectives

Jun Yue,Leyuan Fang,Pedram Ghamisi,Weiying Xie,Jun Li,Jocelyn Chanussot,Antonio J Plaza

DOI: https://doi.org/10.1109/mgrs.2022.3161377

IF: 14.6

2022-01-01

IEEE Geoscience and Remote Sensing Magazine

Abstract:In recent years, supervised learning has been widely used in various tasks of optical remote sensing image (RSI) understanding, including RSI classification, pixel-wise segmentation, change detection, and object detection. The methods based on supervised learning need a large amount of high-quality training data, and their performance highly depends on the quality of the labels. However, in practical remote sensing applications, it is often expensive and time consuming to obtain large-scale data sets with high-quality labels, which leads to a lack of sufficient supervised information. In some cases, only coarse-grained labels can be obtained, resulting in the lack of exact supervision. In addition, the supervised information obtained manually may be wrong, resulting in a lack of accurate supervision. Therefore, RSI understanding often faces the problems of incomplete, inexact, and inaccurate supervised information, which will affect the breadth and depth of remote sensing applications.

geochemistry & geophysics,remote sensing,imaging science & photographic technology
Single-pixel imaging based on deep learning

Kai Song,Yaoxing Bian,Ku Wu,Hongrui Liu,Shuangping Han,Jiaming Li,Jiazhao Tian,Chengbin Qin,Jianyong Hu,Liantuan Xiao

2023-11-17

Abstract:Single-pixel imaging can collect images at the wavelengths outside the reach of conventional focal plane array detectors. However, the limited image quality and lengthy computational times for iterative reconstruction still impede the practical application of single-pixel imaging. Recently, deep learning has been introduced into single-pixel imaging, which has attracted a lot of attention due to its exceptional reconstruction quality, fast reconstruction speed, and the potential to complete advanced sensing tasks without reconstructing images. Here, this advance is discussed and some opinions are offered. Firstly, based on the fundamental principles of single-pixel imaging and deep learning, the principles and algorithms of single-pixel imaging based on deep learning are described and analyzed. Subsequently, the implementation technologies of single-pixel imaging based on deep learning are reviewed. They are divided into super-resolution single-pixel imaging, single-pixel imaging through scattering media, photon-level single-pixel imaging, optical encryption based on single-pixel imaging, color single-pixel imaging, and image-free sensing according to diverse application fields. Finally, major challenges and corresponding feasible approaches are discussed, as well as more possible applications in the future.

Image and Video Processing,Optics
Simple and Efficient: A Semisupervised Learning Framework for Remote Sensing Image Semantic Segmentation

Xu Liu,Puhua Chen,Zhixi Feng,Lingling Li,F. Liu,Licheng Jiao,Shuyuan Yang,Xiaoqiang Lu

DOI: https://doi.org/10.1109/TGRS.2022.3220755

IF: 8.2

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Semantic segmentation based on deep learning has achieved impressive results in recent years, but these results are supported by a large amount of labeled data, which requires intensive annotation at the pixel level, particularly for high-resolution remote sensing (RS) images. In this work, we propose a simple yet efficient semisupervised learning framework based on linear sampling (LS) self-training, named LSST, to improve the performance of RS image semantic segmentation. Specifically, the classical pseudolabeling-based self-training paradigm is enhanced by injecting strong data augmentations (SDAs) applicable to RS images, based on which a powerful baseline is constructed. Nevertheless, the problem of insufficient data training to generate pseudolabels with a high level of noise persists, and the noisy pseudolabels will continue to accumulate and impede model improvement during the retraining phase. Previous works commonly employ a predefined threshold to remove noise, but it will lead to overfitting the model to easily identified classes. To address it, a method using LS is presented for assigning thresholds to different classes in an adaptive manner, which provides noiseless regions for retraining. Experiments prove that the proposed pixelwise selection is more available for segmentation than image-level selection in RS images. Finally, LSST achieves state of the art on several datasets and different evaluation metrics. The source code of this article is available at https://github.com/xiaoqiang-lu/LSST.

Environmental Science,Computer Science
Image-free single-pixel segmentation

Haiyan Liu,Liheng Bian,Jun Zhang

DOI: https://doi.org/10.48550/arXiv.2108.10617

2021-08-24

Abstract:The existing segmentation techniques require high-fidelity images as input to perform semantic segmentation. Since the segmentation results contain most of edge information that is much less than the acquired images, the throughput gap leads to both hardware and software waste. In this letter, we report an image-free single-pixel segmentation technique. The technique combines structured illumination and single-pixel detection together, to efficiently samples and multiplexes scene's segmentation information into compressed one-dimensional measurements. The illumination patterns are optimized together with the subsequent reconstruction neural network, which directly infers segmentation maps from the single-pixel measurements. The end-to-end encoding-and-decoding learning framework enables optimized illumination with corresponding network, which provides both high acquisition and segmentation efficiency. Both simulation and experimental results validate that accurate segmentation can be achieved using two-order-of-magnitude less input data. When the sampling ratio is 1%, the Dice coefficient reaches above 80% and the pixel accuracy reaches above 96%. We envision that this image-free segmentation technique can be widely applied in various resource-limited platforms such as UAV and unmanned vehicle that require real-time sensing.

Image and Video Processing,Computer Vision and Pattern Recognition
Index Your Position: A Novel Self-Supervised Learning Method for Remote Sensing Images Semantic Segmentation

Dilxat Muhtar,Xueliang Zhang,Pengfeng Xiao

DOI: https://doi.org/10.1109/tgrs.2022.3177770

IF: 8.2

2022-01-01

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Learning effective visual representations without human supervision is a critical problem for the task of semantic segmentation of remote sensing images (RSIs), where pixel-level annotations are difficult to obtain. Self-supervised learning (SSL), which learns useful representations by creating artificial supervised learning problems, has recently emerged as an effective method to learn from unlabelled data. Current SSL methods are generally trained on ImageNet through image-level prediction tasks. We argue that this is suboptimal for application in semantic segmentation of RSIs since it does not take into account spatial position information between objects, which is critical for the segmentation of RSIs characterized by multiobject. In this study, we propose a novel self-supervised dense representation learning method, IndexNet, for the semantic segmentation of RSIs. On the one hand, considering the multiobject characteristics of RSIs, IndexNet learns pixel-level representations by tracking object positions, while maintaining sensitivity to object position changes to ensure that no mismatches are caused. On the other hand, by combining image-level contrast and pixel-level contrast, IndexNet can learn spatiotemporal invariant features. Experimental results show that our method works better than ImageNet pretraining and outperforms state-of-the-art (SOTA) SSL methods. Code and pretrained models will be available at https://github.com/pUmpKin-Co/offical-IndexNet.
Optical Machine Learning with Single-pixel Imaging

Shuming Jiao,Jun Feng,Yang Gao,Ting Lei,Zhenwei Xie,Xiaocong Yuan

DOI: https://doi.org/10.1364/3d.2020.jw2a.43

2020-01-01

Abstract:An optical learning machine is designed based on single-pixel imaging, which can perform pattern recognition full-optically without object image reconstruction. It has advantages of incoherent illumination, low experimental complexity and being programmable.
Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning

Siyi Deng,Yang Ning,Jiwei Zhao,Heping Zhang

DOI: https://doi.org/10.48550/arXiv.2011.14185

2023-03-18

Abstract:We consider the estimation problem in high-dimensional semi-supervised learning. Our goal is to investigate when and how the unlabeled data can be exploited to improve the estimation of the regression parameters of linear model in light of the fact that such linear models may be misspecified in data analysis. We first establish the minimax lower bound for parameter estimation in the semi-supervised setting, and show that this lower bound cannot be achieved by supervised estimators using the labeled data only. We propose an optimal semi-supervised estimator that can attain this lower bound and therefore improves the supervised estimators, provided that the conditional mean function can be consistently estimated with a proper rate. We further propose a safe semi-supervised estimator. We view it safe, because this estimator is always at least as good as the supervised estimators. We also extend our idea to the aggregation of multiple semi-supervised estimators caused by different misspecifications of the conditional mean function. Extensive numerical simulations and a real data analysis are conducted to illustrate our theoretical results.

Methodology,Statistics Theory,Machine Learning
Low-Shot Learning for the Semantic Segmentation of Remote Sensing Imagery

Ronald Kemker,Ryan Luu,Christopher Kanan

DOI: https://doi.org/10.48550/arXiv.1803.09824

2018-03-26

Computer Vision and Pattern Recognition

Abstract:Recent advances in computer vision using deep learning with RGB imagery (e.g., object recognition and detection) have been made possible thanks to the development of large annotated RGB image datasets. In contrast, multispectral image (MSI) and hyperspectral image (HSI) datasets contain far fewer labeled images, in part due to the wide variety of sensors used. These annotations are especially limited for semantic segmentation, or pixel-wise classification, of remote sensing imagery because it is labor intensive to generate image annotations. Low-shot learning algorithms can make effective inferences despite smaller amounts of annotated data. In this paper, we study low-shot learning using self-taught feature learning for semantic segmentation. We introduce 1) an improved self-taught feature learning framework for HSI and MSI data and 2) a semi-supervised classification algorithm. When these are combined, they achieve state-of-the-art performance on remote sensing datasets that have little annotated training data available. These low-shot learning frameworks will reduce the manual image annotation burden and improve semantic segmentation performance for remote sensing imagery.
Discrete Cosine Single-Pixel Salient Object Detection Base On Deep Learning Via Fast Binary Illumination

Yonghao Li,Jianhong Shi,Lei Sun,Xiaoyan Wu,Guihua Zeng,Jun Sun,Lulu Tian,Feng Su

DOI: https://doi.org/10.1364/CLEO_AT.2020.JTH2E.33

2020-01-01

Abstract:Discrete cosine single-pixel imaging with binary illumination can fastly reconstruct a scene where area of the salient object can be detected rapidly and accurately even in the case of undersampling, utilizing deep learning model. (C) 2020 The Author(s)
Learning distance metric for semi-supervised image segmentation

Yangqing Jia,Changshui Zhang

DOI: https://doi.org/10.1109/ICIP.2008.4712477

2008-01-01

ICIP

Abstract:Semi-supervised image segmentation is an important issue in many image processing applications, and has been a popular research area recently, the most popular are graph-based methods. However, parameter selection in these methods is still largely heuristic. In this paper, we introduce distance metric learning into graph-based semi-supervised segmentation to automatically obtain good results for images with different appearances. We first derive the optimization problem with respect to the distance metric as well as the segmentation labels, and use gradient descent method to find a local optimum solution. Experiments on general images and the fungal disease analysis application have shown that our method provides a steady performance under casual user annotations and different image appearances.
Single-pixel imaging based on metasurface fuzzy coding

Yusheng Liu,Yan Shi,Yingying Hu,Yadong Zhou,Rui Xu,Chunlian Zhan

DOI: https://doi.org/10.1364/ao.504410

IF: 1.9

2024-01-13

Applied Optics

Abstract:Yusheng Liu, Yan Shi, Yingying Hu, Yadong Zhou, Rui Xu, Chunlian Zhan Single-pixel imaging, renowned for its high sensitivity, robustness against interference, and superior resolution, has become increasingly ... [Appl. Opt. 63, 549-556 (2024)]

optics
Rapid and brief communication: An algorithm for semi-supervised learning in image retrieval

Ke Lu,Jidong Zhao,Deng Cai

DOI: https://doi.org/10.1016/j.patcog.2005.11.009

2006-01-01

Abstract:AbstractWe study the problem of image retrieval based on semi-supervised learning. Semi-supervised learning has attracted a lot of attention in recent years. Different from traditional supervised learning. Semi-supervised learning makes use of both labeled and unlabeled data. In image retrieval, collecting labeled examples costs human efforts, while vast amounts of unlabeled data are often readily available and offer some additional information. In this paper, based on support vector machine (SVM), we introduce a semi-supervised learning method for image retrieval. The basic consideration of the method is that, if two data points are close to each, they should share the same label. Therefore, it is reasonable to search a projection with maximal margin and locality preserving property. We compare our method to standard SVM and transductive SVM. Experimental results show efficiency and effectiveness of our method.

Global-optimal semi-supervised learning for single-pixel image-free sensing

Single-pixel Multispectral Imaging Based on Macropixel Segmentation Method

Compressive Single-Pixel Hyperspectral Imaging Using RGB Sensors

Spatial-spectral Encoding and Dictionary Optimization in Compressive Single-Pixel Hyperspectral Imaging Based on Mutual Coherence Minimization

Low Light Enhancement by Unsupervised Network.

Low-Quality Sensor Data-Based Semi-Supervised Learning for Medical Image Segmentation

Non-imaging single-pixel sensing with optimized binary modulation

Weighted sampling-adaptive single-pixel sensing

Optical Remote Sensing Image Understanding with Weak Supervision: Concepts, Methods, and Perspectives

Single-pixel imaging based on deep learning

Simple and Efficient: A Semisupervised Learning Framework for Remote Sensing Image Semantic Segmentation

Image-free single-pixel segmentation

Index Your Position: A Novel Self-Supervised Learning Method for Remote Sensing Images Semantic Segmentation

Optical Machine Learning with Single-pixel Imaging

Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning

Low-Shot Learning for the Semantic Segmentation of Remote Sensing Imagery

Discrete Cosine Single-Pixel Salient Object Detection Base On Deep Learning Via Fast Binary Illumination

Learning distance metric for semi-supervised image segmentation

Single-pixel imaging based on metasurface fuzzy coding

Rapid and brief communication: An algorithm for semi-supervised learning in image retrieval