Influenza vaccinations: should they really be encouraged for IBD patients being treated with immunosuppressives?

S. Fields,P. Baiocco,B. Korelitz

DOI: https://doi.org/10.1002/ibd.20731

2009-05-01

Inflammatory Bowel Diseases

Abstract:

What problem does this paper attempt to address?

Spatial Anomaly Detection in Hyperspectral Imaging Using Optical Neural Networks

Lingfeng Liu,Dong Ni,Liankui Dai

DOI: https://doi.org/10.1109/mis.2023.3241431

IF: 6.744

2023-01-01

IEEE Intelligent Systems

Abstract:Hyperspectral imaging (HSI) is a widely used technology, yet hard to implement in real-time anomaly detection due to its extensive data flow volume. An autoencoder structured hybrid optical–electrical neural network method is proposed in this work that realizes feature exaction and anomaly detection during the hyperspectral data acquisition process to address such issues. In the proposed method, a digital micromirror device functions as the core optical processor to extract low-level features from the hyperspectral data flow. Weight binarization and a conditional subnetwork are utilized to suit optical computation. Pretraining by artificial data is implemented to ease the training data burden. Case studies on defect detection and foreign object detection have demonstrated that the proposed method can significantly reduce the sampling time by orders of magnitude without loss of detection accuracy.
Deep learning approach for hyperspectral image demosaicking, spectral correction and high-resolution RGB reconstruction

Peichao Li,Michael Ebner,Philip Noonan,Conor Horgan,Anisha Bahl,Sébastien Ourselin,Jonathan Shapey,Tom Vercauteren

DOI: https://doi.org/10.1080/21681163.2021.1997646

2021-11-30

Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization

Abstract:Hyperspectral imaging is one of the most promising techniques for intraoperative tissue characterisation. Snapshot mosaic cameras, which can capture hyperspectral data in a single exposure, have the potential to make a real-time hyperspectral imaging system for surgical decision-making possible. However, optimal exploitation of the captured data requires solving an ill-posed demosaicking problem and applying additional spectral corrections. In this work, we propose a supervised learning-based image demosaicking algorithm for snapshot hyperspectral images. Due to the lack of publicly available medical images acquired with snapshot mosaic cameras, a synthetic image generation approach is proposed to simulate snapshot images from existing medical image datasets captured by high-resolution, but slow, hyperspectral imaging devices. Image reconstruction is achieved using convolutional neural networks for hyperspectral image super-resolution, followed by spectral correction using a sensor-specific calibration matrix. The results are evaluated both quantitatively and qualitatively, showing clear improvements in image quality compared to a baseline demosaicking method using linear interpolation. Moreover, the fast processing time of 45 ms of our algorithm to obtain super-resolved RGB or oxygenation saturation maps per image for a state-of-the-art snapshot mosaic camera demonstrates the potential for its seamless integration into real-time surgical hyperspectral imaging applications.

English Else
mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics

Yuhyun Ji,Sang Mok Park,Semin Kwon,Jung Woo Leem,Vidhya Vijayakrishnan Nair,Yunjie Tong,Young L. Kim

DOI: https://doi.org/10.1093/pnasnexus/pgad111

2023-04-05

Abstract:Hyperspectral imaging acquires data in both the spatial and frequency domains to offer abundant physical or biological information. However, conventional hyperspectral imaging has intrinsic limitations of bulky instruments, slow data acquisition rate, and spatiospectral tradeoff. Here we introduce hyperspectral learning for snapshot hyperspectral imaging in which sampled hyperspectral data in a small subarea are incorporated into a learning algorithm to recover the hypercube. Hyperspectral learning exploits the idea that a photograph is more than merely a picture and contains detailed spectral information. A small sampling of hyperspectral data enables spectrally informed learning to recover a hypercube from an RGB image. Hyperspectral learning is capable of recovering full spectroscopic resolution in the hypercube, comparable to high spectral resolutions of scientific spectrometers. Hyperspectral learning also enables ultrafast dynamic imaging, leveraging ultraslow video recording in an off-the-shelf smartphone, given that a video comprises a time series of multiple RGB images. To demonstrate its versatility, an experimental model of vascular development is used to extract hemodynamic parameters via statistical and deep-learning approaches. Subsequently, the hemodynamics of peripheral microcirculation is assessed at an ultrafast temporal resolution up to a millisecond, using a conventional smartphone camera. This spectrally informed learning method is analogous to compressed sensing; however, it further allows for reliable hypercube recovery and key feature extractions with a transparent learning algorithm. This learning-powered snapshot hyperspectral imaging method yields high spectral and temporal resolutions and eliminates the spatiospectral tradeoff, offering simple hardware requirements and potential applications of various machine-learning techniques.

Image and Video Processing,Machine Learning,Optics
Machine learning for direct oxygen saturation and hemoglobin concentration assessment using diffuse reflectance spectroscopy

Ingemar Fredriksson,Marcus Larsson,Tomas Strömberg

DOI: https://doi.org/10.1117/1.JBO.25.11.112905

Abstract:Significance: Diffuse reflectance spectroscopy (DRS) is frequently used to assess oxygen saturation and hemoglobin concentration in living tissue. Methods solving the inverse problem may include time-consuming nonlinear optimization or artificial neural networks (ANN) determining the absorption coefficient one wavelength at a time. Aim: To present an ANN-based method that directly outputs the oxygen saturation and the hemoglobin concentration using the shape of the measured spectra as input. Approach: A probe-based DRS setup with dual source-detector separations in the visible wavelength range was used. ANNs were trained on spectra generated from a three-layer tissue model with oxygen saturation and hemoglobin concentration as target. Results: Modeled evaluation data with realistic measurement noise showed an absolute root-mean-square (RMS) deviation of 5.1% units for oxygen saturation estimation. The relative RMS deviation for hemoglobin concentration was 13%. This accuracy is at least twice as good as our previous nonlinear optimization method. On blood-intralipid phantoms, the RMS deviation from the oxygen saturation derived from partial oxygen pressure measurements was 5.3% and 1.6% in two separate measurement series. Results during brachial occlusion showed expected patterns. Conclusions: The presented method, directly assessing oxygen saturation and hemoglobin concentration, is fast, accurate, and robust to noise.
A Sparse Deep Learning Approach for Automatic Segmentation of Human Vasculature in Multispectral Optoacoustic Tomography

Nikolaos-Kosmas Chlis,Angelos Karlas,Nikolina-Alexia Fasoula,Michael Kallmayer,Hans-Henning Eckstein,Fabian J. Theis,Vasilis Ntziachristos,Carsten Marr

DOI: https://doi.org/10.1016/j.pacs.2020.100203

IF: 9.656

2020-01-01

Photoacoustics

Abstract:Multispectral Optoacoustic Tomography (MSOT) resolves oxy- (HbO2) and deoxy-hemoglobin (Hb) to perform vascular imaging. MSOT suffers from gradual signal attenuation with depth due to light-tissue interactions: an effect that hinders the precise manual segmentation of vessels. Furthermore, vascular assessment requires functional tests, which last several minutes and result in recording thousands of images. Here, we introduce a deep learning approach with a sparse-UNET (S-UNET) for automatic vascular segmentation in MSOT images to avoid the rigorous and time-consuming manual segmentation. We evaluated the S-UNET on a test-set of 33 images, achieving a median DICE score of 0.88. Apart from high segmentation performance, our method based its decision on two wavelengths with physical meaning for the task-at-hand: 850 nm (peak absorption of oxy-hemoglobin) and 810 nm (isosbestic point of oxy-and deoxy-hemoglobin). Thus, our approach achieves precise data-driven vascular segmentation for automated vascular assessment and may boost MSOT further towards its clinical translation.
Spectrum-based deep learning framework for dermatological pigment analysis and simulation

Geunho Jung,Jongha Lee,Semin Kim

DOI: https://doi.org/10.1016/j.compbiomed.2024.108741

Abstract:Background: Deep learning in dermatology presents promising tools for automated diagnosis but faces challenges, including labor-intensive ground truth preparation and a primary focus on visually identifiable features. Spectrum-based approaches offer professional-level information like pigment distribution maps, but encounter practical limitations such as complex system requirements. Methods: This study introduces a spectrum-based framework for training a deep learning model to generate melanin and hemoglobin distribution maps from skin images. This approach eliminates the need for manually prepared ground truth by synthesizing output maps into skin images for regression analysis. The framework is applied to acquire spectral data, create pigment distribution maps, and simulate pigment variations. Results: Our model generated reflectance spectra and spectral images that accurately reflect pigment absorption properties, outperforming spectral upsampling methods. It produced pigment distribution maps with correlation coefficients of 0.913 for melanin and 0.941 for hemoglobin compared to the VISIA system. Additionally, the model's simulated images of pigment variations exhibited a proportional correlation with adjustments made to pigment levels. These evaluations are based on pigment absorption properties, the Individual Typology Angle (ITA), and pigment indices. Conclusion: The model produces pigment distribution maps comparable to those from specialized clinical equipment and simulated images with numerically adjusted pigment variations. This approach demonstrates significant promise for developing professional-level diagnostic tools for future clinical applications.
Data-driven imaging of tissue inflammation using RGB-based hyperspectral reconstruction toward personal monitoring of dermatologic health

Taehoon Kim,Michelle A Visbal-Onufrak,Raymond L Konger,Young L Kim

DOI: https://doi.org/10.1364/BOE.8.005282

2017-10-26

Abstract:Sensitive and accurate assessment of dermatologic inflammatory hyperemia in otherwise grossly normal-appearing skin conditions is beneficial to laypeople for monitoring their own skin health on a regular basis, to patients for looking for timely clinical examination, and to primary care physicians or dermatologists for delivering effective treatments. We propose that mathematical hyperspectral reconstruction from RGB images in a simple imaging setup can provide reliable visualization of hemoglobin content in a large skin area. Without relying on a complicated, expensive, and slow hyperspectral imaging system, we demonstrate the feasibility of determining heterogeneous or multifocal areas of inflammatory hyperemia associated with experimental photocarcinogenesis in mice. We envision that RGB-based reconstructed hyperspectral imaging of subclinical inflammatory hyperemic foci could potentially be integrated with the built-in camera (RGB sensor) of a smartphone to develop a simple imaging device that could offer affordable monitoring of dermatologic health.
Hyperspectral imaging and artificial intelligence to detect oral malignancy – part 1 - automated tissue classification of oral muscle, fat and mucosa using a light-weight 6-layer deep neural network

Daniel G. E. Thiem,Paul Römer,Matthias Gielisch,Bilal Al-Nawas,Martin Schlüter,Bastian Plaß,Peer W. Kämmerer

DOI: https://doi.org/10.1186/s13005-021-00292-0

2021-09-03

Abstract:Abstract Background Hyperspectral imaging (HSI) is a promising non-contact approach to tissue diagnostics, generating large amounts of raw data for whose processing computer vision (i.e. deep learning) is particularly suitable. Aim of this proof of principle study was the classification of hyperspectral (HS)-reflectance values into the human-oral tissue types fat, muscle and mucosa using deep learning methods. Furthermore, the tissue-specific hyperspectral signatures collected will serve as a representative reference for the future assessment of oral pathological changes in the sense of a HS-library. Methods A total of about 316 samples of healthy human-oral fat, muscle and oral mucosa was collected from 174 different patients and imaged using a HS-camera, covering the wavelength range from 500 nm to 1000 nm. HS-raw data were further labelled and processed for tissue classification using a light-weight 6-layer deep neural network (DNN). Results The reflectance values differed significantly ( p 80% each. Conclusion Oral fat, muscle and mucosa can be classified sufficiently and automatically by their specific HS-signature using a deep learning approach. Early detection of premalignant-mucosal-lesions using hyperspectral imaging and deep learning is so far represented rarely in in medical and computer vision research domain but has a high potential and is part of subsequent studies.

dentistry, oral surgery & medicine
Combining hyperspectral imaging techniques with deep learning to aid in early pathological diagnosis of melanoma

Chongxuan Tian,Yanjing Xu,Yanbing Zhang,Zhenlei Zhang,Haoyuan An,Yelin Liu,Yuzhuo Chen,Hanzhu Zhao,Zhenyu Zhang,Qi Zhao,Wei Li

DOI: https://doi.org/10.1016/j.pdpdt.2023.103708

IF: 3.577

2023-07-23

Photodiagnosis and Photodynamic Therapy

Abstract:Background Cutaneous melanoma, an exceedingly aggressive form of skin cancer, holds the top rank in both malignancy and mortality among skin cancers. In early stages, distinguishing malignant melanomas from benign pigmented nevi pathologically becomes a significant challenge due to their indistinguishable traits. Traditional skin histological examination techniques, largely reliant on light microscopic imagery, offer constrained information and yield low-contrast results, underscoring the necessity for swift and effective early diagnostic methodologies. As a non-contact, non-ionizing, and label-free imaging tool, hyperspectral imaging offers potential in assisting pathologists with identification procedures sans contrast agents. Methods This investigation leverages hyperspectral cameras to ascertain the optical properties and to capture the spectral features of malignant melanoma and pigmented nevus tissues, intending to facilitate early pathological diagnostic applications. We further enhance the diagnostic process by integrating transfer learning with deep convolutional networks to classify melanomas and pigmented nevi in hyperspectral pathology images. The study encompasses pathological sections from 50 melanoma and 50 pigmented nevus patients. To accurately represent the spectral variances between different tissues, we employed reflectance calibration, highlighting that the most distinctive spectral differences emerged within the 500-675 nm band range. Results The classification accuracy of pigmented tumors and pigmented nevi was 89% for one-dimensional sample data and 98% for two-dimensional sample data. Conclusions Our findings have the potential to expedite pathological diagnoses, enhance diagnostic precision, and offer novel research perspectives in differentiating melanoma and nevus.

oncology
Deep Learning-Based Correction and Unmixing of Hyperspectral Images for Brain Tumor Surgery

David Black,Jaidev Gill,Andrew Xie,Benoit Liquet,Antonio Di leva,Walter Stummer,Eric Suero Molina

2024-02-06

Abstract:Hyperspectral Imaging (HSI) for fluorescence-guided brain tumor resection enables visualization of differences between tissues that are not distinguishable to humans. This augmentation can maximize brain tumor resection, improving patient outcomes. However, much of the processing in HSI uses simplified linear methods that are unable to capture the non-linear, wavelength-dependent phenomena that must be modeled for accurate recovery of fluorophore abundances. We therefore propose two deep learning models for correction and unmixing, which can account for the nonlinear effects and produce more accurate estimates of abundances. Both models use an autoencoder-like architecture to process the captured spectra. One is trained with protoporphyrin IX (PpIX) concentration labels. The other undergoes semi-supervised training, first learning hyperspectral unmixing self-supervised and then learning to correct fluorescence emission spectra for heterogeneous optical and geometric properties using a reference white-light reflectance spectrum in a few-shot manner. The models were evaluated against phantom and pig brain data with known PpIX concentration; the supervised model achieved Pearson correlation coefficients (R values) between the known and computed PpIX concentrations of 0.997 and 0.990, respectively, whereas the classical approach achieved only 0.93 and 0.82. The semi-supervised approach's R values were 0.98 and 0.91, respectively. On human data, the semi-supervised model gives qualitatively more realistic results than the classical method, better removing bright spots of specular reflectance and reducing the variance in PpIX abundance over biopsies that should be relatively homogeneous. These results show promise for using deep learning to improve HSI in fluorescence-guided neurosurgery.

Image and Video Processing,Machine Learning,Tissues and Organs
Automatic Tissue Differentiation in Parotidectomy using Hyperspectral Imaging

Eric L. Wisotzky,Alexander Schill,Anna Hilsmann,Peter Eisert,Michael Knoke

2024-12-06

Abstract:In head and neck surgery, continuous intraoperative tissue differentiation is of great importance to avoid injury to sensitive structures such as nerves and vessels. Hyperspectral imaging (HSI) with neural network analysis could support the surgeon in tissue differentiation. A 3D Convolutional Neural Network with hyperspectral data in the range of $400-1000$ nm is used in this work. The acquisition system consisted of two multispectral snapshot cameras creating a stereo-HSI-system. For the analysis, 27 images with annotations of glandular tissue, nerve, muscle, skin and vein in 18 patients undergoing parotidectomy are included. Three patients are removed for evaluation following the leave-one-subject-out principle. The remaining images are used for training, with the data randomly divided into a training group and a validation group. In the validation, an overall accuracy of $98.7\%$ is achieved, indicating robust training. In the evaluation on the excluded patients, an overall accuracy of $83.4\%$ has been achieved showing good detection and identification abilities. The results clearly show that it is possible to achieve robust intraoperative tissue differentiation using hyperspectral imaging. Especially the high sensitivity in parotid or nerve tissue is of clinical importance. It is interesting to note that vein was often confused with muscle. This requires further analysis and shows that a very good and comprehensive data basis is essential. This is a major challenge, especially in surgery.

Image and Video Processing,Computer Vision and Pattern Recognition,Medical Physics
Spectral Analysis Comparison of Pushbroom and Snapshot Hyperspectral Cameras for In-Vivo Brain Tissues and Chromophores Identification

Alberto Martin-Perez,Alejandro Martinez de Ternero,Alfonso Lagares,Eduardo Juarez,Cesar Sanz

DOI: https://doi.org/10.1101/2024.06.06.24308500

2024-06-07

Abstract:Significance: Hyperspectral imaging sensors have rapidly advanced, aiding in tumor diagnostics for invivo brain tumors. Linescan cameras effectively distinguish between pathological and healthy tissue, while snapshot cameras offer a potential alternative to reduce acquisition time. Aim: Our research compares linescan and snapshot hyperspectral cameras for in-vivo brain tissues and chromophores identification. Approach: We compared a linescan pushbroom camera and a snapshot camera using images from 10 patients with various pathologies. Objective comparisons were made using unnormalized and normalized data for healthy and pathological tissues. We utilized Interquartile Range (IQR) for the Spectral Angle Mapping (SAM), the Goodness-of-Fit Coefficient (GFC), and the Root Mean Square Error (RMSE) within the 659.95 to 951.42 nm range. Additionally, we assessed the ability of both cameras to capture tissue chromophores by analyzing absorbance from reflectance information. Results: The SAM metric indicates reduced dispersion and high similarity between cameras for pathological samples, with a 9.68% IQR for normalized data compared to 2.38% for unnormalized data. This pattern is consistent across GFC and RMSE metrics, regardless of tissue type. Moreover, both cameras could identify absorption peaks of certain chromophores. For instance, using the absorbance measurements of the linescan camera we obtained SAM values below 0.235 for four peaks, regardless of the tissue and type of data under inspection. These peaks are: one for cytochrome b in its oxidised form at λ = 422 nm, two for HbO2 at λ = 542 nm and λ = 576 nm, and one for water at λ = 976 nm. Conclusion: The spectral signatures of the cameras show more similarity with unnormalized data, likely due to snapshot sensor noise, resulting in noisier signatures post-normalization. Comparisons in this study suggest that snapshot cameras might be viable alternatives to linescan cameras for real-time brain tissues identification
Deep Ensemble Model for Quantitative Optical Property and Chromophore Concentration Images of Biological Tissues

Bingbao Yan,Bowen Song,Chang Ge,Xinman Yin,Wenchao Jia,Gen Mu,Yanyu Zhao

DOI: https://doi.org/10.1101/2024.12.04.626749

2024-12-07

Abstract:The ability to quantify widefield tissue optical properties (OPs, i.e., absorption and scattering) has major implications on the characterization of various physiological and disease processes. However, conventional image processing methods for tissue optical properties are either limited to qualitative analysis, or have tradeoffs in speed and accuracy. The key to quantification of optical properties is the extraction of amplitude maps from reflectance images under sinusoidal illumination of different spatial frequencies. Conventional three-phase demodulation (TPD) method has been demonstrated for the mapping of OPs, but it requires as many as 14 measurement images for accurate OP extraction, which leads to limited throughput and hinders practical translation. Although single-phase demodulation (SPD) method has been proposed to map OPs with a single measurement image, it is typically subject to image artifacts and decreased measurement accuracy. To tackle those challenges, here we develop a deep ensemble model (DEM) that can map tissue optical properties with high accuracy in a single snapshot, increasing the measurement speed by 14x compared to conventional TPD method. The proposed method was validated with measurements on an array of optical phantoms, ex vivo tissues, and in vivo tissues. The errors for OP extraction were 0.83 +/- 5.0% for absorption and 0.40 +/- 1.9% for reduced scattering, dramatically lower than that of the state-of-the-art SPD method (2.5 +/- 15% for absorption and -1.2 +/- 11% for reduced scattering). It was further demonstrated that while trained with data from a single wavelength, the DEM can be directly applied to other wavelengths and effectively obtain optical property and chromophore concentration images of biological tissues. Together, these results highlight the potential of DEM to enable new capabilities for quantitative monitoring of tissue physiological and disease processes.

Biophysics
Using spectral derivatives to remove the influence of hair on optical images of the static absorbing properties of tissue-like turbid media

Jeremy C Hebden,Gianna Forrester,Haoyang Zhang,Danica M Pacis

DOI: https://doi.org/10.1117/1.NPh.11.2.025002

Neurophotonics

Abstract:Significance: Although measurements of near-infrared light diffusely reflected from the head and other biological tissues are commonly used to generate images revealing changes in the concentrations of oxy- and deoxy-hemoglobin, static imaging of absolute concentrations has been inhibited by the unknown and variable coupling between the optical probe and the skin, to which hair is often a significant contributor. Measurements of spectral derivatives provide a means of overcoming this shortcoming. Aim: The aim is to demonstrate experimentally that measurements of the derivative of the attenuation of the detected signal with respect to wavelength can be used to achieve images that are immune to the spatial variation of hair on the surface. The objective is to generate topographic images representative of static absorbing properties rather than retrieving absolute optical coefficients, which requires a tomographic approach. Approach: The surface of a tissue-equivalent phantom, containing targets with different concentrations of absorbing dye, was coated with a layer of dark hair. The phantom was then imaged using a broadband source and spectrometer, and prior knowledge of the absorbing characteristics of the dye and of melanin was used to acquire separate images of each. Results: The targets within the phantom are revealed with remarkable clarity, although a nonlinear relationship between the target contrast and absorption was observed. This nonlinear behavior was confirmed and explained using a Monte Carlo model of light propagation in a slab of similar absorbing properties. Conclusions: A spectral derivative approach could be an effective tool for in vivo topographic imaging of the static optical properties of the brain and other tissues, avoiding the deleterious effects of hair.
Separating Surface Reflectance from Volume Reflectance in Medical Hyperspectral Imaging

Lynn-Jade S. Jong,Anouk L. Post,Freija Geldof,Behdad Dashtbozorg,Theo J. M. Ruers,Henricus J. C. M. Sterenborg

DOI: https://doi.org/10.3390/diagnostics14161812

IF: 3.6

2024-08-21

Diagnostics

Abstract:Hyperspectral imaging has shown great promise for diagnostic applications, particularly in cancer surgery. However, non-bulk tissue-related spectral variations complicate the data analysis. Common techniques, such as standard normal variate normalization, often lead to a loss of amplitude and scattering information. This study investigates a novel approach to address these spectral variations in hyperspectral images of optical phantoms and excised human breast tissue. Our method separates surface and volume reflectance, hypothesizing that spectral variability arises from significant variations in surface reflectance across pixels. An illumination setup was developed to measure samples with a hyperspectral camera from different axial positions but with identical zenith angles. This configuration, combined with a novel data analysis approach, allows for the estimation and separation of surface reflectance for each direction and volume reflectance across all directions. Validated with optical phantoms, our method achieved an 83% reduction in spectral variability. Its functionality was further demonstrated in excised human breast tissue. Our method effectively addresses variations caused by surface reflectance or glare while conserving surface reflectance information, which may enhance sample analysis and evaluation. It benefits samples with unknown refractive index spectra and can be easily adapted and applied across a wide range of fields where hyperspectral imaging is used.

medicine, general & internal
Single-shot hyperspectral imaging based on dual attention neural network with multi-modal learning

Tianyue He,Qican Zhang,Mingwei Zhou,Tingdong Kou,Junfei Shen

DOI: https://doi.org/10.1364/OE.446483

2022-03-14

Abstract:Hyperspectral imaging is being extensively investigated owing to its promising future in critical applications such as medical diagnostics, sensing, and surveillance. However, current techniques are complex with multiple alignment-sensitive components and spatiospectral parameters predetermined by manufacturers. In this paper, we demonstrate an end-to-end snapshot hyperspectral imaging technique and build a physics-informed dual attention neural network with multimodal learning. By modeling the 3D spectral cube reconstruction procedure and solving that compressive-imaging inverse problem, the hyperspectral volume can be directly recovered from only one scene RGB image. Spectra features and camera spectral sensitivity are jointly leveraged to retrieve the multiplexed spatiospectral correlations and realize hyperspectral imaging. With the help of integrated attention mechanism, useful information supplied by disparate modal components is adaptively learned and aggregated to make our network flexible for variable imaging systems. Results show that the proposed method is ultra-faster than the traditional scanning method, and 3.4 times more precise than the existing hyperspectral imaging convolutional neural network. We provide theory for network design, demonstrate training process, and present experimental results with high accuracy. Without bulky benchtop setups and strict experimental limitations, this simple and effective method offers great potential for future spectral imaging applications such as pathological digital stain, computational imaging and virtual/augmented reality display, etc.
Spectral analysis comparison of pushbroom and snapshot hyperspectral cameras for in vivo brain tissues and chromophore identification

Alberto Martín-Pérez,Alejandro Martinez de Ternero,Alfonso Lagares,Eduardo Juarez,César Sanz

DOI: https://doi.org/10.1117/1.jbo.29.9.093510

IF: 3.758

2024-09-25

Journal of Biomedical Optics

Abstract:SignificanceHyperspectral imaging sensors have rapidly advanced, aiding in tumor diagnostics for in vivo brain tumors. Linescan cameras effectively distinguish between pathological and healthy tissue, whereas snapshot cameras offer a potential alternative to reduce acquisition time.AimOur research compares linescan and snapshot hyperspectral cameras for in vivo brain tissues and chromophore identification.ApproachWe compared a linescan pushbroom camera and a snapshot camera using images from 10 patients with various pathologies. Objective comparisons were made using unnormalized and normalized data for healthy and pathological tissues. We utilized the interquartile range (IQR) for the spectral angle mapping (SAM), the goodness-of-fit coefficient (GFC), and the root mean square error (RMSE) within the 659.95 to 951.42 nm range. In addition, we assessed the ability of both cameras to capture tissue chromophores by analyzing absorbance from reflectance information.ResultsThe SAM metric indicates reduced dispersion and high similarity between cameras for pathological samples, with a 9.68% IQR for normalized data compared with 2.38% for unnormalized data. This pattern is consistent across GFC and RMSE metrics, regardless of tissue type. Moreover, both cameras could identify absorption peaks of certain chromophores. For instance, using the absorbance measurements of the linescan camera, we obtained SAM values below 0.235 for four peaks, regardless of the tissue and type of data under inspection. These peaks are one for cytochrome b in its oxidized form at λ=422 nm, two for HbO2 at λ=542 nm and λ=576 nm, and one for water at λ=976 nm.ConclusionThe spectral signatures of the cameras show more similarity with unnormalized data, likely due to snapshot sensor noise, resulting in noisier signatures post-normalization. Comparisons in this study suggest that snapshot cameras might be viable alternatives to linescan cameras for real-time brain tissue identification.

radiology, nuclear medicine & medical imaging,biochemical research methods,optics
Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: Application to surgical imaging

Peichao Li,Muhammad Asad,Conor Horgan,Oscar MacCormac,Jonathan Shapey,Tom Vercauteren

DOI: https://doi.org/10.1007/s11548-023-02865-7

2023-02-22

Abstract:Hyperspectral imaging has the potential to improve intraoperative decision making if tissue characterisation is performed in real-time and with high-resolution. Hyperspectral snapshot mosaic sensors offer a promising approach due to their fast acquisition speed and compact size. However, a demosaicking algorithm is required to fully recover the spatial and spectral information of the snapshot images. Most state-of-the-art demosaicking algorithms require ground-truth training data with paired snapshot and high-resolution hyperspectral images, but such imagery pairs with the exact same scene are physically impossible to acquire in intraoperative settings. In this work, we present a fully unsupervised hyperspectral image demosaicking algorithm which only requires exemplar snapshot images for training purposes. We regard hyperspectral demosaicking as an ill-posed linear inverse problem which we solve using a deep neural network. We take advantage of the spectral correlation occurring in natural scenes to design a novel inter spectral band regularisation term based on spatial gradient consistency. By combining our proposed term with standard regularisation techniques and exploiting a standard data fidelity term, we obtain an unsupervised loss function for training deep neural networks, which allows us to achieve real-time hyperspectral image demosaicking. Quantitative results on hyperspetral image datasets show that our unsupervised demosaicking approach can achieve similar performance to its supervised counter-part, and significantly outperform linear demosaicking. A qualitative user study on real snapshot hyperspectral surgical images confirms the results from the quantitative analysis. Our results suggest that the proposed unsupervised algorithm can achieve promising hyperspectral demosaicking in real-time thus advancing the suitability of the modality for intraoperative use.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning
Classification of Microscopic Hyperspectral Images of Blood Cells Based on Lightweight Convolutional Neural Network

Jinghui Fang

DOI: https://doi.org/10.3390/electronics13081578

IF: 2.9

2024-04-21

Electronics

Abstract:Hyperspectral imaging has emerged as a novel imaging modality in the medical field, offering the ability to acquire images of biological tissues while simultaneously providing biochemical insights for in-depth tissue analysis. This approach facilitates early disease diagnosis, presenting advantages over traditional medical imaging techniques. Addressing challenges such as the computational burden of existing convolutional neural networks (CNNs) and imbalances in sample data, this paper introduces a lightweight GhostMRNet for the classification of microscopic hyperspectral images of human blood cells. The proposed model employs Ghost Modules to replace conventional convolutional layers and a cascading approach with small convolutional kernels for multiscale feature extraction, aiming to enhance feature extraction capabilities while reducing computational complexity. Additionally, an SE (Squeeze-and-Excitation) module is introduced to selectively allocate weights to features in each channel, emphasizing informative features and efficiently achieving spatial–spectral feature extraction in microscopic hyperspectral imaging. We evaluated the performance of the proposed GhostMRNet and compared it with other state-of-the-art models using two real medical hyperspectral image datasets. The experimental results demonstrate that GhostMRNet exhibits a superior performance, with an overall accuracy (OA), average accuracy (AA), and Kappa coefficient reaching 99.965%, 99.565%, and 0.9925, respectively. In conclusion, the proposed GhostMRNet achieves a superior classification performance at a smaller computational cost, thereby providing a novel approach for blood cell detection.

engineering, electrical & electronic,computer science, information systems,physics, applied
Hyperspectral Imaging for Non-invasive Blood Oxygen Saturation Assessment

Jiangbo Wu

DOI: https://doi.org/10.1016/j.pdpdt.2024.104003

IF: 3.577

2024-02-10

Photodiagnosis and Photodynamic Therapy

Abstract:Hyperspectral Imaging (HSI) seamlessly integrates imaging and spectroscopy, capturing both spatial and spectral data concurrently. With widespread applications in medical diagnostics, HSI serves as a noninvasive tool for gaining insights into tissue characteristics. The distinctive spectral profiles of biological tissues set HSI apart from traditional microscopy in enabling in vivo tissue analysis. Despite its potential, existing HSI techniques face challenges such as alignment issues, low light throughput, and tissue heating due to intense illumination. This study introduces an innovative HSI system featuring active sequential bandpass illumination seamlessly integrated into conventional optical instruments. The primary focus is on analyzing oxyhemoglobin and deoxyhemoglobin saturation in animal tissue samples using multivariate linear regression. This approach holds promise for enhancing noninvasive medical diagnostics. A key feature of the system, active bandpass illumination, effectively prevents tissue overheating, thereby bolstering its suitability for medical applications.

oncology

Influenza vaccinations: should they really be encouraged for IBD patients being treated with immunosuppressives?

Spatial Anomaly Detection in Hyperspectral Imaging Using Optical Neural Networks

Deep learning approach for hyperspectral image demosaicking, spectral correction and high-resolution RGB reconstruction

mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics

Machine learning for direct oxygen saturation and hemoglobin concentration assessment using diffuse reflectance spectroscopy

A Sparse Deep Learning Approach for Automatic Segmentation of Human Vasculature in Multispectral Optoacoustic Tomography

Spectrum-based deep learning framework for dermatological pigment analysis and simulation

Data-driven imaging of tissue inflammation using RGB-based hyperspectral reconstruction toward personal monitoring of dermatologic health

Hyperspectral imaging and artificial intelligence to detect oral malignancy – part 1 - automated tissue classification of oral muscle, fat and mucosa using a light-weight 6-layer deep neural network

Combining hyperspectral imaging techniques with deep learning to aid in early pathological diagnosis of melanoma

Deep Learning-Based Correction and Unmixing of Hyperspectral Images for Brain Tumor Surgery

Automatic Tissue Differentiation in Parotidectomy using Hyperspectral Imaging

Spectral Analysis Comparison of Pushbroom and Snapshot Hyperspectral Cameras for In-Vivo Brain Tissues and Chromophores Identification

Deep Ensemble Model for Quantitative Optical Property and Chromophore Concentration Images of Biological Tissues

Using spectral derivatives to remove the influence of hair on optical images of the static absorbing properties of tissue-like turbid media

Separating Surface Reflectance from Volume Reflectance in Medical Hyperspectral Imaging

Single-shot hyperspectral imaging based on dual attention neural network with multi-modal learning

Spectral analysis comparison of pushbroom and snapshot hyperspectral cameras for in vivo brain tissues and chromophore identification

Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: Application to surgical imaging

Classification of Microscopic Hyperspectral Images of Blood Cells Based on Lightweight Convolutional Neural Network

Hyperspectral Imaging for Non-invasive Blood Oxygen Saturation Assessment