Abstract:Multimodal imaging analyses of dosed tissue samples can provide more comprehensive insights into the effects of a therapeutically active compound on a target tissue compared to single-modal imaging. For example, simultaneous spatial mapping of pharmaceutical compounds and endogenous macromolecule receptors is difficult to achieve in a single imaging experiment. Herein, we present a multimodal workflow combining imaging mass spectrometry with immunohistochemistry (IHC) fluorescence imaging and brightfield microscopy imaging. Imaging mass spectrometry enables direct mapping of pharmaceutical compounds and metabolites, IHC fluorescence imaging can visualize large proteins, and brightfield microscopy imaging provides tissue morphology information. Single-cell resolution images are generally difficult to acquire using imaging mass spectrometry but are readily acquired with IHC fluorescence and brightfield microscopy imaging. Spatial sharpening of mass spectrometry images would thus allow for higher fidelity coregistration with other higher-resolution microscopy images. Imaging mass spectrometry spatial resolution can be predicted to a finer value via a computational image fusion workflow, which models the relationship between the intensity values in the mass spectrometry image and the features of a high-spatial resolution microscopy image. As a proof of concept, our multimodal workflow was applied to brain tissue extracted from a Sprague-Dawley rat dosed with a kratom alkaloid, corynantheidine. Four candidate mathematical models, including linear regression, partial least-squares regression, random forest regression, and two-dimensional convolutional neural network (2-D CNN), were tested. The random forest and 2-D CNN models most accurately predicted the intensity values at each pixel as well as the overall patterns of the mass spectrometry images, while also providing the best spatial resolution enhancements. Herein, image fusion enabled predicted mass spectrometry images of corynantheidine, GABA, and glutamine to approximately 2.5 μm spatial resolutions, a significant improvement compared to the original images acquired at 25 μm spatial resolution. The predicted mass spectrometry images were then coregistered with an H&E image and IHC fluorescence image of the μ-opioid receptor to assess colocalization of corynantheidine with brain cells. Our study also provides insights into the different evaluation parameters to consider when utilizing image fusion for biological applications.

Multimodal MALDI imaging mass spectrometry for improved diagnosis of melanoma

Multimodal Metabolic Imaging Reveals Pigment Reduction and Lipid Accumulation in Metastatic Melanoma

Advanced Registration and Analysis of MALDI Imaging Mass Spectrometry Measurements through Autofluorescence Microscopy

Cutaneous squamous cell carcinoma characterized by MALDI mass spectrometry imaging in combination with machine learning

Combining hyperspectral imaging techniques with deep learning to aid in early pathological diagnosis of melanoma

A Multimodal Pipeline for Image Correction and Registration of Mass Spectrometry Imaging with Microscopy

Classification of melanocytic lesions using direct illumination multispectral imaging

Multimodal Lung Cancer Subtyping Using Deep Learning Neural Networks on Whole Slide Tissue Images and MALDI MSI

Automated identification of malignancy in whole-slide pathological images: identification of eyelid malignant melanoma in gigapixel pathological slides using deep learning

Deep learning for tumor classification in imaging mass spectrometry

Enhancing Classification of Mass Spectrometry Imaging Data with Deep Neural Networks.

Multimodal integration of image, epigenetic and clinical data to predict BRAF mutation status in melanoma

MALDI imaging mass spectrometry in cancer research: combining proteomic profiling and histological evaluation

MALDI IMS-Derived Molecular Contour Maps: Augmenting Histology Whole-Slide Images

Weakly-supervised learning for image-based classification of primary melanomas into genomic immune subgroups

Machine-learning convergent melanocytic morphology despite noisy archival slides

Integrated photoacoustic and hyperspectral dual modality microscope for co-imaging of melanoma and cutaneous squamous cell carcinoma in vivo.

Immune subtyping of melanoma whole slide images using multiple instance learning

A Streamlined Workflow for Microscopy-Driven MALDI Imaging Mass Spectrometry Data Collection

Multimodal Image Fusion Workflow Incorporating MALDI Imaging Mass Spectrometry and Microscopy for the Study of Small Pharmaceutical Compounds

Histology-Guided Single-Cell Mass Spectrometry Imaging using Integrated Bright-field and Fluorescence Microscopy