Abstract:The advent of biobanks with vast quantities of medical imaging and paired genetic measurements creates huge opportunities for a new generation of genotype-phenotype association studies. However, disentangling biological signals from the many sources of bias and artifacts remains difficult. Using diverse medical images and time-series (ie, magnetic resonance imagings [MRIs], electrocardiograms [ECGs], and dual-energy X-ray absorptiometries [DXAs]), we show how registration, both spatial and temporal, guided by domain knowledge or learned de novo, helps uncover biological information. A multimodal autoencoder comparison framework quantifies and characterizes how registration affects the representations that unsupervised and self-supervised encoders learn. In this study we (1) train autoencoders before and after registration with nine diverse types of medical image, (2) demonstrate how neural network-based methods (VoxelMorph, DeepCycle, and DropFuse) can effectively learn registrations allowing for more flexible and efficient processing than is possible with hand-crafted registration techniques, and (3) conduct exhaustive phenotypic screening, comprised of millions of statistical tests, to quantify how registration affects the generalizability of learned representations. Genome- and phenome-wide association studies (GWAS and PheWAS) uncover significantly more associations with registered modality representations than with equivalently trained and sized representations learned from native coordinate spaces. Specifically, registered PheWAS yielded 61 more disease associations for ECGs, 53 more disease associations for cardiac MRIs, and 10 more disease associations for brain MRIs. Registration also yields significant increases in the coefficient of determination when regressing continuous phenotypes (eg, 0.36 ± 0.01 with ECGs and 0.11 ± 0.02 for DXA scans). Our findings reveal the crucial role registration plays in enhancing the characterization of physiological states across a broad range of medical imaging data types. Importantly, this finding extends to more flexible types of registration, such as the cross-modal and the circular mapping methods presented here.

Autoencoder-based phenotyping of ophthalmic images highlights genetic loci influencing retinal morphology and provides informative biomarkers

Large-scale machine learning-based phenotyping significantly improves genomic discovery for optic nerve head morphology

Expert-level Automated Biomarker Identification in Optical Coherence Tomography Scans

Ocular biomarkers: useful incidental findings by deep learning algorithms in fundus photographs

Genetic Architectures of Medical Images Revealed by Registration of Multiple Modalities

Deep Learning and Glaucoma Specialists: The Relative Importance of Optic Disc Features to Predict Glaucoma Referral in Fundus Photos

Efficient multi-phenotype genome-wide analysis identifies genetic associations for unsupervised deep-learning-derived high-dimensional brain imaging phenotypes

Describing the Structural Phenotype of the Glaucomatous Optic Nerve Head Using Artificial Intelligence

Large-scale multitrait genome-wide association analyses identify hundreds of glaucoma risk loci

Phenome- and genome-wide analyses of retinal optical coherence tomography images identify links between ocular and systemic health

Integrating Deep Learning with Fundus and Optical Coherence Tomography for Cardiovascular Disease Prediction

Computing linkage disequilibrium aware genome embeddings using autoencoders

Deep Learning-Based Identification of Intraocular Pressure-Associated Genes Influencing Trabecular Meshwork Cell Morphology

Phenotypic and Genetic Characteristics of Retinal Vascular Parameters and their Association with Diseases

Retinal OCT disease classification with variational autoencoder regularization

Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning

Finding New Diagnostic Information for Detecting Glaucoma using Neural Networks

Predicting sex from retinal fundus photographs using automated deep learning

Can artificial intelligence accelerate the diagnosis of inherited retinal diseases? Protocol for a data-only retrospective cohort study (Eye2Gene)

Interpreting artificial neural networks to detect genome-wide association signals for complex traits

Deep-Learning–Aided Diagnosis of Diabetic Retinopathy, Age-Related Macular Degeneration, and Glaucoma Based on Structural and Angiographic OCT