Abstract:Deep learning methods have been applied when working to enhance the prediction accuracy of traditional statistical methods in the field of plant breeding. Although deep learning seems to be a promising approach for genomic prediction, it has proven to have some limitations, since its conventional methods fail to leverage all available information. Multimodal deep learning methods aim to improve the predictive power of their unimodal counterparts by introducing several modalities (sources) of input information. In this review, we introduce some theoretical basic concepts of multimodal deep learning and provide a list of the most widely used neural network architectures in deep learning, as well as the available strategies to fuse data from different modalities. We mention some of the available computational resources for the practical implementation of multimodal deep learning problems. We finally performed a review of applications of multimodal deep learning to genomic selection in plant breeding and other related fields. We present a meta-picture of the practical performance of multimodal deep learning methods to highlight how these tools can help address complex problems in the field of plant breeding. We discussed some relevant considerations that researchers should keep in mind when applying multimodal deep learning methods. Multimodal deep learning holds significant potential for various fields, including genomic selection. While multimodal deep learning displays enhanced prediction capabilities over unimodal deep learning and other machine learning methods, it demands more computational resources. Multimodal deep learning effectively captures intermodal interactions, especially when integrating data from different sources. To apply multimodal deep learning in genomic selection, suitable architectures and fusion strategies must be chosen. It is relevant to keep in mind that multimodal deep learning, like unimodal deep learning, is a powerful tool but should be carefully applied. Given its predictive edge over traditional methods, multimodal deep learning is valuable in addressing challenges in plant breeding and food security amid a growing global population.

Artificial Neural Networks and Deep Learning for Genomic Prediction of Binary, Ordinal, and Mixed Outcomes

A Benchmarking Between Deep Learning, Support Vector Machine and Bayesian Threshold Best Linear Unbiased Prediction for Predicting Ordinal Traits in Plant Breeding

A review of deep learning applications for genomic selection

Deep Kernel and Deep Learning for Genome-Based Prediction of Single Traits in Multienvironment Breeding Trials

A Multivariate Poisson Deep Learning Model for Genomic Prediction of Count Data

Genome‐based prediction of Bayesian linear and non‐linear regression models for ordinal data

Random Forest for Genomic Prediction

A review of multimodal deep learning methods for genomic-enabled prediction in plant breeding

Genome-Wide Prediction of Complex Traits in Two Outcrossing Plant Species Through Deep Learning and Bayesian Regularized Neural Network

On the combination of omics data for prediction of binary outcomes

Genomic prediction using machine learning: a comparison of the performance of regularized regression, ensemble, instance-based and deep learning methods on synthetic and empirical data

Deep learning methods improve genomic prediction of wheat breeding

A New Deep Learning Calibration Method Enhances Genome-Based Prediction of Continuous Crop Traits

Application of deep learning with bivariate models for genomic prediction of sow lifetime productivity-related traits

Machine learning algorithms translate big data into predictive breeding accuracy

Sub-sampling graph neural networks for genomic prediction of quantitative phenotypes

CGBayesNets: Conditional Gaussian Bayesian Network Learning and Inference with Mixed Discrete and Continuous Data

Machine Learning Using Neural Networks for Metabolomic Pathway Analyses

Machine Learning and Deep Learning in Genetics and Genomics

A Fully Nonparametric Modelling Approach to Binary Regression

Learning Clinical Outcomes from Heterogeneous Genomic Data Sources