Abstract:Abstract Integrative taxonomy is central to modern taxonomy and systematic biology, including behavior, niche preference, distribution, morphological analysis, and DNA barcoding. However, decades of use demonstrate that these methods can face challenges when used in isolation, for instance, potential misidentifications due to phenotypic plasticity for morphological methods, and incorrect identifications because of introgression, incomplete lineage sorting, and horizontal gene transfer for DNA barcoding. Although researchers have advocated the use of integrative taxonomy, few detailed algorithms have been proposed. Here, we develop a convolutional neural network method (morphology-molecule network [MMNet]) that integrates morphological and molecular data for species identification. The newly proposed method (MMNet) worked better than four currently available alternative methods when tested with 10 independent data sets representing varying genetic diversity from different taxa. High accuracies were achieved for all groups, including beetles (98.1% of 123 species), butterflies (98.8% of 24 species), fishes (96.3% of 214 species), and moths (96.4% of 150 total species). Further, MMNet demonstrated a high degree of accuracy ($>$98%) in four data sets including closely related species from the same genus. The average accuracy of two modest subgenomic (single nucleotide polymorphism) data sets, comprising eight putative subspecies respectively, is 90%. Additional tests show that the success rate of species identification under this method most strongly depends on the amount of training data, and is robust to sequence length and image size. Analyses on the contribution of different data types (image vs. gene) indicate that both morphological and genetic data are important to the model, and that genetic data contribute slightly more. The approaches developed here serve as a foundation for the future integration of multimodal information for integrative taxonomy, such as image, audio, video, 3D scanning, and biosensor data, to characterize organisms more comprehensively as a basis for improved investigation, monitoring, and conservation of biodiversity. [Convolutional neural network; deep learning; integrative taxonomy; single nucleotide polymorphism; species identification.]

Species identification through deep learning and geometrical morphology in oaks (Quercus spp.): Pros and cons

Deep learning to capture leaf shape in plant images: Validation by geometric morphometrics

Identification of broad-leaf trees using deep learning based on field photographs of multiple leaves

Verification of a Deep Learning-Based Tree Species Identification Model Using Images of Broadleaf and Coniferous Tree Leaves

A Novel Set of Single-Copy Nuclear Gene Markers in White Oak and Implications for Species Delimitation

A Novel Identification Method for Apple (Malus domestica Borkh.) Cultivars Based on a Deep Convolutional Neural Network with Leaf Image Input

Identification of Species by Combining Molecular and Morphological Data Using Convolutional Neural Networks

Deep Learning for Plant Species Classification Using Leaf Vein Morphometric

Mapping Tree Species Composition Using OHS-1 Hyperspectral Data and Deep Learning Algorithms in Changbai Mountains, Northeast China

Asymmetric character displacement in mixed oak stands

Computer vision cracks the leaf code

Machine Learning Classification Reveals Robust Morphometric Biomarker of Glial and Neuronal Arbors

A DEEP LEARNING METHODOLOGY FOR PLANT SPECIES RECOGNITION USING MORPHOLOGY OF LEAVES

Harnessing Deep Learning to Analyze Cryptic Morphological Variability of Marchantia polymorpha

The effects of environmental heterogeneity on the morphological similarity of toothed-leafed red oak species (Acutifoliae; Quercus ) in Mexico and Central America

A Composite Deep Learning Access for Leaf Species Classification

Leveraging remote sensing to distinguish closely related beech species in assisted gene flow scenarios

Deep Neural Network Identification of Limnonectes Species and New Class Detection Using Image Data

Automated Leaf Physiognomic Character Identification from Digital Images

Comparing modelling performance of chemometric methods for wood discrimination by near infrared spectroscopy

Introgression and incomplete lineage sorting blurred phylogenetic relationships across the genomes of sclerophyllous oaks from southwest China