Abstract:Abstract Integrative taxonomy is central to modern taxonomy and systematic biology, including behavior, niche preference, distribution, morphological analysis, and DNA barcoding. However, decades of use demonstrate that these methods can face challenges when used in isolation, for instance, potential misidentifications due to phenotypic plasticity for morphological methods, and incorrect identifications because of introgression, incomplete lineage sorting, and horizontal gene transfer for DNA barcoding. Although researchers have advocated the use of integrative taxonomy, few detailed algorithms have been proposed. Here, we develop a convolutional neural network method (morphology-molecule network [MMNet]) that integrates morphological and molecular data for species identification. The newly proposed method (MMNet) worked better than four currently available alternative methods when tested with 10 independent data sets representing varying genetic diversity from different taxa. High accuracies were achieved for all groups, including beetles (98.1% of 123 species), butterflies (98.8% of 24 species), fishes (96.3% of 214 species), and moths (96.4% of 150 total species). Further, MMNet demonstrated a high degree of accuracy ($>$98%) in four data sets including closely related species from the same genus. The average accuracy of two modest subgenomic (single nucleotide polymorphism) data sets, comprising eight putative subspecies respectively, is 90%. Additional tests show that the success rate of species identification under this method most strongly depends on the amount of training data, and is robust to sequence length and image size. Analyses on the contribution of different data types (image vs. gene) indicate that both morphological and genetic data are important to the model, and that genetic data contribute slightly more. The approaches developed here serve as a foundation for the future integration of multimodal information for integrative taxonomy, such as image, audio, video, 3D scanning, and biosensor data, to characterize organisms more comprehensively as a basis for improved investigation, monitoring, and conservation of biodiversity. [Convolutional neural network; deep learning; integrative taxonomy; single nucleotide polymorphism; species identification.]

Biodiversity Image Quality Metadata Augments Convolutional Neural Network Classification of Fish Species

Classification of Freshwater Zooplankton by Pre-trained Convolutional Neural Network in Underwater Microscopy

Automatic Metadata Generation for Fish Specimen Image Collections

Metadata augmented deep neural networks for wild animal classification

Improving the accuracy of automated labeling of specimen images datasets via a confidence-based process

Microscopic image recognition of diatoms based on deep learning.

Plant Species Classification and Biodiversity Estimation from UAV Images with Deep Learning

Improving Transfer Learning and Squeeze- And-Excitation Networks for Small-Scale Fine-Grained Fish Image Classification

Improving plankton image classification using context metadata

Improving the classification accuracy of fishes and invertebrates using residual convolutional neural networks

Classifying aquatic organism via evolving residual convolutional neural networks by optimized random vector functional link

Fish species classification in unconstrained underwater environments based on deep learning

Identification of Species by Combining Molecular and Morphological Data Using Convolutional Neural Networks

Few-shot Fish Image Generation and Classification

Fish Species Classification Using Deep Learning and Appearance-Based Feature Extraction

Enhanced fish species classification using dynamic multilayer perceptron and transformer encoders with extra distribution data

Enhancing microalgae classification accuracy in marine ecosystems through convolutional neural networks and support vector machines

Taxonomic resolution of coral image classification with Convolutional Neural Network

Color Image Quality Assessment with Biologically Inspired Feature and Machine Learning

Fish Species Image Classification Using Convolutional Neural Networks

Hierarchical Classification for Automated Image Annotation of Coral Reef Benthic Structures