Abstract:Abstract Integrative taxonomy is central to modern taxonomy and systematic biology, including behavior, niche preference, distribution, morphological analysis, and DNA barcoding. However, decades of use demonstrate that these methods can face challenges when used in isolation, for instance, potential misidentifications due to phenotypic plasticity for morphological methods, and incorrect identifications because of introgression, incomplete lineage sorting, and horizontal gene transfer for DNA barcoding. Although researchers have advocated the use of integrative taxonomy, few detailed algorithms have been proposed. Here, we develop a convolutional neural network method (morphology-molecule network [MMNet]) that integrates morphological and molecular data for species identification. The newly proposed method (MMNet) worked better than four currently available alternative methods when tested with 10 independent data sets representing varying genetic diversity from different taxa. High accuracies were achieved for all groups, including beetles (98.1% of 123 species), butterflies (98.8% of 24 species), fishes (96.3% of 214 species), and moths (96.4% of 150 total species). Further, MMNet demonstrated a high degree of accuracy ($>$98%) in four data sets including closely related species from the same genus. The average accuracy of two modest subgenomic (single nucleotide polymorphism) data sets, comprising eight putative subspecies respectively, is 90%. Additional tests show that the success rate of species identification under this method most strongly depends on the amount of training data, and is robust to sequence length and image size. Analyses on the contribution of different data types (image vs. gene) indicate that both morphological and genetic data are important to the model, and that genetic data contribute slightly more. The approaches developed here serve as a foundation for the future integration of multimodal information for integrative taxonomy, such as image, audio, video, 3D scanning, and biosensor data, to characterize organisms more comprehensively as a basis for improved investigation, monitoring, and conservation of biodiversity. [Convolutional neural network; deep learning; integrative taxonomy; single nucleotide polymorphism; species identification.]

Classification of freshwater snails of the genus Radomaniola with multimodal triplet networks

Classification of Freshwater Zooplankton by Pre-trained Convolutional Neural Network in Underwater Microscopy

Fish Detection and Classification Based on Improved ViT

Deep Neural Network Identification of Limnonectes Species and New Class Detection Using Image Data

A snail species identification method based on deep learning in food safety

Identifying geographically differentiated features of Ethopian Nile tilapia (Oreochromis niloticus) morphology with machine learning

A Deep CNN-Based Salinity and Freshwater Fish Identification and Classification Using Deep Learning and Machine Learning

Identification of Species by Combining Molecular and Morphological Data Using Convolutional Neural Networks

Similarity learning networks uniquely identify individuals of four marine and terrestrial species

HELIX: Data-driven characterization of Brazilian land snails

Multi-class Fish Stock Statistics Technology Based on Object Classification and Tracking Algorithm

Applications of 3D modeling in cryptic species classification of molluscs

Using machine learning to alleviate the allometric effect in otolith shape-based species discrimination: the role of a triplet loss function

Digital Taxonomist: Identifying Plant Species in Community Scientists' Photographs

A machine learning application for the automatic recognition of planktonic foraminifera in thin sections

One-Shot Learning with Triplet Loss for Vegetation Classification Tasks

Assessing fish abundance from underwater video using deep neural networks

Multi-classification deep neural networks for identification of fish species using camera captured images

Classification of Lakebed Geologic Substrate in Autonomously Collected Benthic Imagery Using Machine Learning

Crosspooled FishNet: transfer learning based fish species classification model

Automated identification of aquatic insects: A case study using deep learning and computer vision techniques