Abstract:Abstract Rapid and reliable identification of insects is important in many contexts, from the detection of disease vectors and invasive species to the sorting of material from biodiversity inventories. Because of the shortage of adequate expertise, there has long been an interest in developing automated systems for this task. Previous attempts have been based on laborious and complex handcrafted extraction of image features, but in recent years it has been shown that sophisticated convolutional neural networks (CNNs) can learn to extract relevant features automatically, without human intervention. Unfortunately, reaching expert-level accuracy in CNN identifications requires substantial computational power and huge training data sets, which are often not available for taxonomic tasks. This can be addressed using feature transfer: a CNN that has been pretrained on a generic image classification task is exposed to the taxonomic images of interest, and information about its perception of those images is used in training a simpler, dedicated identification system. Here, we develop an effective method of CNN feature transfer, which achieves expert-level accuracy in taxonomic identification of insects with training sets of 100 images or less per category, depending on the nature of data set. Specifically, we extract rich representations of intermediate to high-level image features from the CNN architecture VGG16 pretrained on the ImageNet data set. This information is submitted to a linear support vector machine classifier, which is trained on the target problem. We tested the performance of our approach on two types of challenging taxonomic tasks: 1) identifying insects to higher groups when they are likely to belong to subgroups that have not been seen previously and 2) identifying visually similar species that are difficult to separate even for experts. For the first task, our approach reached $CDATA[$CDATA[$>$$92% accuracy on one data set (884 face images of 11 families of Diptera, all specimens representing unique species), and $CDATA[$CDATA[$>$$96% accuracy on another (2936 dorsal habitus images of 14 families of Coleoptera, over 90% of specimens belonging to unique species). For the second task, our approach outperformed a leading taxonomic expert on one data set (339 images of three species of the Coleoptera genus Oxythyrea; 97% accuracy), and both humans and traditional automated identification systems on another data set (3845 images of nine species of Plecoptera larvae; 98.6 % accuracy). Reanalyzing several biological image identification tasks studied in the recent literature, we show that our approach is broadly applicable and provides significant improvements over previous methods, whether based on dedicated CNNs, CNN feature transfer, or more traditional techniques. Thus, our method, which is easy to apply, can be highly successful in developing automated taxonomic identification systems even when training data sets are small and computational budgets limited. We conclude by briefly discussing some promising CNN-based research directions in morphological systematics opened up by the success of these techniques in providing accurate diagnostic tools.

Image-Based Recognition of Parasitoid Wasps Using Advanced Neural Networks

Image-based recognition of parasitoid wasps using advanced neural networks

Effective Insect Recognition Using a Stacked Autoencoder with Maximum Correntropy Criterion

Classifying the Unknown: Identification of Insects by Deep Open-set Bayesian Learning

Illuminating Entomological Dark Matter with DNA Barcodes in an Era of Insect Decline, Deep Learning, and Genomics

Morphology-Based Identification of Bemisia tabaci Cryptic Species Puparia via Embedded Group-Contrast Convolution Neural Network Analysis

Classifying the unknown: Insect identification with deep hierarchical Bayesian learning

Deep learning for identifying bee species from images of wings and pinned specimens

Deep learning powered real-time identification of insects using citizen science data

Assessing the potential for deep learning and computer vision to identify bumble bee species from images

An annotated wing interferential pattern dataset of dipteran insects of medical interest for deep learning

Automated Taxonomic Identification of Insects with Expert-Level Accuracy Using Effective Feature Transfer from Convolutional Networks

DeepWings©: Automatic Wing Geometric Morphometrics Classification of Honey Bee (Apis mellifera) Subspecies Using Deep Learning for Detecting Landmarks

Species identification of phlebotomine sandflies using deep learning and wing interferential pattern (WIP)

Artificial intelligence correctly classifies developmental stages of monarch caterpillars enabling better conservation through the use of community science photographs

Wing Interferential Patterns (WIPs) and machine learning for the classification of some Aedes species of medical interest

Insect Pest Image Recognition: A Few-Shot Machine Learning Approach including Maturity Stages Classification

Deep Neural Network Identification of Limnonectes Species and New Class Detection Using Image Data

A Deep-Learning-Based Detection Approach for the Identification of Insect Species of Economic Importance

An Improved DCGAN-Based Recognition Enhancement Method for American Hyphantria Cunea Larvae Net Curtain Image Dataset

Using DNA barcoding to identify high-priority taxa (Hymenoptera: Ichneumonidae) from Great Smoky Mountains National Park