Abstract:Abstract Cheap and ubiquitous sensing has made collecting large agricultural datasets relatively straightforward. These large datasets (for instance, citizen science data curation platforms like iNaturalist) can pave the way for developing powerful artificial intelligence (AI) models for detection and counting. However, traditional supervised learning methods require labeled data, and manual annotation of these raw datasets with useful labels (such as bounding boxes or segmentation masks) can be extremely laborious, expensive, and error‐prone. In this paper, we demonstrate the power of zero‐shot computer vision methods—a new family of approaches that require (almost) no manual supervision—for plant phenomics applications. Focusing on insect detection as the primary use case, we show that our models enable highly accurate detection of insects in a variety of challenging imaging environments. Our technical contributions are two‐fold: (a) We curate the Insecta rank class of iNaturalist to form a new benchmark dataset of approximately 6 million images consisting of 2526 agriculturally and ecologically important species, including pests and beneficial insects. (b) Using a vision‐language object detection method coupled with weak language supervision, we are able to automatically annotate images in this dataset with bounding box information localizing the insect within each image. Our method succeeds in detecting diverse insect species present in a wide variety of backgrounds, producing high‐quality bounding boxes in a zero‐shot manner with no additional training cost. This open dataset can serve as a use‐inspired benchmark for the AI community. We demonstrate that our method can also be used for other applications in plant phenomics, such as fruit detection in images of strawberry and apple trees. Overall, our framework highlights the promise of zero‐shot approaches to make high‐throughput plant phenotyping more affordable.

Integrative zero-shot learning for fruit recognition

Zero-Shot Ingredient Recognition by Multi-Relational Graph Convolutional Network

Fine-Grained Grocery Product Recognition by One-Shot Learning.

Intelligent Integrated System for Fruit Detection Using Multi-UAV Imaging and Deep Learning

Zero‐shot insect detection via weak language supervision

Zero-shot transfer learned generic AI models for prediction of optimally ripe climacteric fruits

Recent Advancements in Fruit Detection and Classification Using Deep Learning Techniques

Robust deep learning method for fruit decay detection and plant identification: enhancing food security and quality control

A Novel Greenhouse-Based System for the Detection and Plumpness Assessment of Strawberry Using an Improved Deep Learning Technique

Deep Count: Fruit Counting Based on Deep Simulated Learning

Fruit Detection and Recognition Based on Deep Learning for Automatic Harvesting: An Overview and Review

SE-COTR: A Novel Fruit Segmentation Model for Green Apples Application in Complex Orchard

Zero-Shot Recognition Based on Semantic Embeddings and Deep Clustering

Lightweight and efficient deep learning models for fruit detection in orchards

An Edge Computing Visual System for Vegetable Categorization

Can SAM recognize crops? Quantifying the zero-shot performance of a semantic segmentation foundation model on generating crop-type maps using satellite imagery for precision agriculture

Automated Counting of Rice Panicle by Applying Deep Learning Model to Images from Unmanned Aerial Vehicle Platform

YOLO-SDL: a lightweight wheat grain detection technology based on an improved YOLOv8n model

Zero-Shot Object Counting with Language-Vision Models

Weakly supervised image segmentation for defect-based grading of fresh produce

Distilling knowledge from multiple foundation models for zero-shot image classification