Abstract:This paper presents a novel approach to agricultural disease diagnostics through the integration of Deep Learning (DL) techniques with Visual Question Answering (VQA) systems, specifically targeting the detection of wheat rust. Wheat rust is a pervasive and destructive disease that significantly impacts wheat production worldwide. Traditional diagnostic methods often require expert knowledge and time-consuming processes, making rapid and accurate detection challenging. We drafted a new, WheatRustDL2024 dataset (7998 images of healthy and infected leaves) specifically designed for VQA in the context of wheat rust detection and utilized it to retrieve the initial weights on the federated learning server. This dataset comprises high-resolution images of wheat plants, annotated with detailed questions and answers pertaining to the presence, type, and severity of rust infections. Our dataset also contains images collected from various sources and successfully highlights a wide range of conditions (different lighting, obstructions in the image, etc.) in which a wheat image may be taken, therefore making a generalized universally applicable model. The trained model was federated using Flower. Following extensive analysis, the chosen central model was ResNet. Our fine-tuned ResNet achieved an accuracy of 97.69% on the existing data. We also implemented the BLIP (Bootstrapping Language-Image Pre-training) methods that enable the model to understand complex visual and textual inputs, thereby improving the accuracy and relevance of the generated answers. The dual attention mechanism, combined with BLIP techniques, allows the model to simultaneously focus on relevant image regions and pertinent parts of the questions. We also created a custom dataset (WheatRustVQA) with our augmented dataset containing 1800 augmented images and their associated question-answer pairs. The model fetches an answer with an average BLEU score of 0.6235 on our testing partition of the dataset. This federated model is lightweight and can be seamlessly integrated into mobile phones, drones, etc. without any hardware requirement. Our results indicate that integrating deep learning with VQA for agricultural disease diagnostics not only accelerates the detection process but also reduces dependency on human experts, making it a valuable tool for farmers and agricultural professionals. This approach holds promise for broader applications in plant pathology and precision agriculture and can consequently address food security issues.

Visual Large Language Model for Wheat Disease Diagnosis in the Wild

Crop Disease Identification by Fusing Multiscale Convolution and Vision Transformer.

An in-field automatic wheat disease diagnosis system

Few-Shot Image Classification of Crop Diseases Based on Vision–Language Models

Strawberry Verticillium Wilt Detection Network Based on Multi-Task Learning and Attention

Evaluation of Diverse Convolutional Neural Networks and Training Strategies for Wheat Leaf Disease Identification with Field-Acquired Photographs

Multimodal large language model for wheat breeding: a new exploration of smart breeding

Experimental analysis and evaluation of wide residual networks based agricultural disease identification in smart agriculture system

A Multi-Modal Open Object Detection Model for Tomato Leaf Diseases with Strong Generalization Performance Using PDC-VLD

An Integrated Multi-Model Fusion System for Automatically Diagnosing the Severity of Wheat Fusarium Head Blight

YOLO-SDL: a lightweight wheat grain detection technology based on an improved YOLOv8n model

MnasNet-SimAM: An Improved Deep Learning Model for the Identification of Common Wheat Diseases in Complex Real-Field Environments

Identification of winter wheat pests and diseases based on improved convolutional neural network

DAE-Mask: a Novel Deep-Learning-based Automatic Detection Model for In-Field Wheat Diseases

Integrating deep learning for visual question answering in Agricultural Disease Diagnostics: Case Study of Wheat Rust

A Vegetable Leaf Disease Identification Model Based on Image-Text Cross-Modal Feature Fusion

Real-Time Plant Disease Dataset Development and Detection of Plant Disease Using Deep Learning

Assessing the severity of cotton Verticillium wilt disease from in situ canopy images and spectra using convolutional neural networks

Optimized Detection Models For Wheat Black Tip Disease And Multiple Classification Results

Enhancing Plant Disease Detection: A Novel CNN-Based Approach with Tensor Subspace Learning and HOWSVD-MD

IMPROVED YOLOv8-BASED AUTOMATED DETECTION OF WHEAT LEAF DISEASES