Abstract:In this study, an innovative approach based on multimodal data and the transformer model was proposed to address challenges in agricultural disease detection and question-answering systems. This method effectively integrates image, text, and sensor data, utilizing deep learning technologies to profoundly analyze and process complex agriculture-related issues. The study achieved technical breakthroughs and provides new perspectives and tools for the development of intelligent agriculture. In the task of agricultural disease detection, the proposed method demonstrated outstanding performance, achieving a precision, recall, and accuracy of 0.95, 0.92, and 0.94, respectively, significantly outperforming the other conventional deep learning models. These results indicate the method's effectiveness in identifying and accurately classifying various agricultural diseases, particularly excelling in handling subtle features and complex data. In the task of generating descriptive text from agricultural images, the method also exhibited impressive performance, with a precision, recall, and accuracy of 0.92, 0.88, and 0.91, respectively. This demonstrates that the method can not only deeply understand the content of agricultural images but also generate accurate and rich descriptive texts. The object detection experiment further validated the effectiveness of our approach, where the method achieved a precision, recall, and accuracy of 0.96, 0.91, and 0.94. This achievement highlights the method's capability for accurately locating and identifying agricultural targets, especially in complex environments. Overall, the approach in this study not only demonstrated exceptional performance in multiple tasks such as agricultural disease detection, image captioning, and object detection but also showcased the immense potential of multimodal data and deep learning technologies in the application of intelligent agriculture.

Transformer-based Detection of Multiword Expressions in Flower and Plant Names

BERT(s) to Detect Multiword Expressions

Semantics of Multiword Expressions in Transformer-Based Models: A Survey

Deep Learning Methods for Extracting Metaphorical Names of Flowers and Plants

MFE-transformer: Adaptive English text named entity recognition method based on multi-feature extraction and transformer

Processing Plant Diseases Using Transformer Model

Enhancing Automatic Modulation Recognition for IoT Applications Using Transformers

Dual-Branch Cross-Attention Network for Micro-Expression Recognition with Transformer Variants

Application of Multimodal Transformer Model in Intelligent Agricultural Disease Detection and Question-Answering Systems

Hybrid convolutional neural network and multilayer perceptron vision transformer model for wheat species classification task: E-ResMLP+

Transformer-based approaches to Sentiment Detection

Multilingual Multiword Expression Identification Using Lateral Inhibition and Domain Adaptation

Transformer Fault-based Named Entity Recognition Method

A wheat spike detection method based on Transformer

Transformer-based Named Entity Recognition with Combined Data Representation

Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition

Wavelet Tree Transformer: Multihead Attention With Frequency-Selective Representation and Interaction for Remote Sensing Object Detection

MDETR - Modulated Detection for End-to-End Multi-Modal Understanding

Multimodal Fine-Grained Transformer Model for Pest Recognition

Detecting Unseen Multiword Expressions in American Sign Language

Transformer-Based Multimodal Emotional Perception for Dynamic Facial Expression Recognition in the Wild