Harnessing Large Vision and Language Models in Agriculture: A Review

Hongyan Zhu,Shuai Qin,Min Su,Chengzhi Lin,Anjie Li,Junfeng Gao

2024-07-29

Abstract:Large models can play important roles in many domains. Agriculture is another key factor affecting the lives of people around the world. It provides food, fabric, and coal for humanity. However, facing many challenges such as pests and diseases, soil degradation, global warming, and food security, how to steadily increase the yield in the agricultural sector is a problem that humans still need to solve. Large models can help farmers improve production efficiency and harvest by detecting a series of agricultural production tasks such as pests and diseases, soil quality, and seed quality. It can also help farmers make wise decisions through a variety of information, such as images, text, etc. Herein, we delve into the potential applications of large models in agriculture, from large language model (LLM) and large vision model (LVM) to large vision-language models (LVLM). After gaining a deeper understanding of multimodal large language models (MLLM), it can be recognized that problems such as agricultural image processing, agricultural question answering systems, and agricultural machine automation can all be solved by large models. Large models have great potential in the field of agriculture. We outline the current applications of agricultural large models, and aims to emphasize the importance of large models in the domain of agriculture. In the end, we envisage a future in which famers use MLLM to accomplish many tasks in agriculture, which can greatly improve agricultural production efficiency and yield.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

The paper primarily explores the potential applications of large models (specifically large language models, LLMs, and large vision models, LVMs) in the agricultural sector and outlines how these models can help address the current challenges faced by agriculture. Specifically, the paper aims to:

1. **Address key issues in agriculture**: In the face of challenges such as pests and diseases, soil degradation, global warming, and food safety, the paper explores how large models can be used to steadily improve agricultural productivity.

2. **Enhance agricultural production efficiency**: By detecting crop pests and diseases, soil quality, seed quality, and other agricultural production tasks, large models can help farmers increase production efficiency and yield.

3. **Assist decision-making**: Large models can process various types of information (such as images, text, etc.) to help farmers make informed decisions.

4. **Promote agricultural technological advancement**: The paper reviews the development history of large language models and large vision models and discusses how they have been applied in other fields, thereby emphasizing the importance of large models in agriculture.

5. **Solve agricultural data processing challenges**: The paper details how large language models handle and generate agricultural data, including information extraction and data generation, to support agricultural production and decision-making.

6. **Build a comprehensive analysis framework**: The paper provides a comprehensive analysis framework, from the history of large models to their applications in agriculture, and the challenges and directions for the future.

In summary, the goal of this paper is to study how large models can be applied in the agricultural sector to solve existing problems and improve agricultural production efficiency. By deeply exploring the potential applications of large models in agriculture, the paper hopes to provide farmers with effective tools and technologies to meet the growing population demands and environmental challenges.