Abstract:Introduction: The difficulties in tea shoot recognition are that the recognition is affected by lighting conditions, it is challenging to segment images with similar backgrounds to the shoot color, and the occlusion and overlap between leaves. Methods: To solve the problem of low accuracy of dense small object detection of tea shoots, this paper proposes a real-time dense small object detection algorithm based on multimodal optimization. First, RGB, depth, and infrared images are collected form a multimodal image set, and a complete shoot object labeling is performed. Then, the YOLOv5 model is improved and applied to dense and tiny tea shoot detection. Secondly, based on the improved YOLOv5 model, this paper designs two data layer-based multimodal image fusion methods and a feature layerbased multimodal image fusion method; meanwhile, a cross-modal fusion module (FFA) based on frequency domain and attention mechanisms is designed for the feature layer fusion method to adaptively align and focus critical regions in intra- and inter-modal channel and frequency domain dimensions. Finally, an objective-based scale matching method is developed to further improve the detection performance of small dense objects in natural environments with the assistance of transfer learning techniques. Results and discussion: The experimental results indicate that the improved YOLOv5 model increases the mAP50 value by 1.7% compared to the benchmark model with fewer parameters and less computational effort. Compared with the single modality, the multimodal image fusion method increases the mAP50 value in all cases, with the method introducing the FFA module obtaining the highest mAP50 value of 0.827. After the pre-training strategy is used after scale matching, the mAP values can be improved by 1% and 1.4% on the two datasets. The research idea of multimodal optimization in this paper can provide a basis and technical support for dense small object detection.

Concealed Object Detection for Millimeter-Wave Images With Normalized Accumulation Map

Enhancing concealed object detection in Active Millimeter Wave Images using wavelet transform

Real-time dangerous objects detection in millimeter wave images

Concealed Object Detection in Millimeter Wave Image Based on Global Correlation of Multi-level Features in Cross-section Sequence

Multi-Dimensional Information Fusion You Only Look Once Network for Suspicious Object Detection in Millimeter Wave Images

Self-Paced Feature Attention Fusion Network for Concealed Object Detection in Millimeter-Wave Image

Real-Time Detection of Concealed Threats with Passive Millimeter Wave and Visible Images via Deep Neural Networks

A Suspicious Multi-Object Detection and Recognition Method for Millimeter Wave SAR Security Inspection Images Based on Multi-Path Extraction Network

Lightweight Detection Method for X-ray Security Inspection with Occlusion

Domain adaptive detection system for concealed objects using millimeter wave images

SMR–YOLO: Multi-Scale Detection of Concealed Suspicious Objects in Terahertz Images

Suspicious Object Detection for Millimeter-Wave Images With Multi-View Fusion Siamese Network

CNN with Pose Segmentation for Suspicious Object Detection in MMW Security Images

X-YOLO: An Efficient Detection Network of Dangerous Objects in X-Ray Baggage Images

Real-time Concealed Object Detection from Passive Millimeter Wave Images Based on the YOLOv3 Algorithm

Fine-YOLO: A Simplified X-ray Prohibited Object Detection Network Based on Feature Aggregation and Normalized Wasserstein Distance

Real-time dense small object detection algorithm based on multi-modal tea shoots

GLNet for Target Detection in Millimeter Wave Images

MMW-Carry: Enhancing Carry Object Detection through Millimeter-Wave Radar-Camera Fusion

High-Performance Detection of Concealed Forbidden Objects on Human Body with Deep Neural Networks Based on Passive Millimeter Wave and Visible Imagery

Learning Coated Adversarial Camouflages for Object Detectors