Abstract:When monitoring crop growth using top-down images of the plant canopies, leaves in agricultural fields appear very dense and significantly overlap each other. Moreover, the image can be affected by external conditions such as background environment and light intensity, impacting the effectiveness of image segmentation. To address the challenge of segmenting dense and overlapping plant leaves under natural lighting conditions, this study employed a Bilayer Convolutional Network (BCNet) method for accurate leaf segmentation across various lighting environments. The major contributions of this study are as follows: (1) Utilized Fully Convolutional Object Detection (FCOS) for plant leaf detection, incorporating ResNet-50 with the Convolutional Block Attention Module (CBAM) and Feature Pyramid Network (FPN) to enhance Region of Interest (RoI) feature extraction from canopy top-view images. (2) Extracted the sub-region of the RoI based on the position of the detection box, using this region as input for the BCNet, ensuring precise segmentation. (3) Utilized instance segmentation of canopy top-view images using BCNet, improving segmentation accuracy. (4) Applied the Varifocal Loss Function to improve the classification loss function in FCOS, leading to better performance metrics. The experimental results on cucumber canopy top-view images captured in glass greenhouse and plastic greenhouse environments show that our method is highly effective. For cucumber leaves at different growth stages and under various lighting conditions, the Precision, Recall and Average Precision (AP) metrics for object recognition are 97%, 94% and 96.57%, respectively. For instance segmentation, the Precision, Recall and Average Precision (AP) metrics are 87%, 83% and 84.71%, respectively. Our algorithm outperforms commonly used deep learning algorithms such as Faster R-CNN, Mask R-CNN, YOLOv4 and PANet, showcasing its superior capability in complex agricultural settings. The results of this study demonstrate the potential of our method for accurate recognition and segmentation of highly overlapping leaves in diverse agricultural environments, significantly contributing to the application of deep learning algorithms in smart agriculture.

BCMask: a Finer Leaf Instance Segmentation with Bilayer Convolution Mask

LeafMask: Towards Greater Accuracy on Leaf Segmentation

Cucumber Leaf Segmentation Based on Bilayer Convolutional Network

Growing Instance Mask on Leaf

Instance Mask Growing on Leaf.

Instance Segmentation and Classification Method for Plant Leaf Images Based on ISC-MRCNN and APS-DCCNN

Fusion of Mask RCNN and Attention Mechanism for Instance Segmentation of Apples under Complex Background

RefineMask: Towards High-Quality Instance Segmentation with Fine-Grained Features

Improving Image Segmentation with Boundary Patch Refinement

An Improved Mask R-CNN Method for Weed Segmentation

Hierarchical Mask2Former: Panoptic Segmentation of Crops, Weeds and Leaves

Boundary-Area Enhanced Module for Instance Segmentation

MIX-NET: Deep Learning-Based Point Cloud Processing Method for Segmentation and Occlusion Leaf Restoration of Seedlings

Accurate segmentation of green fruit based on optimized mask RCNN application in complex orchard

DynaMask: Dynamic Mask Selection for Instance Segmentation

Mask Transfiner for High-Quality Instance Segmentation

Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation

Multi-Species Individual Tree Segmentation and Identification Based on Improved Mask R-CNN and UAV Imagery in Mixed Forests

DB-BlendMask: Decomposed Attention and Balanced BlendMask for Instance Segmentation of High-Resolution Remote Sensing Images

Occluded Object Segmentation Based on Bilayer Decoupling Strategy and Attention Mechanism br

Exploiting Boundary Loss for the Hierarchical Panoptic Segmentation of Plants and Leaves