Abstract:Accurately parsing (i.e., segmenting and recognizing) muscles of freely-moving animals such as Drosophila larva in light-sheet fluorescence microscopy images is necessary to study the relationship between muscle activity and animal motions. However, this task is challenging due to the large inter-class similarity and intra-class variance of muscles, as well as the in-homogeneous intensity and blurred boundaries of neighboring muscles. Existing semantic and instance segmentation methods cannot effectively overcome these challenges, resulting in poor segmentation and unreliable classification. In this work, we propose a novel framework named MuscleParseNet that explicitly utilizes sequential and spatial contexts to address these challenges. MuscleParseNet contains a deformable muscle candidate detector (D-CMD) to detect candidate muscles, and a sequential and spatial context-based fine muscle parser (SS-FMP) to refine the candidates. D-CMD boosts Mask RCNN with deformable convolutions to capture shape variations for more accurate muscle segmentation. Moreover, SS-FMP re-classifies the detected candidates by establishing a global spatial context to explicitly reflect spatial relative location, then optimizes the classification using the sequential associations of candidates in adjacent frames, which significantly improves muscle recognition accuracy. Experiments on the synchronized muscle-motion dataset of nearly freely-moving larvae show that MuscleParseNet produces promising results, outperforming state-of-the-art semantic and instance segmentation methods.

MuscleParseNet: a Novel Framework for Parsing Muscles of Drosophila Larva in Light-sheet Fluorescence Microscopy Images