Abstract:AbstractWith the rise of image-based transcriptomics, spatial gene expression data has become increasingly important for understanding gene regulations from the tissue level down to the cell level. Especially, the gene expression images of Drosophila embryos provide a new data source in the study of Drosophila embryogenesis. It is imperative to develop automatic annotation tools since manual annotation is labor-intensive and requires professional knowledge. Although a lot of image annotation methods have been proposed in the computer vision field, they may not work well for gene expression images, due to the great difference between these two annotation tasks. Besides the apparent difference on images, the annotation is performed at the gene level rather than the image level, where the expression patterns of a gene are recorded in multiple images. Moreover, the annotation terms often correspond to local expression patterns of images, yet they are assigned collectively to groups of images and the relations between the terms and single images are unknown. In order to learn the spatial expression patterns comprehensively for genes, we propose a new method, called FlyIT (image annotation based on Image Tiling and convolutional neural networks for fruit Fly). We implement two versions of FlyIT, learning at image-level and gene-level, respectively. The gene-level version employs an image tiling strategy to get a combined image feature representation for each gene. FlyIT uses a pre-trained ResNet model to obtain feature representation and a new loss function to deal with the class imbalance problem. As the annotation of Drosophila images is a multi-label classification problem, the new loss function considers the difficulty levels for recognizing different labels of the same sample and adjusts the sample weights accordingly. The experimental results on the FlyExpress database show that both the image tiling strategy and the deep architecture lead to the great enhancement of the annotation performance. FlyIT outperforms the existing annotators by a large margin (over 9 percent on AUC and 12 percent on macro F1 for predicting the top 10 terms). It also shows advantages over other deep learning models, including both single-instance and multi-instance learning frameworks.

Flynet 2.0: Drosophila Heart 3d (2d+Time) Segmentation In Optical Coherence Microscopy Images Using A Convolutional Long Short-Term Memory Neural Network

FlyNet 20: drosophila heart 3D (2D + time) segmentation in optical coherence microscopy images using a convolutional long short-term memory neural network

Attention LSTM U-Net model for Drosophila melanogaster heart tube segmentation in optical coherence microscopy images

Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retinal OCT Images with Full and Annotations

High‐precision Drosophila heart segmentation and dynamic cardiac parameter measurement for optogenetics‐OCT‐based cardiac function research

DeU-Net 2.0: Enhanced deformable U-Net for 3D cardiac cine MRI segmentation

FlyIT: <italic>Drosophila</italic> Embryogenesis Image Annotation based on Image Tiling and Convolutional Neural Networks

RotCAtt-TransUNet++: Novel Deep Neural Network for Sophisticated Cardiac Segmentation

Automated neuropil segmentation of fluorescent images for Drosophila brains

Multi-feature fusion CNNs for Drosophila embryo of interest detection

TranSegNet: Hybrid CNN-Vision Transformers Encoder for Retina Segmentation of Optical Coherence Tomography

Retinal Fluid Segmentation and Detection in Optical Coherence Tomography Images using Fully Convolutional Neural Network

ODDF-Net: Multi-object Segmentation in 3D Retinal OCTA Using Optical Density and Disease Features

C2FTFNet: Coarse-to-fine transformer network for joint optic disc and cup segmentation

A Coarse-Fine Collaborative Learning Model for Three Vessel Segmentation in Fetal Cardiac Ultrasound Images

Fully Convolutional Network-Based Multi-Output Model for Automatic Segmentation of Organs at Risk in Thorax.

NeuroFly: A framework for whole-brain single neuron reconstruction

Deep Retinal Image Segmentation: A FCN-Based Architecture with Short and Long Skip Connections for Retinal Image Segmentation

Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retina OCT Images

Ω-Net (Omega-Net): Fully Automatic, Multi-View Cardiac MR Detection, Orientation, and Segmentation with Deep Neural Networks

DCFU‐Net: Rethinking an Effective Attention and Convolutional Architecture for Retinal Vessel Segmentation