Abstract:Purpose Differential counting of blood cells is the basis of diagnostic hematology. In many circumstances, identification of cells in bone marrow smears is the golden standard for diagnosis. Presently, methods for automatic differential counting of peripheral blood are readily available commercially. However, morphological assessment and differential counting of bone marrow smears are still performed manually. This procedure is tedious, time-consuming and laden with high inter-operator variation. In recent years, deep neural networks have proven useful in many medical image recognition tasks, such as diagnosis of diabetic retinopathy, and detection of cancer metastasis in lymph nodes. However, there has been no published work on using deep neural networks for complete differential counting of entire bone marrow smear. In this work, we present the results of using deep convolutional neural network for automatic differential counting of bone marrow nucleated cells. Materials & Methods The bone marrow smears from patients with either benign or malignant disorders in National Taiwan University Hospital were recruited in this study. The bone marrow smears are stained with Liu's stain, a modified Romanowsky stain. Digital images of the bone marrow smears were taken using 1000x oil immersion lens and 20MP color CCD camera on a single microscope with standard illumination and white-balance settings. The contour of each nucleated cell was artificially defined. These cells were then divided into a training/validation set and a test set. Each cell was then classified into 1 of the 11 categories (blast, promyelocyte, neutrophilic myelocyte, neutrophilic metamyelocyte, neutrophils, eosinophils and precursors, basophil, monocyte and precursors, lymphocyte, erythroid lineage cells, and invalid cell). In training/validation set, the classification of each cell was annotated once by experienced medical technician or hematologist. The annotated dataset was used to train a Path-Aggregation Network for instance segmentation task. In test set, cell classification was annotated by three medical technicians or hematologists; only over 2/3 consensus was regarded as valid. After the neural network model was fully trained, the ability of the model to classify and detect bone marrow nucleated cells was evaluated in terms of precision, recall and accuracy. During the model training, we used group normalization and stochastic gradient descent optimizer for training. Random noise, Gaussian blur, rotation, contrast and color shift were also used as means for data augmentation. Results The digital images of 150 bone marrow aspirate smears were taken for this study. They included 61 for acute leukemia, 39 for lymphoma, 2 for myelodysplastic syndrome (MDS), 2 for myeloproliferative neoplasm (MPN), 10 for MDS/MPN, 12 for multiple myeloma, 4 for hemolytic anemia, 9 for aplastic anemia, 8 for infectious etiology and 3 for solid cancers. The final data contained 5927 images and 187730 nucleated bone marrow cells, which were divided into 2 sets: 5630 images containing 170966 cells as the training/validation set, and 297 images containing 16764 cells as the test set. Among the 16764 cells annotated in test set, 15676 cells (93.6 %) reached over 2/3 consensus. The trained neural network achieved 0.832 recall and 0.736 precision for cell detection task, 0.79 mean intersection over union (IOU) for cell segmentation task, mean average precision of 0.659 and accuracy of 0.801 for cell classification. For individual cell categories, the model performs the best with "erythroid-lineage-cells" (0.971 recall, 0.935 precision) and the worst with "monocyte-and-precursors" (0.825 recall, 0.337 precision). Conclusions We have created the largest and the most comprehensive annotated bone marrow smear image dataset for deep neural network training. Compared with previous works, our approach is more practical for clinical application because it is able to take in an entire field of smear and generate differential counts without any other preprocessing steps. Current results are highly encouraging. With continued expansion of dataset, our model would be more precise and clinically useful. Figure Disclosures Yeh: aether AI: Other: CEO and co-founder. Yang:aether AI: Employment. Tien:Novartis: Honoraria; Daiichi Sankyo: Honoraria; Celgene: Research Funding; Roche: Honoraria; Johnson &Johnson: Honoraria; Alexion: Honoraria; BMS: Honoraria; Roche: Research Funding; Celgene: Honoraria; Pfizer: Honoraria; Abbvie: Honoraria. Hsu:aether AI: Employment.

A deep learning approach for automatic recognition of abnormalities in the cytoplasm of neutrophils

Classification of peripheral blood neutrophils using deep learning

Deep Learning for Discovering and Identifying Morphological Heterogeneity of Neutrophils in Primary Hematological Diseases Based on Bone Marrow Neutrophils Analysis

Fine-Grained Classification of Neutrophils with Hybrid Loss

A Deep Learning Approach for the Morphological Recognition of Reactive Lymphocytes in Patients with COVID-19 Infection

Acute Lymphoblastic Leukemia Classification from Microscopic Images using Convolutional Neural Networks

Evaluation of deep learning training strategies for the classification of bone marrow cell images

Transfer Learning for Identifying Morphological Heterogeneity of Neutrophils Nuclei in Hematological Diseases Based on Nuclei Semantic Segmentations of Bone Marrow Smear

Highly accurate differentiation of bone marrow cell morphologies using deep neural networks on a large image data set

Neural Stain Normalization and Unsupervised Classification of Cell Nuclei in Histopathological Breast Cancer Images

Convolutional Neural Network and decision support in medical imaging: case study of the recognition of blood cell subtypes

Microcell‐Net: A deep neural network for multi‐class classification of microscopic blood cell images

Immunohistochemistry annotations enhance AI identification of lymphocytes and neutrophils in digitized H&E slides from inflammatory bowel disease

Deep Learning-based Glomerulus Detection and Classification with Generative Morphology Augmentation in Renal Pathology Images

Detecting cells in intravital video microscopy using a deep convolutional neural network

A Hematologist-Level Deep Learning Algorithm (BMSNet) for Assessing the Morphologies of Single Nuclear Balls in Bone Marrow Smears: Algorithm Development

Automatic Bone Marrow Cell Identification and Classification By Deep Neural Network

Establishment and Verification of Neural Network for Rapid and Accurate Cytological Examination of Four Types of Cerebrospinal Fluid Cells

Classification and counting of cells in brightfield microscopy images: an application of convolutional neural networks

Segmentation, feature extraction and classification of leukocytes leveraging neural networks, a comparative study

Deep Generative Classification of Blood Cell Morphology