Abstract:PURPOSE: To describe and evaluate a new segmentation method using deep convolutional neural network (CNN), 3D fully connected conditional random field (CRF), and 3D simplex deformable modeling to improve the efficiency and accuracy of knee joint tissue segmentation.METHODS: A segmentation pipeline was built by combining a semantic segmentation CNN, 3D fully connected CRF, and 3D simplex deformable modeling. A convolutional encoder-decoder network was designed as the core of the segmentation method to perform high resolution pixel-wise multi-class tissue classification for 12 different joint structures. The 3D fully connected CRF was applied to regularize contextual relationship among voxels within the same tissue class and between different classes. The 3D simplex deformable modeling refined the output from 3D CRF to preserve the overall shape and maintain a desirable smooth surface for joint structures. The method was evaluated on 3D fast spin-echo (3D-FSE) MR image data sets. Quantitative morphological metrics were used to evaluate the accuracy and robustness of the method in comparison to the ground truth data.RESULTS: The proposed segmentation method provided good performance for segmenting all knee joint structures. There were 4 tissue types with high mean Dice coefficient above 0.9 including the femur, tibia, muscle, and other non-specified tissues. There were 7 tissue types with mean Dice coefficient between 0.8 and 0.9 including the femoral cartilage, tibial cartilage, patella, patellar cartilage, meniscus, quadriceps and patellar tendon, and infrapatellar fat pad. There was 1 tissue type with mean Dice coefficient between 0.7 and 0.8 for joint effusion and Baker's cyst. Most musculoskeletal tissues had a mean value of average symmetric surface distance below 1 mm.CONCLUSION: The combined CNN, 3D fully connected CRF, and 3D deformable modeling approach was well-suited for performing rapid and accurate comprehensive tissue segmentation of the knee joint. The deep learning-based segmentation method has promising potential applications in musculoskeletal imaging.

Arthroscopic Multi-Spectral Scene Segmentation Using Deep Learning

3D Semantic Mapping from Arthroscopy using Out-of-distribution Pose and Depth and In-distribution Segmentation Training

Towards Robotic Knee Arthroscopy: Multi-Scale Network for Tissue-Tool Segmentation

Self-supervised Depth Estimation to Regularise Semantic Segmentation in Knee Arthroscopy

An Automated Real-Time Approach for Image Processing and Segmentation of Fluoroscopic Images and Videos Using a Single Deep Learning Network

One step surgical scene restoration for robot assisted minimally invasive surgery

Collaborative Multi-agent Learning for MR Knee Articular Cartilage Segmentation

Seamless Augmented Reality Integration in Arthroscopy: A Pipeline for Articular Reconstruction and Guidance

m2caiSeg: Semantic Segmentation of Laparoscopic Images using Convolutional Neural Networks

Hierarchical segmentation of surgical scenes in laparoscopy

A Deep Learning-Based Method for Knee Articular Cartilage Segmentation in MRI Images

Multi-task Localization and Segmentation for X-ray Guided Planning in Knee Surgery

A Comprehensive Survey on Bone Segmentation Techniques in Knee Osteoarthritis Research: From Conventional Methods to Deep Learning

A comprehensive survey on recent deep learning-based methods applied to surgical data

Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery

Real-time Surgical Tools Recognition in Total Knee Arthroplasty Using Deep Neural Networks

Surgery Scene Restoration for Robot Assisted Minimally Invasive Surgery

Real-Time Instrument Segmentation in Robotic Surgery using Auxiliary Supervised Deep Adversarial Learning

Deep convolutional neural network for segmentation of knee joint anatomy

Deep Residual Learning for Instrument Segmentation in Robotic Surgery

Segmentation and Classification of Knee Joint Ultrasonic Image Via Deep Learning