Inpainting Pathology in Lumbar Spine MRI with Latent Diffusion

Colin Hansen,Simas Glinskis,Ashwin Raju,Micha Kornreich,JinHyeong Park,Jayashri Pawar,Richard Herzog,Li Zhang,Benjamin Odry
2024-06-05
Abstract:Data driven models for automated diagnosis in radiology suffer from insufficient and imbalanced datasets due to low representation of pathology in a population and the cost of expert annotations. Datasets can be bolstered through data augmentation. However, even when utilizing a full suite of transformations during model training, typical data augmentations do not address variations in human anatomy. An alternative direction is to synthesize data using generative models, which can potentially craft datasets with specific attributes. While this holds promise, commonly used generative models such as Generative Adversarial Networks may inadvertently produce anatomically inaccurate features. On the other hand, diffusion models, which offer greater stability, tend to memorize training data, raising concerns about privacy and generative diversity. Alternatively, inpainting has the potential to augment data through directly inserting pathology in medical images. However, this approach introduces a new challenge: accurately merging the generated pathological features with the surrounding anatomical context. While inpainting is a well established method for addressing simple lesions, its application to pathologies that involve complex structural changes remains relatively unexplored. We propose an efficient method for inpainting pathological features onto healthy anatomy in MRI through voxelwise noise scheduling in a latent diffusion model. We evaluate the method's ability to insert disc herniation and central canal stenosis in lumbar spine sagittal T2 MRI, and it achieves superior Frechet Inception Distance compared to state-of-the-art methods.
Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning
What problem does this paper attempt to address?
The paper aims to address key issues in medical image data augmentation, particularly the lack of data and class imbalance in lumbar spine MRI images. Specifically, the authors propose a weighted noise scheduling method based on Latent Diffusion Models (LDMs) to insert pathological features into healthy anatomical structures. This method can better handle complex human anatomical variations and maintain consistency with the surrounding anatomical environment. Existing data augmentation techniques, while helpful in increasing the diversity of the training set, struggle to generate anatomically reasonable variations. Generative Adversarial Networks (GANs) may produce anatomically inaccurate features, and although diffusion models are more stable, they tend to memorize training data when the training samples are limited, raising concerns about privacy and generative diversity. The interpolation method based on LDMs proposed in this paper can effectively insert pathological features such as disc herniation and central canal stenosis into MRI images while preserving anatomical consistency. Experimental results show that this method outperforms existing baseline methods.