Anatomical Conditioning for Contrastive Unpaired Image-to-Image Translation of Optical Coherence Tomography Images

Marc S. Seibel,Hristina Uzunova,Timo Kepp,Heinz Handels
2024-04-08
Abstract:For a unified analysis of medical images from different modalities, data harmonization using image-to-image (I2I) translation is desired. We study this problem employing an optical coherence tomography (OCT) data set of Spectralis-OCT and Home-OCT images. I2I translation is challenging because the images are unpaired, and a bijective mapping does not exist due to the information discrepancy between both domains. This problem has been addressed by the Contrastive Learning for Unpaired I2I Translation (CUT) approach, but it reduces semantic consistency. To restore the semantic consistency, we support the style decoder using an additional segmentation decoder. Our approach increases the similarity between the style-translated images and the target distribution. Importantly, we improve the segmentation of biomarkers in Home-OCT images in an unsupervised domain adaptation scenario. Our data harmonization approach provides potential for the monitoring of diseases, e.g., age related macular disease, using different OCT devices.
Image and Video Processing,Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the data - mismatch problem during image - style transfer between different optical coherence tomography (OCT) devices. Specifically, the paper focuses on how to use image - to - image (I2I) translation techniques to make the Spectralis - OCT images in the clinical environment consistent in style with the Home - OCT images in the home environment, so as to facilitate the monitoring and analysis of diseases (such as age - related macular degeneration, AMD). Since there are significant differences in noise and motion artifacts in OCT images obtained by different devices, direct image - style transfer is very challenging. In addition, these image data are unpaired, that is, there is no direct one - to - one correspondence, which makes the traditional bidirectional mapping method difficult to apply. To overcome these challenges, the paper proposes an unpaired image - to - image translation method based on contrastive learning and introduces additional anatomical conditions to enhance the effect of style transfer. This method not only improves the similarity of style - transferred images in the target distribution, but also significantly improves the segmentation effect of biomarkers in Home - OCT images in the unsupervised domain adaptation scenario. In this way, the paper aims to provide an effective data homogenization scheme to support the monitoring of diseases using different OCT devices.