RATING: Medical knowledge-guided rheumatoid arthritis assessment from multimodal ultrasound images via deep learning

Zhanping Zhou,Chenyang Zhao,Hui Qiao,Ming Wang,Yuchen Guo,Qian Wang,Rui Zhang,Huaiyu Wu,Fajin Dong,Zhenhong Qi,Jianchu Li,Xinping Tian,Xiaofeng Zeng,Yuxin Jiang,Feng Xu,Qionghai Dai,Meng Yang

DOI: https://doi.org/10.1016/j.patter.2022.100592

2022-09-29

Abstract:Multimodal ultrasound has demonstrated its power in the clinical assessment of rheumatoid arthritis (RA). However, for radiologists, it requires strong experience. In this paper, we propose a rheumatoid arthritis knowledge guided (RATING) system that automatically scores the RA activity and generates interpretable features to assist radiologists' decision-making based on deep learning. RATING leverages the complementary advantages of multimodal ultrasound images and solves the limited training data problem with self-supervised pretraining. RATING outperforms all of the existing methods, achieving an accuracy of 86.1% on a prospective test dataset and 85.0% on an external test dataset. A reader study demonstrates that the RATING system improves the average accuracy of 10 radiologists from 41.4% to 64.0%. As an assistive tool, not only can RATING indicate the possible lesions and enhance the diagnostic performance with multimodal ultrasound but it can also enlighten the road to human-machine collaboration in healthcare.