Automated Radiological Report Generation For Chest X-Rays With Weakly-Supervised End-to-End Deep Learning

Shuai Zhang,Xiaoyan Xin,Yang Wang,Yachong Guo,Qiuqiao Hao,Xianfeng Yang,Jun Wang,Jian Zhang,Bing Zhang,Wei Wang
DOI: https://doi.org/10.48550/arXiv.2006.10347
2020-06-18
Abstract:The chest X-Ray (CXR) is the one of the most common clinical exam used to diagnose thoracic diseases and abnormalities. The volume of CXR scans generated daily in hospitals is huge. Therefore, an automated diagnosis system able to save the effort of doctors is of great value. At present, the applications of artificial intelligence in CXR diagnosis usually use pattern recognition to classify the scans. However, such methods rely on labeled databases, which are costly and usually have large error rates. In this work, we built a database containing more than 12,000 CXR scans and radiological reports, and developed a model based on deep convolutional neural network and recurrent network with attention mechanism. The model learns features from the CXR scans and the associated raw radiological reports directly; no additional labeling of the scans are needed. The model provides automated recognition of given scans and generation of reports. The quality of the generated reports was evaluated with both the CIDEr scores and by radiologists as well. The CIDEr scores are found to be around 5.8 on average for the testing dataset. Further blind evaluation suggested a comparable performance against human radiologist.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to develop an automated system capable of automatically generating radiology reports for chest X - rays (CXR). Specifically, the paper proposes a model based on deep convolutional neural networks (CNN) and recurrent neural networks (RNN) with an attention mechanism, aiming to learn from a large number of chest X - rays and their corresponding original radiology reports, and being able to directly generate descriptive reports without the need for additional manual annotation of the images. The goal of this system is to reduce the workload of doctors, improve the standardization of diagnosis, and may promote the development of new applications such as remote diagnosis. The paper points out that currently, artificial intelligence methods applied in chest X - ray diagnosis usually rely on pattern recognition to classify scan results, but these methods rely on annotated databases, which are costly and have a relatively high error rate. Therefore, developing a weakly - supervised end - to - end deep - learning model that can directly learn from images and original reports has important practical application value.