Abstract:Background: Middle ear diseases such as otitis media and middle ear effusion, for which diagnoses are often delayed or misdiagnosed, are among the most common issues faced by clinicians providing primary care for children and adolescents. Artificial intelligence (AI) has the potential to assist clinicians in the detection and diagnosis of middle ear diseases through imaging. Methods: Otoendoscopic images obtained by otolaryngologists from Taipei Veterans General Hospital in Taiwan between Jany 1, 2011 to Dec 31, 2019 were collected retrospectively and de-identified. The images were entered into convolutional neural network (CNN) training models after data pre-processing, augmentation and splitting. To differentiate sophisticated middle ear diseases, nine CNN-based models were constructed to recognize middle ear diseases. The best-performing models were chosen and ensembled in a small CNN for mobile device use. The pretrained model was converted into the smartphone-based program, and the utility was evaluated in terms of detecting and classifying ten middle ear diseases based on otoendoscopic images. A class activation map (CAM) was also used to identify key features for CNN classification. The performance of each classifier was determined by its accuracy, precision, recall, and F1-score. Findings: A total of 2820 clinical eardrum images were collected for model training. The programme achieved a high detection accuracy for binary outcomes (pass/refer) of otoendoscopic images and ten different disease categories, with an accuracy reaching 98.0% after model optimisation. Furthermore, the application presented a smooth recognition process and a user-friendly interface and demonstrated excellent performance, with an accuracy of 97.6%. A fifty-question questionnaire related to middle ear diseases was designed for practitioners with different levels of clinical experience. The AI-empowered mobile algorithm's detection accuracy was generally superior to that of general physicians, resident doctors, and otolaryngology specialists (36.0%, 80.0% and 90.0%, respectively). Our results show that the proposed method provides sufficient treatment recommendations that are comparable to those of specialists. Interpretation: We developed a deep learning model that can detect and classify middle ear diseases. The use of smartphone-based point-of-care diagnostic devices with AI-empowered automated classification can provide real-world smart medical solutions for the diagnosis of middle ear diseases and telemedicine. Funding: This study was supported by grants from the Ministry of Science and Technology (MOST110-2622-8-075-001, MOST110-2320-B-075-004-MY3, MOST-110-2634-F-A49 -005, MOST110-2745-B-075A-001A and MOST110-2221-E-075-005), Veterans General Hospitals and University System of Taiwan Joint Research Program (VGHUST111-G6-11-2, VGHUST111c-140), and Taipei Veterans General Hospital (V111E-002-3).

Detecting abnormal heart sound using mobile phones and on-device IConNet

An edge-device-compatible algorithm for valvular heart diseases screening using phonocardiogram signals with a lightweight convolutional neural network and self-supervised learning

Segmental Convolutional Neural Networks for Detection of Cardiac Abnormality With Noisy Heart Sound Recordings

[Artificial Intelligence Technology in Cardiac Auscultation Screening for Congenital Heart Disease: Present and Future].

Real-time Arrhythmia Detection Using Convolutional Neural Networks

Personal Heart Health Monitoring Based on 1D Convolutional Neural Network

Heart sound recognition technology based on convolutional neural network

Cardiac anomaly detection considering an additive noise and convolutional distortion model of heart sound recordings

A Computer-Aided Heart Valve Disease Diagnosis System Based on Machine Learning

Classification of Heart Sounds Using Multi-Branch Deep Convolutional Network and LSTM-CNN

Automatic Classification of Normal–Abnormal Heart Sounds Using Convolution Neural Network and Long-Short Term Memory

Transfer Learning Models for Detecting Six Categories of Phonocardiogram Recordings

Heart Sound Classification Considering Additive Noise and Convolutional Distortion

Heart Sounds Monitor and Analysis in Noisy Environments

A Machine Learning Approach for Delineating Similar Sound Symptoms of Respiratory Conditions on a Smartphone

CardioHelp: A smartphone application for beat-by-beat ECG signal analysis for real-time cardiac disease detection using edge-computing AI classifiers

Data Augmentation based Convolutional Neural Network for Auscultation

Heart Sound Abnormality Detection From Multi-Institutional Collaboration: Introducing a Federated Learning Framework

Recognizing Abnormal Heart Sounds Using Deep Learning

A VLSI Chip for the Abnormal Heart Beat Detection Using Convolutional Neural Network

Smartphone-based artificial intelligence using a transfer learning algorithm for the detection and diagnosis of middle ear diseases: A retrospective deep learning study