Abstract:Abstract Background Electroencephalogram (EEG) monitoring and objective seizure identification is an essential clinical investigation for some patients with epilepsy. Accurate annotation is done through a time-consuming process by EEG specialists. Computer-assisted systems for seizure detection currently lack extensive clinical utility due to retrospective, patient-specific, and/or irreproducible studies that result in low sensitivity or high false positives in clinical tests. We aim to significantly reduce the time and resources on data annotation by demonstrating a continental generalization of seizure detection that balances sensitivity and specificity. Methods This is a prospective inference test of artificial intelligence on nearly 14,590 hours of adult EEG data from patients with epilepsy between 2011 and 2019 in a hospital in Sydney, Australia. The inference set includes patients with different types and frequencies of seizures across a wide range of ages and EEG recording hours. The artificial intelligence (AI) is a convolutional long short-term memory network that is trained on a USA-based dataset. The Australian set is about 16 times larger than the US training dataset with very long interictal periods (between seizures), which is way more realistic than the training set and makes our false positives highly reliable. We validated our inference model in an AI-assisted mode with a human expert arbiter and a result review panel of expert neurologists and EEG specialists on 66 sessions to demonstrate achievement of the same performance with over an order-of-magnitude reduction in time. Findings Our inference on 1,006 EEG recording sessions on the Australian dataset achieved 76.68% with nearly 56 [0, 115] false alarms per 24 hours on average, against legacy ground-truth annotations by human experts, conducted independently over nine years. Our pilot test of 66 sessions with a human arbiter, and reviewed ground truth by a panel of experts, confirmed an identical human performance of 92.19% with an AI-assisted system, while the time requirements reduce significantly from 90 to 7.62 minutes on average. Interpretation Accurate and objective seizure counting is an important factor in epilepsy. An AI-assisted system can help improve efficiency and accuracy alongside human experts, particularly in low and middle-income countries with limited expert human resources. Fundings SOAR Fellowship from The University of Sydney, a Microsoft AI for Accessibility grant, and a Research Training Program (RTP) support provided by the Australian Government. Research in context Evidence before this study During the development of our artificial intelligence (AI) system, we did a systematic review of the scientific literature with search via PubMed for research articles published on seizure detection with the following inclusion criteria: (1) Tests or inference evaluation is conducted on large-scale clinical EEG data; (2) Generalization is attempted or potentials for generalization is considered, e.g., in commercialized tools; (3) Seizure detection delay and real-time (aka. online) operation were not considered critical in this context as long as the test was conducted on raw EEG data. Note that ICU seizure detection or portable seizure alert systems are relying on detection delay and real-time needs. Our keywords include “prospective seizure detection”, “automated seizure detection”, “non-patient specific seizure detection”, “seizure detection on continuous EEG”, and “deep learning-based seizure detection” and “machine learning-based seizure detection”. We found that the only two categories of works meet our criteria: two research papers published in 2020 and works published by commercial tools developers. We cited a recent review of 89 deep learning-based seizure detection, all of which are retrospective. One work from Stanford reported seizure detection on all ages (pediatric to adult ages) using post-acquisition EEG recordings and provided an avenue for independent evaluation by providing a test on a publicly available Temple University Hospital (TUH) EEG dataset. The other work pivoted on algorithmic-assisted real-time seizure risk monitoring in continuous EEG in neonatal intensive care unit (NICU) with 128 neonates (32 with seizures) showing about 20% improvement in seizure identification over 130 neonates (38 with seizures) with no algorithmic assistance. Commercial tools we studied are Encevis (EpiScan), Besa, and Persyst. There is a recent comparative study on these tools on 81 patients. Encevis is reported as the best performing tool, and hence we provided a comparative study with Encevis ver. 1.9.2. Encevis is also the only tool that provided an avenue for comparative study on publicly available EEG data. The Stanford work, published in 2020, confirms many false positives with Persyst 13. We excluded our tests on Persyst 14 as it highly under-performed relative to Encevis. Only Stanford’s work provides code availability. We compared our results with Stanford’s work outcome and provided pilot test results with the Encevis (EpiScan) tool on the Australian dataset, which shows a considerably lower sensitivity. Added value of this study To the best of our knowledge, the current study is the first continental generalization that demonstrates the potential to achieve an expert human-level seizure recognition rate in a clinical setting and in just a fraction of time. The two datasets used in this study are recorded with different infrastructure, which adds to the independence of inference from hardware types and improves clinical utility. This is particularly important as 80% of patients with epilepsy live in low and middle-income countries with limited resources, particularly EEG specialists and neurologists. Implications of all the available evidence Our results support the potential benefits of deep learning AI in clinical settings for seizure recognition and its contribution to significant sensitivity over available solutions. Our AI-assisted system achieves more than a ten-fold increase in time efficiency and reports identical performance to human experts for EEG interpretation with access to great neurophysiology support and auxiliary data. Our findings, particularly our tests on an available commercial tool, recommend that the evaluation, test, or inference in AI systems be performed on different datasets, with diverse infrastructures, and on large-scale and realistic sets with long interictal periods.

Updating Beliefs using New Evidence from a Diagnostic Decision-Support Aid in a Nurse-Led First-Seizure Clinic

Estimating the likelihood of epilepsy from clinically noncontributory electroencephalograms using computational analysis: A retrospective, multisite case–control study

A sustainable artificial-intelligence-augmented digital care pathway for epilepsy: Automating seizure tracking based on electroencephalogram data using artificial intelligence

Efficient diagnosis leads to better care: streamlining epilepsy diagnosis pathways

Real-time seizure detection in paediatric intensive care patients: the RESET child brain protocol

Computer-assisted analysis of routine EEG to identify hidden biomarkers of epilepsy: A systematic review

Seizure Assessment and Forecasting With Efficient Rapid-EEG: A Retrospective Multicenter Comparative Effectiveness Study

Perceived seizure risk in epilepsy â€" Chronic electronic surveys with and without concurrent EEG

Evaluating the single seizure clinic model: Findings from a Canadian Center

Evidentiary Significance of Routine EEG in Refractory Cases: A Paradigm Shift in Psychiatry

Impact of an Interactive, Animation-Based Electroencephalography Curriculum on Learner Confidence and Knowledge

Implementation of a Novel Seizure Assessment Tool for Unified Seizure Evaluation Improves Nurse Response

Development and Validation of a Diagnostic Aid for Convulsive Epilepsy in Sub-Saharan Africa: a Retrospective Case-Control Study

Development of Expert-Level Classification of Seizures and Rhythmic and Periodic Patterns During EEG Interpretation

Interrater Reliability of Expert Electroencephalographers Identifying Seizures and Rhythmic and Periodic Patterns in EEGs

Methods for Identifying Epilepsy Surgery Targets Using Invasive EEG: A Systematic Review

The role of EEG in predicting post-stroke seizures and an updated prognostic model (SeLECT-EEG)

Continental generalization of an AI system for clinical seizure recognition

Electrographic seizure monitoring with a novel, wireless, single-channel EEG sensor

Association of an Electroencephalography-Based Risk Score With Seizure Probability in Hospitalized Patients

‘Knowledge exchange’ workshops to optimise development of a risk prediction tool to assist conveyance decisions for suspected seizures – Part of the Risk of ADverse Outcomes after a Suspected Seizure (RADOSS) project