Early adverse physiological event detection using commercial wearables: challenges and opportunities

Jesse Phipps,Bryant Passage,Kaan Sel,Jonathan Martinez,Milad Saadat,Teddy Koker,Natalie Damaso,Shakti Davis,Jeffrey Palmer,Kajal Claypool,Christopher Kiley,Roderic I. Pettigrew,Roozbeh Jafari

DOI: https://doi.org/10.1038/s41746-024-01129-1

IF: 15.2

2024-05-24

npj Digital Medicine

Abstract:Data from commercial off-the-shelf (COTS) wearables leveraged with machine learning algorithms provide an unprecedented potential for the early detection of adverse physiological events. However, several challenges inhibit this potential, including (1) heterogeneity among and within participants that make scaling detection algorithms to a general population less precise, (2) confounders that lead to incorrect assumptions regarding a participant's healthy state, (3) noise in the data at the sensor level that limits the sensitivity of detection algorithms, and (4) imprecision in self-reported labels that misrepresent the true data values associated with a given physiological event. The goal of this study was two-fold: (1) to characterize the performance of such algorithms in the presence of these challenges and provide insights to researchers on limitations and opportunities, and (2) to subsequently devise algorithms to address each challenge and offer insights on future opportunities for advancement. Our proposed algorithms include techniques that build on determining suitable baselines for each participant to capture important physiological changes and label correction techniques as it pertains to participant-reported identifiers. Our work is validated on potentially one of the largest datasets available, obtained with 8000+ participants and 1.3+ million hours of wearable data captured from Oura smart rings. Leveraging this extensive dataset, we achieve pre-symptomatic detection of COVID-19 with a performance receiver operator characteristic (ROC) area under the curve (AUC) of 0.725 without correction techniques, 0.739 with baseline correction, 0.740 with baseline correction and label correction on the training set, and 0.777 with baseline correction and label correction on both the training and the test set. Using the same respective paradigms, we achieve ROC AUCs of 0.919, 0.938, 0.943 and 0.994 for the detection of self-reported fever, and 0.574, 0.611, 0.601, and 0.635 for detection of self-reported shortness of breath. These techniques offer improvements across almost all metrics and events, including PR AUC, sensitivity at 75% specificity, and precision at 75% recall. The ring allows continuous monitoring for detection of event onset, and we further demonstrate an improvement in the early detection of COVID-19 from an average of 3.5 days to an average of 4.1 days before a reported positive test result.

health care sciences & services,medical informatics

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the challenges faced in using commercially available wearable devices (such as smart bracelets) for early detection of adverse physiological events, and proposes solutions to improve the detection performance. Specifically, the research aims to:

1. **Characterize the performance of existing algorithms in the face of challenges**: These challenges include:
- Heterogeneity among participants, making it difficult for algorithms to be extended to the general population.
- Interfering factors leading to false assumptions about the health status of participants.
- Noise at the sensor level limits the sensitivity of detection algorithms.
- The imprecision of self - reported labels, resulting in data values not matching actual physiological events.

2. **Propose improved algorithms**: For each of the above challenges, the research proposes corresponding algorithms to improve the detection performance. Specific techniques include:
- **Baseline Correction (BC)**: Determine the appropriate baseline for each participant to capture important physiological changes.
- **Label Correction (LC)**: Correct the self - reported identifiers of participants to improve the accuracy of labels.

Through these techniques, the research team has verified the effectiveness of its method on a large - scale data set, showing significant improvements in the early detection of COVID - 19, detection of symptoms such as fever and dyspnea. For example, after applying baseline correction and label correction, the ROC AUC of COVID - 19 increased from 0.725 to 0.777, the ROC AUC of fever increased from 0.919 to 0.994, and the ROC AUC of dyspnea increased from 0.574 to 0.635.

In addition, the research also explores the potential of these techniques in practical applications, especially the potential value in reducing the risk of infectious disease outbreaks.