Abstract:Ocean and climate research benefits from global ocean observation initiatives such as Argo, GLOSS, and EMSO. The Argo network, dedicated to ocean profiling, generates a vast volume of observatory data. However, data quality issues from sensor malfunctions and transmission errors necessitate stringent quality assessment. Existing methods, including machine learning, fall short due to limited labeled data and imbalanced datasets. To address these challenges, we propose an ODEAL framework for ocean data quality assessment, employing AL to reduce human experts' workload in the quality assessment workflow and leveraging outlier detection algorithms for effective model initialization. We also conduct extensive experiments on five large-scale realistic Argo datasets to gain insights into our proposed method, including the effectiveness of AL query strategies and the initial set construction approach. The results suggest that our framework enhances quality assessment efficiency by up to 465.5% with the uncertainty-based query strategy compared to random sampling and minimizes overall annotation costs by up to 76.9% using the initial set built with outlier detectors.

What problem does this paper attempt to address?

This paper attempts to solve the problem of quality assessment of ocean observation data. Specifically, the paper targets data quality problems caused by sensor failures and transmission errors, which may affect the accuracy of scientific research. Existing quality assessment methods, including machine - learning methods, have limitations in dealing with such problems, mainly due to the lack of labeled data and the problem of unbalanced datasets. To address these challenges, the paper proposes an outlier - detection - enhanced active learning framework (Outlier Detection - Enhanced Active Learning, ODEAL) for the quality assessment of ocean data. This framework reduces the workload of human experts in the quality assessment workflow through active learning and effectively initializes the model using an outlier detection algorithm. Experimental results show that the framework performs excellently in improving the efficiency of quality assessment, especially on severely unbalanced datasets. ### Main Contributions 1. **Propose a new data quality assessment framework based on active learning**: This framework aims to reduce the workload of human analysts. 2. **Propose using outlier detection to construct an initial set**: Solve the cold - start problem in highly unbalanced datasets and minimize the overall labeling cost. 3. **Provide empirical evidence through extensive experiments**: Verify the effectiveness of the proposed method. ### Method Overview - **Problem Definition**: Given an observation record \(x_t^s\) measured by sensor \(s\) at a time point \(t\), the task is to assign a quality label \(\hat{y}_t\) to each data instance \(x_t\), where \(\hat{y}_t\in\{0, 1\}\), 1 indicates that the data is problematic and 0 indicates that the data is not problematic. - **Outlier - Detection - Enhanced Active Learning Framework**: This framework is divided into two stages: - **Initial Set Construction**: Use an outlier detector to select the most abnormal samples to form an initial set \(D_I\). - **Active Learning Loop**: Select the most informative samples for labeling through an active learning strategy to gradually optimize the model. - **Initial Set Construction**: Use outlier detection algorithms (such as Isolation Forest, One - Class SVM, and Local Outlier Factor) to identify abnormal samples and construct an initial set. - **Quality Assessment Model Selection**: Investigate multiple classifiers, including k - Nearest Neighbors (KNN), Extreme Gradient Boosting (XGBoost), Category Boosting (CatBoost), and Light Gradient Boosting Machine (LightGBM). - **Query Strategy**: Adopt an uncertainty - based sampling strategy (Uncertainty - based Sampling, US) to select the samples that the model is most uncertain about for labeling. ### Experimental Results - **Effectiveness of AL Query Strategy**: On datasets with high and low error rates, the uncertainty - based sampling strategy (US) outperforms random sampling (RS), especially on severely unbalanced datasets. - **Effectiveness of Using Outlier Detection to Construct an Initial Set**: Compared with random selection, the LOF algorithm can significantly reduce the size of the initial set while containing more abnormal samples, thereby greatly reducing the labeling cost. ### Conclusion The ODEAL framework proposed in the paper performs well in reducing the human workload in ocean data quality assessment, especially when dealing with highly unbalanced datasets. Experimental results show that this framework can significantly improve the performance of the model and reduce the labeling cost. However, there are still some limitations that need further research, such as the optimization of classifier performance on highly unbalanced datasets and the adaptability to real - time applications.

Ocean Data Quality Assessment through Outlier Detection-enhanced Active Learning

YoloXT: A Object Detection Algorithm for Marine Benthos

Machine Learning in Extreme Value Analysis, an Approach to Detecting Harmful Algal Blooms with Long-Term Multisource Satellite Data.

Effective Outlier Detection for Ensuring Data Quality in Flotation Data Modelling Using Machine Learning (ML) Algorithms

Satellite-Based Global Sea Surface Oxygen Mapping and Interpretation with Spatiotemporal Machine Learning.

A Reconstructing Model Based on Time–Space–Depth Partitioning for Global Ocean Dissolved Oxygen Concentration

OAL: Enhancing OOD Detection Using Latent Diffusion

An Ensemble-Based Machine Learning Model for Estimation of Subsurface Thermal Structure in the South China Sea

Reconstructing the Ocean State Using Argo Data and a Data-Driven Method

Orca: Ocean Significant Wave Height Estimation with Spatio-temporally Aware Large Language Models

Using machine learning and Biogeochemical-Argo (BGC-Argo) floats to assess biogeochemical models and optimize observing system design

Online Adaptive Asymmetric Active Learning With Limited Budgets

Predicting Critical Biogeochemistry of the Southern Ocean for Climate Monitoring

Water Resources Quality Indicators Monitoring by Nonlinear Programming and Simulated Annealing Optimization with Ensemble Learning Approaches

A Robust Sparse Sensor Placement Strategy Based on Indicators of Noise for Ocean Monitoring

ESTIMATION OF OCEANIC PARTICULATE ORGANIC CARBON WITH MACHINE LEARNING

Argo Global Ocean Data Assimilation and Its Applications in Short-Term Climate Prediction and Oceanic Analysis

Deep Active Learning in the Open World

Data mining-based machine learning methods for improving hydrological data a case study of salinity field in the Western Arctic Ocean

Enhancing Water Quality Monitoring: an Integrated Machine Learning Framework with Physical Constraints for Imputation and Time Series Downscaling

Remotely sensed estimates of long-term biochemical oxygen demand over Hong Kong marine waters using machine learning enhanced by imbalanced label optimisation