Abstract:As an important driving factor of emerging Internet-of-Things (IoT) applications, machine learning algorithms are currently facing the challenge of how to "clean" data noise, that is introduced during the training process (e.g., asynchronous execution and lossy data compression and quantization). In an attempt to guarantee data quality, various data cleaning approaches have been proposed to filter out abnormal data entries based on the global data distribution. However, most existing data cleaning approaches are based on a centralized paradigm and thus cannot be applied to future edge-based IoT applications, where each edge node (EN) has only a limited view of the global data distribution. Moreover, the increasing demand for privacy preservation largely prevents ENs from combining their data for centralized cleaning. In this study, we propose a federated data cleaning protocol, coined as FedClean, for edge intelligence (EI) scenarios that is designed to achieve data cleaning without compromising data privacy. More specifically, different ENs first generate Boolean shares of their data and distribute them to two noncolluding servers. These two servers then run the FedClean protocol to privately and efficiently compute the attribute value frequency (AVF) scores of the collected data entries, which are then sorted in ascending order via a bitonic sorting network without revealing their values. As a result, data entries with lower AVF scores are considered as abnormal and filtered out. The security, efficiency, and effectiveness of the proposed approach are then demonstrated via concrete security analysis and comprehensive experiments.

RTClean: Context-aware Tabular Data Cleaning using Real-time OFDs

LLMClean: Context-Aware Tabular Data Cleaning via LLM-Generated OFDs

Contextual Data Cleaning with Ontology Functional Dependencies

Robust and Automatic Data Cleansing Method for Short-Term Load Forecasting of Distribution Feeders

Discovery and Contextual Data Cleaning with Ontology Functional Dependencies

Time Series Data Cleaning under Expressive Constraints on Both Rows and Columns

RetClean: Retrieval-Based Data Cleaning Using Foundation Models and Data Lakes

ActiveClean: Interactive Data Cleaning While Learning Convex Loss Models

Detecting Abnormal Events on Binary Sensors in Smart Home Environments

A data cleaning framework based on user feedback

An Open Data Cleaning Framework Based on Semantic Rules for Continuous Auditing

Interactive and Deterministic Data Cleaning

Batchwise Probabilistic Incremental Data Cleaning

Horizon: Scalable Dependency-driven Data Cleaning

EXPERIENCE: Algorithms and Case Study for Explaining Repairs with Uniform Profiles over IoT Data

Crowdcleaner: A Data Cleaning System Based on Crowdsourcing

Temporal rules discovery for web data cleaning

Distance-based Data Cleaning: A Survey (Technical Report)

Cleanits

A Hybrid Data Cleaning Framework Using Markov Logic Networks

Federated Data Cleaning: Collaborative and Privacy-Preserving Data Cleaning for Edge Intelligence