Abstract:In banks, governments, and Internet companies, inconsistent data problems may often arise when various information systems are collecting, processing, and updating data due to human or equipment reasons. The emergence of inconsistent data makes it impossible to obtain correct information from the data and reduces its availability. Such problems may be fatal in data-intensive enterprises, which causes huge economic losses. Moreover, it is very difficult to clean inconsistent data in databases, especially for data containing conditional functional dependencies with built-in predicates (CFDPs), because it tends to contain more candidate repair values. For the inconsistent data containing CFDPs to detect incomplete and repair difficult problems in databases, we propose a dependency lifting algorithm (DLA) based on the maximum dependency set (MDS) and a reparation algorithm (C-Repair) based on integrating the minimum cost and attribute correlation, respectively. In detection, we find recessive dependencies from the original dependency set to obtain the MDS and improve the original algorithm by dynamic domain adjustment, which extends the applicability to continuous attributes and improves the detection accuracy. In reparation, we first set up a priority queue (PQ) for elements to be repaired based on the minimum cost idea to select a candidate element; then, we treat the corresponding conflict-free instance ( I n v ) as the training set to learn the correlation among attributes and compute the weighted distance (WDis) between the tuple of the candidate element and other tuples in I n v according to the correlation; and, lastly, we perform reparation based on the WDis and re-compute the PQ after each reparation round to improve the efficiency, and use a label, flag, to mark the repaired elements to ensure the convergence at the same time. By setting up a contrast experiment, we compare the DLA with the CFDPs based algorithm, and the C-Repair with a cost-based, interpolation-based algorithm on a simulated instance and a real instance. From the experimental results, the DLA and C-Repair algorithms have better detection and repair ability at a higher time cost.

An Improved Forwards Information Repairing Algorithm for Target Identification and Its Applications

Interpretability Based Neural Network Repair

Robust Network Enhancement from Flawed Networks.

The Application of a Damage Detection Method Using Artificial Neural Network and Train-Induced Vibrations on a Simplified Railway Bridge Model

An Improved Multi-Task Approach for SHM Missing Data Reconstruction Using Attentive Neural Process and Meta-Learning

Adaptive Learning of Dynamic Bayesian Networks with Changing Structures by Detecting Geometric Structures of Time Series

Iterative Online Fault Identification Scheme for High-Voltage Circuit Breaker Utilizing a Lost Data Repair Technique

In High Concurrency Environment Network Information Missing Data Repair Method Emulation

Improved Diffluence Algorithm and Its Application to the Best Repairing Path for Distribution Network

Additive Dynamic Bayesian Networks for Enhanced Feature Learning in Soft Sensor Modeling

A Robust Sparse Bayesian Learning Method for the Structural Damage Identification by a Mixture of Gaussians

From data to network structure-Reconstruction of dynamic networks

Repairing Deep Neural Networks Based on Behavior Imitation

Inconsistent Data Cleaning Based on the Maximum Dependency Set and Attribute Correlation

Repairing Neural Networks by Leaving the Right Past Behind

A novel structural damage detection method using a hybrid IDE-BP model

Influence of Measurement Errors on Structural Damage Identification Using Artificial Neural Networks

DeepRepair: Style-Guided Repairing for Deep Neural Networks in the Real-World Operational Environment

NNrepair: Constraint-based Repair of Neural Network Classifiers

Dynamic Information Dissemination Model Incorporating Non-Adjacent Node Interaction

A Neural Network Based on Genetic Algorithm for Structural Damage Detection