Abstract:The importance of difference semantics (e.g., “similar” or “dissimilar”) is recently recognized for declaring dependencies among various types of data, such as numerical values or text values. We propose a novel form of differential dependencies (DDs), which specifies constraints on difference, instead of equality function in traditional dependency notations like functional dependencies. Informally, a differential dependency states that if two tuples have distances on attributes X agreeing with a certain differential function, then their distances on attributes Y should also agree with the corresponding differential function on Y. For example, [date(≤ 7)]→[price(In this thesis, we first report our preliminary work on several theoretical issues of differential dependencies, including formal definitions of DDs and differential keys, subsumption order relation of differential functions, implication of DDs, closure of a differential function, a sound and complete inference system, and minimal cover for DDs. Then, we investigate a practical problem, i.e., how to discover DD s and differential keys from a given sample data. Due to the intrinsic hardness, we develop several pruning methods to improve the discovery efficiency in practice. Moreover, we address differential dependencies that “almost” hold in a given data instance, namely approximate differential dependencies . Several approaches are studied to evaluate how a DD approximately holds in a data instance. Through the extensive experimental evaluation on real data sets, we demonstrate the performance of discovery algorithms, and the effectiveness of DDs in several real applications.

TSDDISCOVER: Discovering Data Dependency for Time Series Data

Effective and Efficient Shape-Based Pattern Detection over Streaming Time Series

Efficient Differential Dependency Discovery

Time Series Data Quality Enhancing Based on Pattern Alignment.

Efficient Determination of Distance Thresholds for Differential Dependencies

DAFDiscover: Robust Mining Algorithm for Dynamic Approximate Functional Dependencies on Dirty Data

Threshold Functional Dependencies for Time Series Data.

Data Dependencies over Big Data: A Family Tree

Data Dependencies Extended for Variety and Veracity: A Family Tree

Discovering Spatio-Temporal Dependencies Based on Time-Lag in Intelligent Transportation Data

Inconsistency Detection in Distributed Big Data

Finding Good-Quality Surprising Patterns in Time Series Data

HATDC: A Holistic Approach for Time Series Data Repairing.

Detecting Inconsistencies in Distributed Data

Data dependencies in the presence of difference

Differential dependencies: Reasoning and discovery

TsQuality: Measuring Time Series Data Quality in Apache IoTDB

Discovering Matching Dependencies

Conditional Functional Dependency Discovery and Data Repair Based on Decision Tree

A Data-Adaptive and Dynamic Segmentation Index for Whole Matching on Time Series

Dependency-based Anomaly Detection: a General Framework and Comprehensive Evaluation