Abstract:Order dependencies (ODs) are recently proposed to describe a relationship of ordering between lists of attributes. It is typically too costly to design ODs manually, since the number of possible ODs is of a factorial complexity in the number of attributes. To this end, automatic discovery techniques for ODs are developed. In practice, data is frequently updated, especially with tuple insertions. Existing techniques do not lend themselves well to these situations, since it is prohibitively expensive to recompute all ODs from scratch after every update. In this paper, we make a first effort to investigate incremental OD discovery techniques in response to tuple insertions. Given a relation D, a set \(\varSigma \) of valid and minimal ODs on D, and a set \(\triangle D\) of tuple insertions to D, it is to find, changes \(\triangle \varSigma \) to \(\varSigma \) that makes \(\varSigma \oplus \triangle \varSigma \) a set of valid and minimal ODs on \(D + \triangle D\). Note that \(\triangle \varSigma \) contains both new ODs to be added to \(\varSigma \) and outdated ODs to be removed from \(\varSigma \). Specifically, (1) We formalize the incremental OD discovery problem. Although the incremental discovery problem has a same complexity as its batch (non-incremental) counterpart in terms of traditional complexity, we show that it has good data locality. It is linear in the size of \(\triangle D\) to validate on \(D+\triangle D\) any OD \(\varphi \) that is valid on D. (2) We present effective incremental OD discovery techniques, leveraging an intelligent traversal strategy for finding \(\triangle \varSigma \) and chosen indexes to minimize access to D. Our approach computes \(\triangle \varSigma \) based on ODs in \(\varSigma \), and is independent of the size of D. (3) Using real-life data, we experimentally verify that our approach substantially outperforms its batch counterpart by orders of magnitude.

Approximate Order Dependency Discovery

Discovery of Approximate Lexicographical Order Dependencies

Efficient Set-Based Order Dependency Discovery with a Level-Wise Hybrid Strategy

Repairing Data Violations With Order Dependencies

Efficient Bidirectional Order Dependency Discovery

Incremental Discovery of Order Dependencies on Tuple Insertions

Discovering approximate implicit domain orders through order dependencies

Discovering Reliable Approximate Functional Dependencies

Fast Algorithm for Embedded Order Dependency Validation (Extended Version)

Fast incremental discovery of pointwise order dependencies

Efficient Differential Dependency Discovery

Approximate Functional Dependencies Based Query Evaluation Improvement

DAFDiscover: Robust Mining Algorithm for Dynamic Approximate Functional Dependencies on Dirty Data

OPTIMIZED ALGORITHM OF DISCOVERING FUNCTIONAL DEPENDENCIES WITH DEGREES OF SATISFACTION

Efficient Discovery of Ontology Functional Dependencies

Discovery and Contextual Data Cleaning with Ontology Functional Dependencies

Discovering Matching Dependencies

EulerFD: an Efficient Double-Cycle Approximation of Functional Dependencies

Efficient Determination of Distance Thresholds for Differential Dependencies

Data dependencies in the presence of difference

Mining Approximate Acyclic Schemes from Relations