Abstract:Low quality of data is a serious problem in the new era of big data, which can severely reduce the usability of data, mislead or bias the querying, analyzing and mining, and leads to huge loss. Incomplete data is common in low quality data, and it is necessary to determine the data completeness of a dataset to provide hints for follow-up operations on it. Little existing work focuses on the completeness of a dataset, and such work views all missing values as unknown values. In this paper, we study how to determine real data completeness of a relational dataset. By taking advantage of given functional dependencies, we aim to determine some missing attribute values by other tuples and capture the really missing attribute cells. We propose a data completeness model, formalize the problem of determining the real data completeness of a relational dataset, and give a lower bound of the time complexity of this problem. Two optimal algorithms to determine the data completeness of a dataset for different cases are proposed. We empirically show the effectiveness and the scalability of our algorithms on both real-world data and synthetic data.

The Or Data Complement Method For Incomplete Decision Tables

Processing Methods for Incomplete Information Systems Based on Rough Sets

Entropy Based Attribute Reduction Approach For Incomplete Decision Table

An Incremental Rule Extracting Algorithm Based on Pawlak Reduction

A Complete Method To Incomplete Information Systems

Data Mining in Incomplete Information

Incomplete Concept Lattice Data Analytical Method Research Based on Rough Set Theory

A Novel Measure Of Compatibility And Methods Of Missing Attribute Values Treatment In Decision Tables

Rough Set Model and Precision Reduction in Incomplete and Fuzzy Decision Information Systems

Survey of Decision Table Research of Attribute Reduction

Reduction in decision table based on pair-wise complementarity of condition attributes.

A Novel Approach of Rough Conditional Entropy-Based Attribute Selection for Incomplete Decision System

Dominance-based Fuzzy Rough Set Approach for Incomplete Interval-Valued Data

A Complete Attribute Reduction Algorithm in Rough Set Theory

A Refined Classification Method with Tolerance Relation-Based Rough Sets for Incomplete Decision Systems

Research on Relative Reducts of Rough Set Model in Consistent and Inconsistent Decision Tables

Minimum Cost Attribute Reduction in Incomplete Systems under Decision-Theoretic Rough Set Model

Decision Table Attribute Reduction Algorithm Based on Extended Information Entropy

The significance measure for attributes in inconsistent decision tables

Determining the Real Data Completeness of a Relational Dataset