Abstract:Retrieving similar data has drawn many research efforts in the literature due to its importance in data mining, database and information retrieval. This problem is challenging when the data is incomplete. In previous research, data incompleteness refers to the fact that data values for some dimensions are unknown. However, in many practical applications (e.g., data collection by sensor network under bad environment), not only data values but even data dimension information may also be missing, which will make most similarity query algorithms infeasible. In this work, we propose the novel similarity query problem on dimension incomplete data and adopt a probabilistic framework to model this problem. For this problem, users can give a distance threshold and a probability threshold to specify their retrieval requirements. The distance threshold is used to specify the allowed distance between query and data objects and the probability threshold is used to require that the retrieval results satisfy the distance condition at least with the given probability. Instead of enumerating all possible cases to recover the missed dimensions, we propose an efficient approach to speed up the retrieval process by leveraging the inherent relations between query and dimension incomplete data objects. During the query process, we estimate the lower/upper bounds of the probability that the query is satisfied by a given data object, and utilize these bounds to filter irrelevant data objects efficiently. Furthermore, a probability triangle inequality is proposed to further speed up query processing. According to our experiments on real data sets, the proposed similarity query method is verified to be effective and efficient on dimension incomplete data.

A probability based approach for processing dimension missing data

Processing Methods for Incomplete Information Systems Based on Rough Sets

Probabilistic Similarity Query on Dimension Incomplete Data

Handling Missing Data in Decision Trees: A Probabilistic Approach

Searching Dimension Incomplete Databases

Missing value imputation using unsupervised machine learning techniques

Missing Data Exploration: Highlighting Graphical Presentation of Missing Pattern.

DBT-DMAE: An Effective Multivariate Time Series Pre-Train Model under Missing Data

Mining Incomplete Data Using Global and Saturated Probabilistic Approximations Based on Characteristic Sets and Maximal Consistent Blocks

Power Grid Missing Data Filling Method Based on Historical Data Mining Assisted Multi-dimensional Scenario Analysis

Handling Nonmonotone Missing Data with Available Complete-Case Missing Value Assumption

Dimension Reduction for Data with Heterogeneous Missingness

A Novel Approach for Imputation of Missing Attribute Values for Efficient Mining of Medical Datasets - Class Based Cluster Approach

Missing Data Filling Algorithm for Big Data-Based Map-Reduce Technology

Toward Systematic Considerations of Missingness in Visual Analytics

An approach to dealing with missing values in heterogeneous data using k-nearest neighbors

CLINCH: clustering incomplete high-dimensional data for data mining application

Missing Value Estimation for Mixed-Attribute Data Sets

Review for Handling Missing Data with special missing mechanism

An Approach to Find Missing Values in Medical Datasets

Internal Data Imputation in Data Warehouse Dimensions