Abstract:Agile analytics can help organizations to gain and sustain a competitive advantage by making timely decisions. Approximate query processing (AQP) is one of the useful approaches in agile analytics, which facilitates fast queries on big data by leveraging a pre-computed sample. One problem such a sample faces is that when new data is being imported, re-sampling is most likely needed to keep the sample fresh and AQP results accurate enough. Re-sampling from scratch for every batch of new data, called the full re-sampling method and adopted by many existing AQP works, is obviously a very costly process, and a much quicker incremental sampling process, such as reservoir sampling, may be used to cover the newly arrived data. However, incremental update methods suffer from the fact that the sample size cannot be increased, which is a problem when the underlying data distribution dramatically changes and the sample needs to be enlarged to maintain the AQP accuracy. This paper proposes an adaptive sample update (ASU) approach that avoids re-sampling from scratch as much as possible by monitoring the data distribution, and uses instead an incremental update method before a re-sampling becomes necessary. The paper also proposes an enhanced approach (T-ASU), which tries to enlarge the sample size without re-sampling from scratch when a bit of query inaccuracy is tolerable to further reduce the sample update cost. These two approaches are integrated into a state-of-the-art AQP engine for an extensive experimental study. Experimental results on both real-world and synthetic datasets show that the two approaches are faster than the full re-sampling method while achieving almost the same AQP accuracy when the underlying data distribution continuously changes.

Remember The Past: Self-Tuning Histograms Based On Query-Feedback Logs

Self-Learning Histograms for Changing Workloads.

Self-adaptive statistics management for efficient query processing

Automatic Statistics Collection In Oscar Dbms

Differentially Private Histogram Publication for Dynamic Datasets: an Adaptive Sampling Approach.

Proactive and Reactive Multi-Dimensional Histogram Maintenance for Selectivity Estimation

Adaptive algorithm of histogram maintain in data stream processing

Enhancing Online Index Tuning with a Learned Tuning Diagnostic.

Learning-based Sample Tuning for Approximate Query Processing in Interactive Data Exploration

Counting Graph Matches with Adaptive Statistics Collection

An Agile Sample Maintenance Approach for Agile Analytics

SHA: QoS-Aware Software and Hardware Auto-Tuning for Database Systems

A Learning-Based Approach to Estimate Statistics of Operators in Continuous Queries: a Case Study.

Flattened Exponential Histogram for Sliding Window Queries over Data Streams

Facilitating Database Tuning with Hyper-Parameter Optimization: a Comprehensive Experimental Evaluation

Survey on performance optimization for database systems

Outlier Detection Techniques for SQL and ETL Tuning

An Adaptive Skew Handling Join Algorithm for Large-scale Data Analysis

Facilitating Database Tuning with Hyper-Parameter Optimization

Improving Fault-Localization Accuracy by Referencing Debugging History to Alleviate Structure Bias in Code Suspiciousness

QCFE: An efficient Feature engineering for query cost estimation