Abstract:In this big data era,the efficiency of join operator is needed to be optimized imperatively,especially for database systems with separated baseline and incremental data.In this database system architecture,the baseline data is stored in the disk as usual,while the incremental data is stored in the memory to achieve both higher transactional processing efficiency and scalability.HBase,BigTable,OceanBase are typical database systems deployed with such separated baseline and incremental data architecture,but they provided join operator with very low efficiency only.The main reasons are as follows:they have to merge the baseline data and incremental data at first;and the network overhead is very heavy because of the complex data model they used.This paper proposes an algorithm for efficient join operator based on separated baseline data and incremental data.It partitions the join attributes into specified ranges first and merges each range on different nodes in parallel.The key point of this algorithm is that it partitions,sorts the baseline data and incremental data separately to achieve even higher parallelism before merge join and avoids the cost of merge of the baseline and incremental data tuples which will not be appeared in the result set.We implement this algorithm based on OceanBase,an open sourced distributed database system.The experimental results confirm that our algorithm can improve the join performance of OceanBase database by a large margin.

The Semi-join Query Optimization In Distributed Database System

Study of query of distributed database based on relation semi join

Research on the Semi-Join-Based Parallel Query Processing Algorithm

Optimization Factor Analysis Of Large-Scale Join Queries On Different Platforms

Join Query Optimization Based on MapReduce under Skewed Data

Distributed Model Based on Data Partition and Load Balance Algorithm

New Distributed Spatial Query Optimization Approach by Using Query Analyzer

Adaptive Multi-join Query Processing in PDBMS

Optimization of sub-query processing in distributed data integration systems

A New Approach of Distributed Query Processing Based on Database Grid

Dynamic Parallel Query Processing for Distributed Objects.

Parallel Join Based on Distributed System OceanBase

Query Optimization of Distributed Database Based on Parallel Genetic Algorithm and Max-Min Ant System

Using Parallel Semi-Join Reduction to Minimize Distributed Query Response Time

The Information Flow Based Query Optimization Techniques In Parallel Object-Oriented Database

A Distributed Join Algorithm on Separated Data Storage

An efficient processing of a chain join with the minimum communication cost in distributed database systems

A SQL transformation based optimization algorithm for distributed spatial query

Dynamic Query Optimization Approach for Semantic Database Grid

Efficient Dynamic Workload Balancing Join Algorithm in Parallel Database

A DSVM BASED PARALLEL HASH JOIN ALGORITHM AND ITS PERFORMANCE EVALUATION