Abstract:Join ordering and join processing has a huge impact on query execution and can easily affect the query response time by orders of magnitude. In particular, when joins are potentially growing n:m joins, execution can be very expensive. This can be seen by examining the sizes of intermediate results: If a join query produces many redundant tuples that are later eliminated, the query is likely expensive, which is not justified by the query result. This gives the query a diamond shape, with intermediate results larger than the inputs and the output. This occurs frequently in various workloads, particularly, in graph workloads, and also in benchmarks like JOB. We call this issue the diamond problem, and to address it, we propose the diamond hardened join framework, which splits join operators into two suboperators: Lookup & Expand. By allowing these suboperators to be freely reordered by the query optimizer, we improve the runtime of queries that exhibit the diamond problem without sacrificing performance for the rest of the queries. Past theoretical work such as worst-case optimal joins similarly try to avoid huge intermediate results. However, these approaches have significant overheads that impact all queries. We demonstrate that our approach leads to excellent performance both in queries that exhibit the diamond problem and in regular queries that can be handled by traditional binary joins. This allows for a unified approach, offering excellent performance across the board. Compared to traditional joins, queries' performance is improved by up to 500x in the CE benchmark and remains excellent in TPC-H and JOB.

Modular Pipeline Architecture for Accelerating Join Operation in RDBMS

High-Parallelism and Pipelined Architecture for Accelerating Sort-Merge Join on FPGA

Resource-Efficient Parallel Tree-Based Join Architecture on FPGA

TJJE: An efficient algorithm for top-k join on massive data

Modular Serial Pipelined Sorting Architecture For Continuous Variable-Length Sequences With A Very Simple Control Strategy

PI-Join: Efficiently Processing Join Queries on Massive Data

Flexible and Adaptive Stream Join Algorithm.

Parallelizing Multiple Pipelines Of One Query In A Main Memory Database Cluster

Scalable Parallel Join for Huge Tables

Robust Join Processing with Diamond Hardened Joins

Efficient Join Algorithms For Large Database Tables in a Multi-GPU Environment

A multi-table connection strategy based on Hadoop

Join Optimization in the MapReduce Environment for Column-wise Data Store

Scheduling Resources to Multiple Pipelines of One Query in a Main Memory Database Cluster.

Runtime-optimized Multi-way Stream Join Operator for Large-scale Streaming data

The Sort-Merge-Shrink join

RPK-table Based Efficient Algorithm for Join-Aggregate Query on MapReduce.

CDDTA-JOIN: one-pass OLAP algorithm for column-oriented databases

A Distributed Join Algorithm on Separated Data Storage

A Filter-Based Multi-Join Algorithm in Cloud Computing Environment

An Efficient Top-k Spatial Join Query Processing Algorithm on Big Spatial Data.