Abstract:The Join operator, as one of the most expensive and commonly used operators in database systems, plays a substantial role in Database Management System (DBMS) performance. Among the many different Join algorithms studied over the last decades, Hybrid Hash Join (HHJ) has proven to be one of the most efficient and widely-used join algorithms. While the performance of HHJ depends largely on accurate statistics and information about the input relations, it may not always be practical or possible for a system to have such information available. The design of HHJ depends on many details to perform well. This paper is an experimental and analytical study of the trade-offs in designing a robust and dynamic HHJ operator. We revisit the design and optimization techniques suggested by previous studies through extensive experiments, comparing them with other algorithms designed by us or used in related studies. We explore the impact of the number of partitions on the performance of HHJ and propose a lower bound and a default value for the number of partitions. We continue by designing and evaluating different partition insertion techniques to maximize memory utilization with the least CPU cost. In addition, we consider a comprehensive set of algorithms for dynamically selecting a partition to spill and compare the results against previously published studies. We then present two alternative growth policies for spilled partitions and study their effectiveness using experimental and model-based analyses. These algorithms have been implemented in the context of Apache AsterixDB and evaluated under different scenarios such as variable record sizes, different distributions of join attributes, and different storage types, including HDD, SSD, and Amazon Elastic Block Store (Amazon EBS).

A DSVM BASED PARALLEL HASH JOIN ALGORITHM AND ITS PERFORMANCE EVALUATION

Performance Analysis for Parallel Hash Join Algorithms Based on DSVM and Message Passing

Performance Evaluation for Distributed Join Based on MapReduce.

Join Query Optimization Based on MapReduce under Skewed Data

Parallel Join Based on Distributed System OceanBase

HASH HASH -1 PARALLEL JOIN ALGORITHM

Research on the Semi-Join-Based Parallel Query Processing Algorithm

Efficient Dynamic Workload Balancing Join Algorithm in Parallel Database

An Effective High-Performance Multiway Spatial Join Algorithm with Spark

Massive Parallel Join in NUMA Architecture

A Distributed Join Algorithm on Separated Data Storage

Design and Performance Evaluation of Parallel Algorithms for Path Expressions in Object Database Systems on NOW

A Design Space Exploration and Evaluation for Main-Memory Hash Joins in Storage Class Memory.

Parallel Structural Join Algorithms

Parallel In-Memory Evaluation of Spatial Joins

Analysis and Evaluation of Resources Utility Strategies for Parallel Path Expression Algorithms on DSVM

Efficient Parallel Partition-Based Algorithms for Similarity Search and Join with Edit Distance Constraints

Efficient String Similarity Join in Multi-Core and Distributed Systems.

Parallel Two-Way Enumeration Join Based on Multi-cores Environment

Design Trade-offs for a Robust Dynamic Hybrid Hash Join (Extended Version)

An Efficient Method for Parallel Multi-Dimensional Join and Aggregation