Abstract:In high performance computing environments, we observe an ongoing increase in the available number of cores. For example, the current TOP500 list reveals that nine clusters have more than 1 million cores. This development calls for re-emphasizing performance (scalability) analysis and speedup laws as suggested in the literature (e.g., Amdahl's law and Gustafson's law), with a focus on asymptotic performance. Understanding speedup and efficiency issues of algorithmic parallelism is useful for several purposes, including the optimization of system operations, temporal predictions on the execution of a program, the analysis of asymptotic properties, and the determination of speedup bounds. However, the literature is fragmented and shows a large diversity and heterogeneity of speedup models and laws. These phenomena make it challenging to obtain an overview of the models and their relationships, to identify the determinants of performance in a given algorithmic and computational context, and, finally, to determine the applicability of performance models and laws to a particular parallel computing setting. In this work, I provide a generic speedup (and thus also efficiency) model for homogeneous computing environments. My approach generalizes many prominent models suggested in the literature and allows showing that they can be considered special cases of a unifying approach. The genericity of the unifying speedup model is achieved through parameterization. Considering combinations of parameter ranges, I identify six different asymptotic speedup cases and eight different asymptotic efficiency cases. Jointly applying these speedup and efficiency cases, I derive eleven scalability cases, from which I build a scalability typology. Researchers can draw upon my suggested typology to classify their speedup model and to determine the asymptotic behavior when the number of parallel processing units increases. Also, the description of two computational experiments demonstrates the practical application of the model and the typology. In addition, my results may be used and extended in future research to address various extensions of my setting.

A sufficient condition for a linear speedup in competitive parallel computing

Speedup and efficiency of computational parallelization: A unifying approach and asymptotic analysis

A Comprehensive Linear Speedup Analysis for Asynchronous Stochastic Parallel Optimization from Zeroth-Order to First-Order

A Parallel Function Evaluation Approach for Solution to Large-Scale Equation-Oriented Models.

Convergence Analysis of Asynchronous Stochastic Recursive Gradient Algorithms

Efficiency Analysis of Asynchronic Parallelization of Randomized Algorithms

A Study of Performance Programming of CPU, GPU accelerated Computers and SIMD Architecture

Derivative estimates parallel simulation algorithm based on performance potentials theory

Exploring Potential Parallelism of Sequential Programs with Superblock Reordering

Speculative Parallelization of Sequential Loops on Multicores

Towards Optimality in Parallel Scheduling

Copy or Discard Execution Model for Speculative Parallelization on Multicores

Using Sequential Runtime Distributions for the Parallel Speedup Prediction of SAT Local Search

Competitive parallelism: getting your priorities right

When to Give Up on a Parallel Implementation

Preemptive Scheduling of Parallel Jobs on Multiprocessors

Automatic Parallelization of Sequential Programs

Parallelism Vs. Speculation

Exploiting Parallelism in the Simulation of General Purpose Graphics Processing Unit Program

Potential Thread-Level-parallelism Exploration with Superblock Reordering