cuPSO: GPU Parallelization for Particle Swarm Optimization Algorithms

Chuan-Chi Wang,Chun-Yen Ho,Chia-Heng Tu,Shih-Hao Hung

DOI: https://doi.org/10.1145/3477314.3507142

2023-12-03

Abstract:Particle Swarm Optimization (PSO) is a stochastic technique for solving the optimization problem. Attempts have been made to shorten the computation times of PSO based algorithms with massive threads on GPUs (graphic processing units), where thread groups are formed to calculate the information of particles and the computed outputs for the particles are aggregated and analyzed to find the best solution. In particular, the reduction-based method is considered as a common approach to handle the data aggregation and analysis for the calculated particle information. Nevertheless, based on our analysis, the reduction-based method would suffer from excessive memory accesses and thread synchronization overheads. In this paper, we propose a novel algorithm to alleviate the above overheads with the atomic functions. The threads within a thread group update the calculated results atomically to the intra-group data queue conditionally, which prevents the frequent accesses to the memory as done by the parallel reduction operations. Furthermore, we develop an enhanced version of the algorithm to alleviate the synchronization barrier among the thread groups, which is achieved by allowing the thread groups to run asynchronously and updating to the global, lock-protected variables occasionally if necessary. Our experimental results show that our proposed algorithm running on the Nvidia GPU is about 200 times faster than the serial version executed by the Intel Xeon CPU. Moreover, the novel algorithm outperforms the state-of-the-art method (the parallel reduction approach) by a factor of 2.2.

Distributed, Parallel, and Cluster Computing,Performance

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use the Graphics Processing Unit (GPU) to accelerate the execution speed of the Particle Swarm Optimization (PSO) algorithm in the PSO algorithm, especially to reduce the overhead caused by memory access and thread synchronization. PSO is a stochastic optimization technique based on swarm intelligence and is used to solve various optimization problems. However, when the PSO algorithm is executed in parallel on the GPU, the traditional parallel reduction method will generate a large performance overhead due to frequent memory access and thread synchronization operations.

To alleviate these problems, the author proposes a new algorithm - the Queue Algorithm, and reduces the frequent access to memory by introducing atomic functions to update the calculation results in the data queue within the group. In addition, the author also develops an enhanced version of the algorithm - the Queue Lock Algorithm, which further reduces the synchronization barriers between thread groups by allowing thread groups to run asynchronously and occasionally update to global, protected variables when necessary.

The experimental results show that the proposed algorithm is approximately 200 times faster on the Nvidia GPU than the serial version on the Intel Xeon CPU, and 2.2 times faster than the state - of - the - art method (i.e., the parallel reduction method). This indicates that by optimizing memory access and reducing synchronization overhead, the execution efficiency of the PSO algorithm can be significantly improved on the GPU.