Abstract:The challenge to fully exploit the potential of existing and upcoming scientific instruments like large single-dish radio telescopes is to process the collected massive data effectively and efficiently. As a "quasi 2D stencil computation" with the "Moore neighborhood pattern", gridding is the most computationally intensive step in data reduction pipeline for radio astronomy studies, enabling astronomers to create correct sky images for further analysis. However, the existing gridding frameworks can either only run on multi-core CPU architecture or do not support high-concurrency, multi-channel data gridding. Their performance is then limited, and there are emerging needs for innovative gridding frameworks to process data from large single-dish radio telescopes like the Five-hundred-meter Aperture Spherical Telescope (FAST). To address those challenges, we developed a High Efficient Gridding framework, HEGrid, by overcoming the above limitations. HEGrid is the first effort to solve the gridding of multi-channel data from the large single-dish radio telescope by multi-pipeline concurrency in the CPU–GPU heterogeneous environment. Specifically, we propose and construct the gridding pipeline in heterogeneous computing environments and achieve multi-pipeline concurrency for high performance multi-channel processing. Furthermore, we propose pipeline-based co-optimization to alleviate the potential negative performance impact of possible intra- and inter-pipeline low computation and I/O utilization, including component share-based redundancy elimination, thread-level data reuse and overlapping I/O and computation. Our experiments are based on both simulated datasets and actual FAST observational datasets. The results show that HEGrid outperforms other state-of-the-art gridding frameworks by up to 5.5x and has robust hardware portability, including AMD Radeon Instinct GPU and NVIDIA GPU.

A Lightweight Execution Framework for Massive Independent Tasks

A Deadline And Budget Constrained Cost-Time Optimization Algorithm For Scheduling Dependent Tasks In Grid Computing

A Public Grid Computing Framework Based on a Hierarchical Combination of Middleware

Application-adaptive Resource Scheduling in a Computational Grid

A DAG-Based XCIGS Algorithm for Dependent Tasks in Grid Environments.

A Study on Task Scheduling in Computational Grid.

Task Scheduling Using Performance-Driven

When Computing Meets Heterogeneous Cluster: Workload Assignment in Graph Computation

A Highly Cost-Effective Task Scheduling Strategy for Very Large Graph Computation.

Design And Implementation Of Task Scheduling Strategies For Massive Remote Sensing Data Processing Across Multiple Data Centers

Multi-objective and multi constrained task scheduling framework for computational grids

HEGrid: A High Efficient Multi-Channel Radio Astronomical Data Gridding Framework in Heterogeneous Computing Environments

An Adaptive Resource Scheduling Algorithm for Computational Grid

Independent Tasks Scheduling on Tree-Based Grid Computing Platforms

Agent-based Grid Load Balancing Using Performance-Driven Task Scheduling

NDA-MM: A New Adaptive Task Scheduling Algorithm Based on the Non-dedicated Constraint Grid

A heuristic scheduling strategy for independent tasks on grid

Taskflow: A Lightweight Parallel and Heterogeneous Task Graph Computing System

A Grid Workflow Framework with High Scalability and Usability

A Dynamic Scheduling Algorithm of Parallel Coarse Grain Tasks in Computational Grid Based on Time-balancing Strategy

GA-Based Algorithm for Task Scheduling on Computational Grid