Abstract:Cross-matching operation, which is to find corresponding data for the same celestial object or region from multiple catalogues,is indispensable to astronomical data analysis and research. Due to the large amount of astronomical catalogues generated by the ongoing and next-generation large-scale sky surveys, the time complexity of the cross-matching is increasing dramatically. Heterogeneous computing environments provide a theoretical possibility to accelerate the cross-matching, but the performance advantages of heterogeneous computing resources have not been fully utilized. To meet the challenge of cross-matching for substantial increasing amount of astronomical observation data, this paper proposes Heterogeneous-computing-enabled Large Catalogue Cross-matcher (HLC2), a high-performance cross-matching framework based on spherical position deviation on CPU-GPU heterogeneous computing platforms. It supports scalable and flexible cross-matching and can be directly applied to the fusion of large astronomical cataloguesfrom survey missions and astronomical data centres. A performance estimation model is proposed to locate the performance bottlenecks and guide the optimizations. A two-level partitioning strategy is designed to generate an optimized data placement according to the positions of celestial objects to increase throughput. To make HLC2 a more adaptive solution, the architecture-aware task splitting, thread parallelization, and concurrent scheduling strategies are designed and integrated. Moreover, a novel quad-direction strategy is proposed for the boundary problem to effectively balance performance and completeness. We have experimentally evaluated HLC2 using public released catalogue data. Experiments demonstrate that HLC2 scales well on different sizes of catalogues and the cross-matching speed is significantly improved compared to the state-of-the-art cross-matchers.

Continuous Cross Identification In Large-Scale Dynamic Astronomical Data Flow

Rapid Star Locating Method for Ccd Image Based on Cross Projection and Differential Extremum Algorithm

AstroServ: Distributed Database for Serving Large-Scale Full Life-Cycle Astronomical Data

A Robust and Adaptive Star Identification Algorithm Based on Linear Assignment for Multiple Large Field of View Visual Imaging Systems

Automated Real-Time Classification and Decision Making in Massive Data Streams from Synoptic Sky Surveys

Real-Time Data Mining of Massive Data Streams from Synoptic Sky Surveys

Automating Discovery and Classification of Transients and Variable Stars in the Synoptic Survey Era

A transient search using combined human and machine classifications

AstroCatR: a Mechanism and Tool for Efficient Time Series Reconstruction of Large-Scale Astronomical Catalogues

Fast coordinate cross-match tool for large astronomical catalogue

A Command-line Cross-matching tool for modern astrophysical pipelines

HLC2: a highly efficient cross-matching framework for large astronomical catalogues on heterogeneous computing environments

Detection, instance segmentation, and classification for astronomical surveys with deep learning ( deepdisc ): detectron2 implementation and demonstration with Hyper Suprime-Cam data

Detection and Classification of Astronomical Targets with Deep Neural Networks in Wide Field Small Aperture Telescopes

CSST Dense Star Field Preparation: A Framework for Astrometry and Photometry for Dense Star Field Images Obtained by the China Space Station Telescope (CSST)

Towards an Automated Classification of Transient Events in Synoptic Sky Surveys

Efficient Search and Detection of Faint Moving Objects in Image Data

DeepAstroUDA: Semi-Supervised Universal Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detection

A Fuzzy Logic Based Algorithm for Finding Astronomical Objects in Wide-Angle Frames

WRAP: A Tool for Efficient Cross-Identification of Proper Motion Objects Spanning Multiple Surveys

CSST Strong-lensing Preparation: A Framework for Detecting Strong Lenses in the Multicolor Imaging Survey by the China Survey Space Telescope (CSST)