TransientX: A high-performance single-pulse search package

Yunpeng Men,Ewan Barr

DOI: https://doi.org/10.1051/0004-6361/202348247

2024-01-29

Abstract:Radio interferometers composed of a large array of small antennas posses large fields of view, coupled with high sensitivities. For example, the Karoo Array Telescope (MeerKAT) achieves a gain of up to 2.8 K/Jy across its $>1\ deg ^2$ field of view. This capability significantly enhances the survey speed for pulsars and fast transients. It also introduces challenges related to the high data rate, which reaches a few Tb/s for MeerKAT, and it requires substantial computing power. To handle the high data rate of surveys, we have developed a high-performance single-pulse search software called "TransientX". This software integrates multiple processes into one pipeline, which includes radio-frequency interference mitigation, dedispersion, matched filtering, clustering, and candidate plotting. In TransientX we developed an efficient CPU-based dedispersion implementation using the sub-band dedispersion algorithm. Additionally TransientX employs the density-based spatial clustering of applications with noise (DBSCAN) algorithm to eliminate duplicate candidates, using an efficient implementation based on the kd-tree data structure. We also calculate the decrease of signal-to-noise ratio resulting from dispersion measure, boxcar width, spectral index, and pulse-shape mismatches. Remarkably, we find that the decrease of signal-to-noise ratio resulting from the mismatch between a boxcar-shaped template and a Gaussian-shaped pulse with scattering remains relatively small, at approximately 9 , even when the scattering timescale is ten times that of the pulse width. Additionally, the decrease in the signal-to-noise ratio resulting from the spectral index mismatch becomes significant with multi-octave receivers. We have benchmarked the individual processes, including dedispersion, matched filtering, and clustering. Our dedispersion implementation can be executed in real time using a single CPU core on data with 4096 dispersion measure trials, which consist of 4096 channels and have a time resolution of 153 microseconds. Overall TransientX offers the capability for efficient CPU-only real-time single-pulse searching.

astronomy & astrophysics

What problem does this paper attempt to address?

This paper aims to address the high - data - rate processing challenges faced by single - pulse searches in radio astronomy. Specifically, the paper introduces a high - performance single - pulse search software package named "TransientX", which integrates multiple processing steps, including radio - frequency interference suppression, de - dispersion, matched filtering, clustering, and candidate map plotting. Solving these problems is crucial for increasing the survey speed of pulsars and fast transient sources, especially when using radio interferometers with large fields of view and high sensitivity (such as MeerKAT), whose data rates can reach several terabytes per second and require enormous computing power.

### Main problems
1. **High - data - rate processing**: With the development of radio interferometer technology, especially arrays composed of a large number of small antennas, such as MeerKAT, large - field - of - view and high - sensitivity observations can be achieved. This significantly increases the survey speed of pulsars and fast transient sources, but at the same time, it also brings the problem of high data rates, which requires powerful computing power to process.

2. **Efficient processing flow**: To deal with the high data rate, the paper has developed the "TransientX" software package, which realizes efficient single - pulse searches by integrating multiple processing steps. These steps include:
- **Radio - frequency interference suppression**: Remove non - astronomical signals to reduce false alarms.
- **De - dispersion**: Correct the delays between different frequencies to improve the signal - to - noise ratio of the pulse.
- **Matched filtering**: Detect pulse signals.
- **Clustering**: Eliminate duplicate candidate pulses.
- **Candidate map plotting**: Generate a chart for each candidate pulse for subsequent analysis.

### Solutions
- **Efficient algorithms**: TransientX adopts a variety of efficient algorithms, such as sub - band - based de - dispersion algorithms and kd - tree - based density - based spatial clustering of applications with noise (DBSCAN) algorithms, to improve processing speed and efficiency.
- **Optimized implementation**: By using AVX2 instructions and the OpenMP library, TransientX achieves real - time processing on a single CPU core, significantly improving performance.
- **Parameter space expansion**: The improved performance enables searches in a larger parameter space, while saving energy and environmental costs and reducing data - processing time to support triggered observations.

### Experimental results
- **Benchmark tests**: The paper has carried out benchmark tests on each processing step, including de - dispersion, matched filtering, and clustering. The results show that TransientX exhibits high efficiency when processing large - scale data sets and can real - time process MMGPS data with 2048 frequency channels and 153 - microsecond time resolution on a single CPU core.
- **Performance improvement**: Compared with the previous - generation software BEAR, the performance of TransientX has been improved by approximately one order of magnitude.

### Discussion
- **Spectral index mismatch**: The paper explores the impact of spectral index mismatch on the signal - to - noise ratio and finds that for a typical pulsar spectral index, the signal - to - noise ratio drops by about 6%. But for a narrow - band FRB spectrum, the signal - to - noise ratio drop may be greater.
- **Pulse - shape mismatch**: The paper also studies the signal - to - noise ratio drop caused by pulse - shape mismatch when using a box - shaped template for matched filtering. This effect is quantified through a likelihood - ratio test.

In conclusion, through the development and optimization of the "TransientX" software package, this paper effectively solves the problems of high - data - rate processing and efficient single - pulse searches, providing a powerful tool for radio astronomy research.