Zhicheng Cao,Natalia A. Schmid,Kevin Bandura,Duncan R. Lorimer,Morgan Dameron,Katelyn Crockett,Clayton Grubick,Andreas Schmid,Shaonan Zheng
Abstract:Radio-frequency interference (RFI) is becoming an increasingly significant problem for most radio telescopes. Working with Green Bank Telescope data from PSR J1730+0747 in the form of complex-valued channelized voltages and their respective high-resolution power spectral densities, we evaluate a variety of statistical measures to characterize RFI. As a baseline for performance comparison, we use median absolute deviation (MAD) in complex channelized voltage data and spectral kurtosis (SK) in power spectral density data to characterize and filter out RFI. From a new perspective, we implement the Shapiro-Wilks (SW) test for normality and two information theoretical measures, spectral entropy (SE) and spectral relative entropy (SRE), and apply them to mitigate RFI. The baseline RFI mitigation algorithms are compared against our novel RFI detection algorithms to determine how effective and robust the performance is. Except for MAD, we find significant improvements in signal-to-noise ratio through the application of SE, symmetrical SRE, asymmetrical SRE, SK, and SW. These algorithms also do a good job of characterizing broadband RFI. Time- and frequency-variable RFI signals are best detected by SK and SW tests.
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is the radio - frequency interference (RFI) detection problem in radio astronomy. Specifically, the author uses data from the Green Bank Telescope (GBT), which are presented in the form of complex - valued channelized voltages and their corresponding high - resolution power spectral densities. In order to better identify and filter out RFI, the author evaluates multiple statistics and proposes new methods to characterize RFI.
### Main research content:
1. **Background and problem description**:
- Radio - frequency interference (RFI) is an increasingly serious problem in modern radio astronomical observations, affecting the observation quality of most radio telescopes.
- RFI has a wide range of sources, including natural phenomena (such as lightning, aurora) and man - made devices (such as radar, television, mobile phones, satellites, etc.).
- Traditional RFI detection methods have limitations in dealing with complex and variable interference signals, especially in terms of real - time processing and efficiency.
2. **Research objectives**:
- Propose and evaluate new RFI detection algorithms, especially for application on the original complex - valued channelized voltage data.
- Use new methods such as Shapiro - Wilks normality test, spectral entropy (SE) and spectral relative entropy (SRE) to detect and characterize RFI.
- Compare the performance of the new methods with traditional methods (such as median absolute deviation MAD and spectral kurtosis SK), and evaluate their effectiveness and robustness in suppressing RFI.
3. **Main contributions**:
- Propose using SE, SRE and Shapiro - Wilks normality test as new RFI detection methods.
- Design an independent channel processing scheme suitable for parallel implementation on FPGA or GPU.
- Analyze the signal - to - noise ratio (S/N) of each method by generating the folded pulse profile of PSR J1713 + 0747, and verify the effectiveness of the new methods.
- Compare in detail the results of RFI mask generation under different thresholds and determine the optimal threshold.
### Key formulas:
- **Shapiro - Wilks normality test statistic**:
\[
W=\frac{\sum_{i = 1}^{n}a_ix_{(i)}}{\sqrt{\sum_{i = 1}^{n}(x_i-\bar{x})^2}}
\]
where \(x_i\) is the \(i\) - th original unsorted sample, \(x_{(i)}\) is the \(i\) - th sorted sample, \(\bar{x}\) is the sample mean, and \(a_i\) is an element of the coefficient vector.
- **Spectral entropy (SE)**:
\[
H(X)=-\sum_{x}p(x)\log p(x)
\]
where \(p(x)\) is the probability mass function of the random variable \(X\).
- **Spectral relative entropy (SRE)**:
\[
D(p, q)=\sum_{x}p(x)\log\frac{p(x)}{q(x)}
\]
where \(p(x)\) and \(q(x)\) are two probability mass functions defined on the same outcome set.
### Conclusion:
Through experimental verification, the author finds that using SE, symmetric SRE, asymmetric SRE, SK and SW tests can significantly improve the signal - to - noise ratio and perform well in characterizing broadband RFI. In particular, SK and SW tests are most effective in detecting RFI signals with time and frequency variations. These new methods provide effective tools and technical support for future RFI detection.