Abstract:The detection problem in statistical signal processing can be succinctly formulated: given $m$ (possibly) signal bearing, $n$-dimensional signal-plus-noise snapshot vectors (samples) and $N$ statistically independent $n$-dimensional noise-only snapshot vectors, can one reliably infer the presence of a signal? This problem arises in the context of applications as diverse as radar, sonar, wireless communications, bioinformatics, and machine learning and is the critical first step in the subsequent signal parameter estimation phase. The signal detection problem can be naturally posed in terms of the sample generalized eigenvalues. The sample generalized eigenvalues correspond to the eigenvalues of the matrix formed by “whitening” the signal-plus-noise sample covariance matrix with the noise-only sample covariance matrix. In this paper, we prove a fundamental asymptotic limit of sample generalized eigenvalue-based detection of signals in arbitrarily colored noise when there are relatively few signal bearing and noise-only samples. Specifically, we show why when the (eigen) signal-to-noise ratio (SNR) is below a critical value, that is a simple function of $n$, $m$, and $N$, then reliable signal detection, in an asymptotic sense, is not possible. If, however, the eigen-SNR is above this critical value then a simple, new random matrix theory-based algorithm, which we present here, will reliably detect the signal even at SNRs close to the critical value. Numerical simulations highlight the accuracy of our analytical prediction, permit us to extend our heuristic definition of the effective number of identifiable signals in colored noise and display the dramatic improvement in performance relative to the classical estimator by Zhao et al. We discuss implications of our result for the detection of weak and/or closely spaced signals in sensor array processing, abrupt change detection in sensor networks, and clustering methodologies in machine learning.

Signal-to-Noise Ratios Required for Short-Term Narrowband Detection of Gaussian Processes

Evaluation of Bistable Systems for Binary Signal Detection in Symmetric Non-Gaussian Noise

Asymptotically Optimal One-Bit Quantizer Design for Weak-Signal Detection in Generalized Gaussian Noise and Lossy Binary Communication Channel.

Sequential joint signal detection and signal-to-noise ratio estimation

On Minimax Robust Detection of Stationary Gaussian Signals in White Gaussian Noise

Extension of the Geometric Mean Constant False Alarm Rate Detector to Multiple Pulses

Fundamental Limit of Sample Generalized Eigenvalue Based Detection of Signals in Noise Using Relatively Few Signal-Bearing and Noise-Only Samples

Data analysis strategies for the detection of gravitational waves in non-Gaussian noise

Effect of Time Delay on Binary Signal Detection Via a Bistable System

On Minimax Detection of Gaussian Stochastic Sequences and Gaussian Stationary Signals

Gravitational-Wave Data Analysis. Formalism and Sample Applications: The Gaussian Case

Adaptive Double Subspace Signal Detection in Gaussian Background—Part I: Homogeneous Environments

A New Noncoherent Gaussian Signaling Scheme for Low Probability of Detection Communications

Adaptive Double Subspace Signal Detection in Gaussian Background—Part II: Partially Homogeneous Environments

Detection of a rank-one signal with limited training data

On the (In)Efficiency of the Cross-Correlation Statistic for Gravitational Wave Stochastic Background Signals with Non-Gaussian Noise and Heterogeneous Detector Sensitivities

Adaptive Detector Performance Given Clutter Covariance Matrix in Subspace Orthogonal to Signal

Performance analysis of spectrum sensing schemes based on energy detector in generalized Gaussian noise

Near-optimal Detection with Constant False Alarm Ratio in Varying Impulsive Interference

Gaussian Signalling for Covert Communications

Calibrating signal-to-noise ratio detection thresholds using gravitational-wave catalogs