Abstract:The detection problem in statistical signal processing can be succinctly formulated: given $m$ (possibly) signal bearing, $n$-dimensional signal-plus-noise snapshot vectors (samples) and $N$ statistically independent $n$-dimensional noise-only snapshot vectors, can one reliably infer the presence of a signal? This problem arises in the context of applications as diverse as radar, sonar, wireless communications, bioinformatics, and machine learning and is the critical first step in the subsequent signal parameter estimation phase. The signal detection problem can be naturally posed in terms of the sample generalized eigenvalues. The sample generalized eigenvalues correspond to the eigenvalues of the matrix formed by “whitening” the signal-plus-noise sample covariance matrix with the noise-only sample covariance matrix. In this paper, we prove a fundamental asymptotic limit of sample generalized eigenvalue-based detection of signals in arbitrarily colored noise when there are relatively few signal bearing and noise-only samples. Specifically, we show why when the (eigen) signal-to-noise ratio (SNR) is below a critical value, that is a simple function of $n$, $m$, and $N$, then reliable signal detection, in an asymptotic sense, is not possible. If, however, the eigen-SNR is above this critical value then a simple, new random matrix theory-based algorithm, which we present here, will reliably detect the signal even at SNRs close to the critical value. Numerical simulations highlight the accuracy of our analytical prediction, permit us to extend our heuristic definition of the effective number of identifiable signals in colored noise and display the dramatic improvement in performance relative to the classical estimator by Zhao et al. We discuss implications of our result for the detection of weak and/or closely spaced signals in sensor array processing, abrupt change detection in sensor networks, and clustering methodologies in machine learning.

RECOGNITION OF SIGNALS FROM COLORED NOISE BACKGROUND IN GENERALIZED S-TRANSFORMATION DOMAIN

Recognition of Signals from Colored Noise Background in Generalized S-Transform Domain

Statistical denoising of signals in the S-transform domain

The Mean Power Spectrum Distribution of Colored Noise via Physical Wavelet Transform

A Noise Reduction Method for Non-Stationary Noise Model of SPECT Sinogram Based on Kalman Filter

Combined Transformation of Ordering SPECT Sinograms for Signal Extraction from Measurements with Poisson Noise.

Detection of a Signal in Colored Noise: A Random Matrix Theory Based Analysis

Local Time-frequency Analysis of Images Based on Generalized S-transform

Dim Target Detection Based on Image Features Analysis in Generalized S-transform Domain

A Sample-Deficient Analysis of the Leading Generalized Eigenvalue for the Detection of Signals in Colored Gaussian Noise

Detection of Signals in Colored Noise: Leading Eigenvalue Test for Non-central $F$-matrices

Estimation of the Number of Signals in the Presence of Color Noise

Recognition of Communication Signals in Noise with Alpha Stable Distribution

Detection of Signals in Colored Noise: Roy's Largest Root Test for Non-central $F$-matrices

Signal Analysis via the Stochastic Geometry of Spectrogram Level Sets

Fundamental Limit of Sample Generalized Eigenvalue Based Detection of Signals in Noise Using Relatively Few Signal-Bearing and Noise-Only Samples

Detection of False Data Injection Attacks in Smart Grid under Colored Gaussian Noise

Aperiodic stochastic resonance in neural information processing with Gaussian colored noise

Nonlinear Gaussian Filter with Multi-Step Colored Noise

Coloured Noise from Stochastic Inflows in Reaction-Diffusion Systems

Background Filter: A method for removing signal contamination during significance estimation of a GstLAL anaysis