Random Signals and LTI-Systems

This jupyter notebook is part of a collection of notebooks on various topics of Digital Signal Processing. Please direct questions and suggestions to Sascha.Spors@uni-rostock.de.

Power Spectral Densitity

For a wide-sense stationary (WSS) real-valued random process $x[k]$, the power spectral density (PSD) ${\Phi }_{xx}({\mathrm{e}}^{\mathrm{j}\Omega })$ is given as the discrete-time Fourier transformation (DTFT) of the auto-correlation function (ACF) ${\phi }_{xx}[\kappa ]$

$$\begin{array}{c}{\Phi }_{xx}({\mathrm{e}}^{\mathrm{j}\Omega })=\sum _{\kappa =-\infty }^{\infty }{\phi }_{xx}[\kappa ]{\mathrm{e}}^{-\mathrm{j}\Omega \kappa }\end{array}$$

Under the assumption of a real-valued LTI system with impulse response $h[k]\in \mathbb{R}$, the PSD ${\Phi }_{yy}({\mathrm{e}}^{\mathrm{j}\Omega })$ of the output signal of an LTI system $y[k]=\mathcal{H}\{x[k]\}$ is derived by taking the DTFT of the ACF of the output signal ${\phi }_{yy}[\kappa ]$

$$\begin{array}{rl}{\Phi }_{yy}({\mathrm{e}}^{\mathrm{j}\Omega })& =\sum _{\kappa =-\infty }^{\infty }\underset{{\phi }_{hh}[\kappa ]}{\underbrace{h[\kappa ]\ast h[-\kappa ]}}\ast {\phi }_{xx}[\kappa ]{\mathrm{e}}^{-\mathrm{j}\Omega \kappa }\\ & =H({\mathrm{e}}^{\mathrm{j}\Omega })\cdot H({\mathrm{e}}^{-\mathrm{j}\Omega })\cdot {\Phi }_{xx}({\mathrm{e}}^{\mathrm{j}\Omega })=|H({\mathrm{e}}^{\mathrm{j}\Omega }){|}^2\cdot {\Phi }_{xx}({\mathrm{e}}^{\mathrm{j}\Omega })\end{array}$$

The PSD of the output signal ${\Phi }_{yy}({\mathrm{e}}^{\mathrm{j}\Omega })$ of an LTI system is given by the PSD of the input signal ${\Phi }_{xx}({\mathrm{e}}^{\mathrm{j}\Omega })$ multiplied with the squared magnitude $|H({\mathrm{e}}^{\mathrm{j}\Omega }){|}^2$ of the transfer function of the system.

Example - Pink Noise

It can be concluded from above findings, that filtering can be applied to a white noise random signal $x[k]$ with ${\Phi }_{yy}({\mathrm{e}}^{\mathrm{j}\Omega })=N_0$ in order to create a random signal $y[k]=x[k]\ast h[k]$ with a desired PSD

$$\begin{array}{c}{\Phi }_{yy}({\mathrm{e}}^{\mathrm{j}\Omega })=N_0\cdot |H({\mathrm{e}}^{\mathrm{j}\Omega }){|}^2\end{array}$$

where $N_0$ denotes the power per frequency of the white noise. Such a random signal is commonly termed as colored noise. Different application specific types of colored noise exist. One of these is pink noise whose PSD is inversely proportional to the frequency. The approximation of a pink noise signal by filtering is illustrated by the following example. The PSDs ${\Phi }_{xx}({\mathrm{e}}^{\mathrm{j}\Omega })$ and ${\Phi }_{yy}({\mathrm{e}}^{\mathrm{j}\Omega })$ are estimated from $x[k]$ and $y[k]$ using the Welch technique.

Let's listen to white and pink noise

White noise

/uniform_white_noise.wav

Pink noise

/uniform_pink_noise.wav

Cross-Power Spectral Densities

The cross-power spectral densities ${\Phi }_{yx}({\mathrm{e}}^{\mathrm{j}\Omega })$ and ${\Phi }_{xy}({\mathrm{e}}^{\mathrm{j}\Omega })$ between the in- and output of an LTI system are given by taking the DTFT of the cross-correlation functions (CCF) ${\phi }_{yx}[\kappa ]$ and ${\phi }_{xy}[\kappa ]$. Hence,

$$\begin{array}{c}{\Phi }_{yx}({\mathrm{e}}^{\mathrm{j}\Omega })=\sum _{\kappa =-\infty }^{\infty }h[\kappa ]\ast {\phi }_{xx}[\kappa ]{\mathrm{e}}^{-\mathrm{j}\Omega \kappa }={\Phi }_{xx}({\mathrm{e}}^{\mathrm{j}\Omega })\cdot H({\mathrm{e}}^{\mathrm{j}\Omega })\end{array}$$

and

$$\begin{array}{c}{\Phi }_{xy}({\mathrm{e}}^{\mathrm{j}\Omega })=\sum _{\kappa =-\infty }^{\infty }h[-\kappa ]\ast {\phi }_{xx}[\kappa ]{\mathrm{e}}^{-\mathrm{j}\Omega \kappa }={\Phi }_{xx}({\mathrm{e}}^{\mathrm{j}\Omega })\cdot H({\mathrm{e}}^{-\mathrm{j}\Omega })\end{array}$$

System Identification by Spectral Division

Using the result above for the cross-power spectral density ${\Phi }_{yx}({\mathrm{e}}^{\mathrm{j}\Omega })$ between out- and input, and the relation of the CCF of finite-length signals to the convolution yields

$$\begin{array}{c}H({\mathrm{e}}^{\mathrm{j}\Omega })=\frac{{\Phi }_{yx}({\mathrm{e}}^{\mathrm{j}\Omega })}{{\Phi }_{xx}({\mathrm{e}}^{\mathrm{j}\Omega })}=\frac{\frac{1}{K}Y({\mathrm{e}}^{\mathrm{j}\Omega })\cdot X({\mathrm{e}}^{-\mathrm{j}\Omega })}{\frac{1}{K}X({\mathrm{e}}^{\mathrm{j}\Omega })\cdot X({\mathrm{e}}^{-\mathrm{j}\Omega })}=\frac{Y({\mathrm{e}}^{\mathrm{j}\Omega })}{X({\mathrm{e}}^{\mathrm{j}\Omega })}\end{array}$$

holding for ${\Phi }_{xx}({\mathrm{e}}^{\mathrm{j}\Omega })\ne 0$ and $X({\mathrm{e}}^{\mathrm{j}\Omega })\ne 0$. Hence, the transfer function $H({\mathrm{e}}^{\mathrm{j}\Omega })$ of an unknown system can be derived by dividing the spectrum of the output signal $Y({\mathrm{e}}^{\mathrm{j}\Omega })$ through the spectrum of the input signal $X({\mathrm{e}}^{\mathrm{j}\Omega })$. This is equal to the definition of the transfer function. However, care has to be taken that the spectrum of the input signal does not contain zeros.

Above relation can be realized by the discrete Fourier transformation (DFT) by taking into account that a multiplication of two spectra $X[\mu ]\cdot Y[\mu ]$ results in the cyclic/periodic convolution $x[k]⊛y[k]$. Since we aim at a linear convolution, zero-padding of the in- and output signal has to be applied.

Example

We consider the estimation of the impulse response $h[k]={\mathcal{F}}_{\ast }^{-1}\{H({\mathrm{e}}^{\mathrm{j}\Omega })\}$ of an unknown system using the spectral division method. Normal distributed white noise with variance ${\sigma }_n^2=1$ is used as wide-sense ergodic input signal $x[k]$. In order to show the effect of sensor noise, normally distributed white noise $n[k]$ with the variance ${\sigma }_n^2=0.01$ is added to the output signal $y[k]=x[k]\ast h[k]+n[k]$.

Exercise

• Change the length N of the input signal. What happens?
• Change the variance ${\sigma }_n^2$ of the additive noise. What happens?

Solution: Increasing the length N of the input signal lowers the uncertainty in estimating the impulse response. The higher the variance of the additive white noise, the higher the uncertainties in the estimated impulse response.

Copyright

This notebook is provided as Open Educational Resource. Feel free to use the notebook for your own purposes. The text is licensed under Creative Commons Attribution 4.0, the code of the IPython examples under the MIT license. Please attribute the work as follows: Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples.