Proportionate adaptive filtering for block-sparse system identification

Jianming Liu,Steven L. Grant

DOI: https://doi.org/10.1109/TASLP.2015.2499602

2016-04-01

Abstract:In this paper, a new family of proportionate normalized least mean square (PNLMS) adaptive algorithms that improve the performance of identifying block-sparse systems is proposed. The main proposed algorithm, called block-sparse PNLMS (BS-PNLMS), is based on the optimization of a mixed

What problem does this paper attempt to address?

Proportionate Total Adaptive Filtering Algorithms for Sparse System Identification

Jingen Ni,Yiwei Xing,Zhanyu Zhu,Jie Chen

DOI: https://doi.org/10.1109/tsmc.2023.3330125

2024-01-01

Abstract:In the application of system identification, not only the output but also the input of the system may be corrupted by noise, which is often characterized by the errors-in-variables (EIV) model. To identify such systems, a gradient-descent total least-squares (GD-TLS) and a maximum total correntropy (MTC) algorithms were proposed. In some scenarios, the weight vector of the unknown system may be sparse, e.g., the echo path in acoustic echo cancelation (AEC). Employing TLS or MTC to estimate such systems may result in slow convergence rate, since they assign the same gain to the update of each weight and therefore cannot make use of the sparsity feature of the system to accelerate convergence. To address the above problem, this article proposes a uniform optimization model for deriving proportionate total adaptive filtering algorithms, and then two proportionate total adaptive filtering algorithms are developed, namely, the proportionate total normalized least mean square (PTNLMS) algorithm for Gaussian noise disturbance and the proportionate MTC (PMTC) algorithm for impulsive noise interference, which are both derived by utilizing the method of Lagrange multipliers. Moreover, this article also makes a steady-state performance analysis of the two proposed algorithms. Simulations are performed to demonstrate the superior performance of the two proposed algorithms and to test the accuracy of the theory on the steady-state performance analysis.
An Improved Set-Membership Proportionate Adaptive Algorithm for a Block-Sparse System.

Zhan Jin,Yingsong Li,Jianming Liu

DOI: https://doi.org/10.3390/sym10030075

2018-01-01

Abstract:In this paper, an improved set-membership proportionate normalized least mean square (SM-PNLMS) algorithm is proposed for block-sparse systems. The proposed algorithm, which is named the block-sparse SM-PNLMS (BS-SMPNLMS), is implemented by inserting a penalty of a mixed l(2,1) norm of weight-taps into the cost function of the SM-PNLMS. Furthermore, an improved BS-SMPNLMS algorithm (the (BS-SMIPNLMS algorithm) is also derived and analyzed. The proposed algorithms are well investigated in the framework of network echo cancellation. The results of simulations indicate that the devised BS-SMPNLMS and BS-SMIPNLMS algorithms converge faster and have smaller estimation errors compared with related algorithms.
Block-Sparsity-Induced Adaptive Filter for Multi-Clustering System Identification

Shuyang Jiang,Yuantao Gu

DOI: https://doi.org/10.1109/tsp.2015.2453133

IF: 4.875

2015-01-01

IEEE Transactions on Signal Processing

Abstract:In order to improve the performance of least mean square (LMS)-based adaptive filtering for identifying block-sparse systems, a new adaptive algorithm called block-sparse LMS (BS-LMS) is proposed in this paper. The basis of the proposed algorithm is to insert a penalty of block-sparsity, which is a mixed l2, 0 norm of adaptive tap-weights with equal group partition sizes, into the cost function of traditional LMS algorithm. To describe a block-sparse system response, we first propose a Markov-Gaussian model, which can generate a kind of system responses of arbitrary average sparsity and arbitrary average block length using given parameters. Then we present theoretical expressions of the steady-state misadjustment and transient convergence behavior of BS-LMS with an appropriate group partition size for white Gaussian input data. Based on the above results, we theoretically demonstrate that BS-LMS has much better convergence behavior than l0-LMS with the same small level of misadjustment. Finally, numerical experiments verify that all of the theoretical analysis agrees well with simulation results in a large range of parameters.
An Improved Μ-Law Proportionate NLMS Algorithm for Estimating Block-Sparse Systems

Zhan Jin,Xiuling Ding,Zhengxiong Jiang,Yingsong Li

DOI: https://doi.org/10.1109/iceict.2019.8846290

2019-01-01

Abstract:An improved μ-law proportionate normalized least mean square (MPNLMS) algorithm is presented and analyzed for giving an estimation of block-sparse systems, which is also named as block-sparse MPNLMS (BS-MPNLMS). The proposed BS-MPNLMS algorithm introduces a hybrid $l_{2,1}$-norm into the MPNLMS’s cost function to create a penalty. The devised new BS-MPNLMS is derived in detail and is analyzed for estimating the network echo signals whose response has a typical block-sparse characteristic. Numerical simulation results show that the devised algorithm has better convergence and stability performance for handling the block-sparse systems compared with related algorithms.
Proportionate Adaptive Filtering Algorithms Based on Mixed Square/fourth Error Criterion with Unbiasedness Criterion for Sparse System Identification

Wentao Ma,Jiandong Duan,Jiuwen Cao,Yingsong Li,Badong Chen

DOI: https://doi.org/10.1002/acs.2928

IF: 3.369

2018-01-01

International Journal of Adaptive Control and Signal Processing

Abstract:SummaryTwo novel adaptive filtering algorithms based on the mixed square/fourth error criterion are proposed for solving sparse system identification problems. Motivated by the fact that the proportionate update scheme can enhance the tracking ability of the system, we develop a proportionate least mean square/fourth (PLMS/F) algorithm in this paper. Combining the proportionate update scheme and the LMS/F algorithm, the proposed PLMS/F algorithm shows superiority for non‐Gaussian noise environments. Moreover, to further improve the performance of the PLMS/F algorithm in the noisy input cases, a bias‐compensated PLMS/F algorithm is developed by incorporating an unbiased criterion to compensate the bias caused by input noises. Simulation results in the context of the sparse system identification framework demonstrate that the proposed PLMS/F and bias‐compensated PLMS/F algorithms can achieve excellent identification performance in terms of steady‐state misalignment and convergence speed under noisy input and non‐Gaussian output noise environments.
A New Block-Sparse Lms Algorithm With Adaptive And Nonuniform Group Sizes

Ye Wei,Yonggang Zhang,Chengcheng Wang,Jonathon A. Chambers

DOI: https://doi.org/10.1109/ICDSP.2018.8631820

2018-01-01

Abstract:This work presents a new block-sparse least mean square (BS-LMS) algorithm with adaptive and nonuniform group sizes. Motivated by the fact that large and zero coefficients in block-sparse systems assemble in clusters of variant sizes, we divide the adaptive filter tap-weights into groups of nonuniform size. A predetermined threshold value is utilized for real-time group partitioning Large coefficient groups consist of clusters of coefficients whose magnitudes are above the threshold value, while zero coefficient groups consist of clusters of coefficients whose magnitudes are below the threshold value. Then, coefficients in large coefficient groups are updated by following the standard LMS algorithm, while those in zero coefficient groups suffer from an attractive force towards zero. A practical choice guideline is provided for the threshold value to facilitate applications. The proposed algorithm is tested in the identification of single- and multiple-block-sparse systems. Simulation results show that the proposed scheme effectively locates and tracks the non-zero blocks in the unknown systems, outperforming existing block-sparsity-inspired LMS algorithms with uniform group sizes.
An improved proportionate normalized least mean square algorithm for sparse impulse response identification

Hao-xiang Wen,Xiao-han Lai,Long-dao Chen,Zhong-fa Cai

DOI: https://doi.org/10.1007/s12204-013-1460-8

2013-01-01

Journal of Shanghai Jiaotong University (Science)

Abstract:In this paper after analyzing the adaptation process of the proportionate normalized least mean square (PNLMS) algorithm, a statistical model is obtained to describe the convergence process of each adaptive filter coefficient. Inspired by this result, a modified PNLMS algorithm based on precise magnitude estimate is proposed. The simulation results indicate that in contrast to the traditional PNLMS algorithm, the proposed algorithm achieves faster convergence speed in the initial convergence state and lower misalignment in the stead stage with much less computational complexity.
Design of a Tap-Amplitude-Based Block Proportional Adaptive Filtering Algorithm

Sheng Zhang,Hongyang Chen,Ali H. Sayed

DOI: https://doi.org/10.1109/tsp.2023.3290659

IF: 4.875

2023-01-01

IEEE Transactions on Signal Processing

Abstract:Proportional-type algorithms have attracted much attention because of their fast convergence ability for sparse system identification. To overcome the drawbacks of existing block proportional methods stemming from inadequate block partitioning, this article develops tap-amplitude-based block partitioning methods. In the procedure, we present two block proportional normalized least-mean-square (PNLMS) algorithms named (i) ABx-PNLMS and (ii) ABy-PNLMS. In the first algorithm, the proportional gain depends on the rank-based tap-weight block on the x-axis. In comparison, the second algorithm determines the proportional gain by dynamically partitioning the y-axis in a non-uniform manner. Nevertheless, both algorithms have a common feature, namely, weights of similar magnitude are grouped together. By further incorporating an adaptive decorrelation mechanism into the ABx-PNLMS and ABy-PNLMS, two improved decorrelation proportional alogrithms are developed to speed up the convergence for colored inputs. Computer simulation results on system identification and acoustic echo cancellation demonstrate the effectiveness of the proposed approaches.

engineering, electrical & electronic
A Mixed $$(\mathbf 2 ,{\varvec{p}}$$(2,p-like)-Norm Penalized Least Mean Squares Algorithm for Block-Sparse System Identification

Ye Wei,Yonggang Zhang,Chengcheng Wang,Yahong Rosa Zheng

DOI: https://doi.org/10.1007/s00034-018-0769-9

2018-01-01

Abstract:This work presents a new mixed (2,p-like)-norm penalized least mean squares (LMS) algorithm for block-sparse system identifications where the nonzero coefficients in the impulse response vector of unknown systems are structured in a single cluster or multiple clusters. The new algorithm divides the tap-weight vector into groups of equal-sized sub-vectors and then introduces a mixed $l_{2,p\text {-like}}$-norm constraint on the filter tap-weight vector in addition to the original mean-square-error cost function. The parameter p in the $l_{2,p\text {-like}}$-norm constraint takes any value between zero and two, thus improving the identification performance of the block-sparse systems. The effect of the parameter p and the group size on the performance of the proposed algorithm is studied, and general guidelines for choosing these two parameters are provided to facilitate practical use. The advantage of the proposed scheme is that no comparison operations are required while algebraic operations are of the same order as the block-sparse LMS algorithm. Numerical simulations show that the proposed $(2,p\text {-like})$-norm penalized LMS algorithm outperforms the existing $l_{2,0}$- and $l_{2,1}$-norm-based block-sparsity-aware algorithms and single-norm penalized LMS strategies.
An Enhanced Proportionate NLMF Algorithm for Group-Sparse System Identification

Zhengxiong Jiang,Wanlu Shi,Xinqi Huang,Yingsong Li

DOI: https://doi.org/10.1016/j.aeue.2020.153178

IF: 3.169

2020-01-01

AEU - International Journal of Electronics and Communications

Abstract:A novel adaptive filtering algorithm is devised and derived for group-sparse system identification. To adequately make use of the group-sparsity in satellite communication and network echo channels, we integrate a mixed-norm constraint into the proportionate normalized least mean fourth (PNLMF) algorithm, which is referred as mixed-norm constrained PNLMF (MNC-PNLMF) algorithm. The MNC-PNLMF algorithm is derived and analyzed in detail. Serval experimental experiments are constructed to validate the effectiveness of the MNC-PNLMF. The experimental results demonstrate that the MNC-PNLMF outperforms the NLMF, PNLMF, zero-attraction NLMF (ZA-NLMF), and reweighted ZA-NLMF (RZA-NLMF) for group-sparse system identification.
Cluster-Sparse Proportionate NLMS Algorithm with the Hybrid Norm Constraint

Yingsong Li,Zhengxiong Jiang,Zhan Jin,Xiao Han,Jingwei Yin

DOI: https://doi.org/10.1109/access.2018.2867561

IF: 3.9

2018-01-01

IEEE Access

Abstract:In this paper, an enhanced proportionate normalized least mean square (PNLMS) algorithm with the hybrid l 2,0 -norm constraint is proposed for block-sparse signal processing. The proposed algorithm penalizes a mixed l 2,0 -norm on the PNLMS to fully exploit the sparsity and handle block-sparse signals, which is called the l 2,0 norm constrained PNLMS (L20-PNLMS). The L20-PNLMS is well derived and carefully analyzed. Various experiments have been constructed to verify the effectiveness of the devised L20-PNLMS. The experimental results demonstrate that the devised L20-PNLMS performs better than the previous PNLMS algorithms do in block sparse signal processing.
Proportionate NLMS with Unbiasedness Criterion for Sparse System Identification in the Presence of Input and Output Noises.

Wentao Ma,Dongqiao Zheng,Xiangqian Tong,Zhiyu Zhang,Badong Chen

DOI: https://doi.org/10.1109/tcsii.2017.2785306

2018-01-01

Abstract:This brief proposes a bias-compensated proportionate normalized least mean square (BCPNLMS) method for identifying sparse system when subjected to the noisy input. The proposed BCPNLMS algorithm, which combines the proportionate scheme and the unbiasedness criterion, is able to identify the system parameters with better steady-state accuracy and faster convergence speed than conventional NLMS, bias-compensated NLMS, and PNLMS algorithms. Robustness and high identification accuracy with noisy input can be achieved by introducing the bias-compensation term derived from the unbiasedness criterion. Simulation results on sparse system identification confirm the excellent performance of the proposed BCPNLMS in the presence of both input and output noises.
A Proportionate Recursive Least Squares Algorithm and Its Performance Analysis

Zhen Qin,Jun Tao,Yili Xia

DOI: https://doi.org/10.1109/tcsii.2020.3004466

2021-01-01

IEEE Transactions on Circuits & Systems II Express Briefs

Abstract:The proportionate updating (PU) mechanism has been widely adopted in least mean squares (LMS) adaptive filtering algorithms to exploit the system sparsity. In this brief, we propose a proportionate recursive least squares (PRLS) algorithm for the sparse system estimation, in which, an independent weight update is assigned to each tap according to the magnitude of that estimated filter coefficient. Its mean square performance is analyzed via the energy conservation principle in both the transient and steady-state stages. In this way, an explicit condition on the control parameter of the proportionate matrix of PRLS can be obtained to ensure a better steady-state performance than that of RLS. Simulation results in a system identification setting support the analysis.
Lawson-norm Constrained Generalized Lncosh Based Adaptive Algorithm for Sparse System Identification

Yingsong Li,Tao Liang,Xiangkun Zhang,Jingshan Jiang

DOI: https://doi.org/10.11999/JEIT210057

2022-01-01

Abstract:A generalized Lawson-lncosh adaptive filtering algorithm for sparse system identification is proposed. The proposed algorithm is derived by constructing a new cost function consisted of Lawson-norm of system coefficient vector and lncosh function of instantaneous error. And the Lawson-norm constraint introduces aparameter p which can realize the dynamic adjustment of sparsity. The proposed algorithm can improve the convergence speed and reduce the steady-state error for sqarse system identification, where the Lncosh function of the error has the property of combating impulsive noise. Then, the range of the step-size and effect of parameter p on the proposed algorithm are analyzed. Computer simulation results show that the proposed algorithm has superior performance compared with other existing algorithms with Gaussian and colored input signals and the sparsity constraint for the proposed algorithm is controllable.
Block-sparse sign algorithm and its performance analysis

Yifan Wu,Zhu Qing,Jingen Ni,Jie Chen

DOI: https://doi.org/10.1016/j.dsp.2022.103620

IF: 2.92

2022-01-01

Digital Signal Processing

Abstract:System identification is often encountered in applications such as echo cancellation, active noise control, and channel equalization. If the unknown system is sparse, using sparsity-induced methods can improve the convergence performance. Recently, the l2,0-norm constraint was used to derive a block-sparse LMS (BS-LMS) algorithm to accelerate convergence for identifying multi-clustering sparse systems. In some cases, output of the unknown system is contaminated by impulsive noise, and BS-LMS performs poorly or even diverges when identifying such systems. To address this problem, this paper constructs a loss function by combining the absolute error and the l2,0-norm of adaptive filter weight vector, and then uses the subgradient descent method to develop a block-sparse sign algorithm (BS-SA). Its mean and mean-square performance is also analyzed based on the Gaussian-Bernoulli impulsive noise model under some frequently used assumptions. Finally, simulations are performed to test the robustness of BS-SA against impulsive noise and to evaluate the accuracy of theoretical expressions derived for statistical performance.(C) 2022 Elsevier Inc. All rights reserved.
Proportionate Minimum Error Entropy Algorithm for Sparse System Identification

Zongze Wu,Siyuan Peng,Badong Chen,Haiquan Zhao,Jose C. Principe

DOI: https://doi.org/10.3390/e17095995

IF: 2.738

2015-01-01

Entropy

Abstract:Sparse system identification has received a great deal of attention due to its broad applicability. The proportionate normalized least mean square (PNLMS) algorithm, as a popular tool, achieves excellent performance for sparse system identification. In previous studies, most of the cost functions used in proportionate-type sparse adaptive algorithms are based on the mean square error (MSE) criterion, which is optimal only when the measurement noise is Gaussian. However, this condition does not hold in most real-world environments. In this work, we use the minimum error entropy (MEE) criterion, an alternative to the conventional MSE criterion, to develop the proportionate minimum error entropy (PMEE) algorithm for sparse system identification, which may achieve much better performance than the MSE based methods especially in heavy-tailed non-Gaussian situations. Moreover, we analyze the convergence of the proposed algorithm and derive a sufficient condition that ensures the mean square convergence. Simulation results confirm the excellent performance of the new algorithm.
A Data Selective Segment Proportionate Affine Projection Algorithm for Echo Cancellation

Jun Xu,Xiao-dong Gu,Tao Liu

DOI: https://doi.org/10.1109/WICOM.2011.6039950

2011-09-01

Abstract:The segment proportionate normalized least mean-square algorithm (SPNLMS) has been proposed with the objective of improving the adaptation convergence rate when modeling high-order sparse impulse response systems as compared to their conventional counterparts.. When the excitation signal is colored, especially the speech, the convergence performance of SPNLMS algorithm demonstrates slow convergence speed. The segment proportionate affine projection (SPAP) algorithm is an useful adaptive filter to improve the convergence speed of PNLMS-type filter by updating the weight vector based on several previous input vectors, Unfortunately, the SPAP algorithm obtains a faster convergence of the SPNLMS comes at the expense of an increase in the computational complexity linked to the amount of reuses employed. For this reason, the idea of proportionate adaptation combined with the framework of set-membership filtering in an attempt to reduce computational complexity of algorithm is presented. The proposed algorithm allows the reduction of the frequency of updates of the filter coefficients, where the filter coefficients are updated such that the output estimation error is upper bounded by a pre-determined threshold. Simulations show that its overall complexity is lower compared to fully updated SPAP and its performance is close to that of the SPAP algorithm.

Engineering,Mathematics,Computer Science
Fast Nlmf-Type Algorithms For Adaptive Sparse System Identifications

Guan Gui,Beiyi Liu,Li Xu,Wentao Ma

DOI: https://doi.org/10.1109/APSIPA.2015.7415414

2015-01-01

Abstract:Adaptive sparse system identification (ASIDE) techniques have been successfully applied in many applications, such as sparse channel estimation and radar target detection. Normalized least mean fourth (NLMF)-type algorithms are considered as one of the stable ASIDE techniques even at low signal-to-noise ratio (SNR). However, the convergence capability of sparse NLMF algorithms is severely decreased by initial mean square error (MSE) and input variance in the high SNR regimes. To improve the convergence speed of the sparse NLMF algorithms in all SNR regions, in this paper, we propose a kind of non-constraint fast sparse NLMF-type algorithms for applying in ASIDE. Unlike the conventional methods, the proposed algorithms provides an alternative way to get rid of the restriction of SNR-dependent initial MSE and input variance. The proposed fast sparse NLMF-type algorithms are validated via computer simulations.
A Family of Blocked Proportionate Hybrid Error Criterion Algorithms.

Yingsong Li,Zhengxiong Jiang,Yanyan Wang,Aleksey Cherednichenko

DOI: https://doi.org/10.1109/access.2019.2899695

IF: 3.9

2019-01-01

IEEE Access

Abstract:A family of blocked proportionate normalized least mean mixed-norm (BPNLMMN) algorithms using hybrid error criterion are devised and investigated for estimating block-sparse channels that are always considered in networks and satellite echo channels. The derived BPNLMMN algorithms are realized by considering two different errors in the least mean algorithm and incorporating hybrid-norm constraints into the constructed cost function to fully use the prior cluster-sparse information in the systems to be estimated. We carefully derive the BPNLMMN algorithms and provide a completed analysis of the BPNLMMN algorithms for cluster channel estimations. The obtained results from various simulations illustrate that the BPNLMMN algorithms provide superior performance under various inputs and perform better than existing least mean mixed-norm algorithms.
Biased-proportionate normalized least-mean-squares adaptation

Mahdi Khosravy,Ustijana Rechkoska Shikoska,Ruoyu Li,Mohammad Reza Asharif,Katsumi Yamashita

2012-01-01

Abstract:This manuscript presents an efficient modification to Proportionate Normalized Least Mean Squares (PNLMS) adaptation algorithm. In a sense, the resultant algorithm is similar to PNLMS wherein the adaptation step for each individual filter coefficient is proportional to its growth from initial value. However, here the proportionality is biased. Indeed, the individual step size of each filter coefficient of each iteration is assigned as superposition of two steps. One is a fixed step size and the other is a proportionate step size. The fixed step size is called here step bias. The proposed approach has been compared with Normalized Least Mean Squared (NLMS), PNLMS and Improved PNLMS (IPNLMS) adaptations approaches. Simulations approve its higher efficiency compared with the others in obtaining a lower error shelf, better convergence. Furthermore, the proposed adaptive algorithm has much faster initial convergence. © 2012 ISSN 1881-803X.

Proportionate adaptive filtering for block-sparse system identification

Proportionate Total Adaptive Filtering Algorithms for Sparse System Identification

An Improved Set-Membership Proportionate Adaptive Algorithm for a Block-Sparse System.

Block-Sparsity-Induced Adaptive Filter for Multi-Clustering System Identification

An Improved Μ-Law Proportionate NLMS Algorithm for Estimating Block-Sparse Systems

Proportionate Adaptive Filtering Algorithms Based on Mixed Square/fourth Error Criterion with Unbiasedness Criterion for Sparse System Identification

A New Block-Sparse Lms Algorithm With Adaptive And Nonuniform Group Sizes

An improved proportionate normalized least mean square algorithm for sparse impulse response identification

Design of a Tap-Amplitude-Based Block Proportional Adaptive Filtering Algorithm

A Mixed $$(\mathbf 2 ,{\varvec{p}}$$(2,p-like)-Norm Penalized Least Mean Squares Algorithm for Block-Sparse System Identification

An Enhanced Proportionate NLMF Algorithm for Group-Sparse System Identification

Cluster-Sparse Proportionate NLMS Algorithm with the Hybrid Norm Constraint

Proportionate NLMS with Unbiasedness Criterion for Sparse System Identification in the Presence of Input and Output Noises.

A Proportionate Recursive Least Squares Algorithm and Its Performance Analysis

Lawson-norm Constrained Generalized Lncosh Based Adaptive Algorithm for Sparse System Identification

Block-sparse sign algorithm and its performance analysis

Proportionate Minimum Error Entropy Algorithm for Sparse System Identification

A Data Selective Segment Proportionate Affine Projection Algorithm for Echo Cancellation

Fast Nlmf-Type Algorithms For Adaptive Sparse System Identifications

A Family of Blocked Proportionate Hybrid Error Criterion Algorithms.

Biased-proportionate normalized least-mean-squares adaptation