Abstract:The broad learning system (BLS) is designed based on the technology of compressed sensing and pseudo-inverse theory, and consists of feature nodes and enhancement nodes, has been proposed recently. Compared with the popular deep learning structures, such as deep neural networks, BLS has the ability of rapid incremental learning and can remodel the system without the usual tedious retraining process. However, given that BLS is still in its infancy, it still needs analysis, improvements, and verification. In this article, we first analyze the principle of fast incremental learning ability of BLS in depth. Second, in order to provide an in-depth analysis of the BLS structure, according to the novel structure design concept of deep neural networks, we present four brand-new BLS variant networks and their incremental realizations. Third, based on our analysis of the effect of feature nodes and enhancement nodes, a new BLS structure with a semantic feature extraction layer has been proposed, which is called SFEBLS. The experimental results show that SFEBLS and its variants can increase the accuracy rate on the NORB dataset 6.18%, Fashion-MNIST dataset by 3.15%, ORL data by 5.00%, street view house number dataset by 12.88%, and CIFAR-10 dataset by 18.42%, respectively, and the four brand-new BLS variant networks also obviously outperform the original BLS.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are the high computational cost, high hardware requirements, and complex parameter adjustment in the training process of existing deep - learning structures (such as deep neural networks). Specifically, the article focuses on a new learning model - the Broad Learning System (BLS), and is committed to analyzing, improving, and verifying the performance of BLS. The following are the specific objectives of this paper: 1. **In - depth analysis of fast incremental learning ability**: The paper first conducts an in - depth analysis of the fast incremental learning ability of BLS. Designed based on compressed sensing technology and pseudo - inverse theory, BLS has the ability of fast incremental learning and can remodel the system without a cumbersome retraining process. 2. **Propose four brand - new BLS variant networks and their incremental implementation**: In order to provide an in - depth analysis of the BLS structure and draw on the novel structural design concepts of deep neural networks, the paper proposes four brand - new BLS variant networks (CFBLS - pyramid, CEBLS - dropout, CFBLS - dense, CEBLS - dense) and their incremental implementation methods. 3. **Introduce a new BLS structure with a semantic feature extraction layer**: Based on the analysis of the effects of feature nodes and enhancement nodes, the paper proposes a new BLS structure with a semantic feature extraction layer (SFEBLS). This new structure significantly improves the classification accuracy on multiple datasets. Through these studies, the paper not only verifies the effectiveness of BLS but also provides new directions and ideas for the further development of BLS. The experimental results show that SFEBLS and its variants perform better than the original BLS on the network and achieve significant performance improvements on multiple benchmark datasets. ### Formula summary - **Basic formula of BLS**: \[ Y = DW \] \[ W = D^+Y \] where \(D^+\) is the pseudo - inverse matrix of \(D\). - **Limit formula of the pseudo - inverse matrix**: \[ D^+=\lim_{\lambda\rightarrow0}(\lambda I + D D^T)^{-1}D^T \] - **Representation of feature nodes in CFBLS - pyramid**: \[ Z_{ij}=\begin{cases} \varphi(XW_{eij}+\beta_{eij}) & \text{if } i = 1,\dots,N \text{ and } j = 1\\ \varphi(Z_{i(j - 1)}W_{eij}+\beta_{eij}) & \text{if } i = 1,\dots,N \text{ and } j = 2,\dots,i \end{cases} \] - **Representation of enhancement nodes in CEBLS - dropout**: \[ H_j=\zeta(Z_NW_{hj}+\beta_{hj}),\quad j = 1,\dots,M \] - **Incremental formula of the pseudo - inverse matrix**: \[ (D_M^{N' + 1})^+=\left[(D_M^{N'})^+ - Q B^T\right](B^T)^{-1} \] where \(Q=(D_M^{N'})^+[Z_{N' + 1}|H_{M'}]\), \(B^T=\begin{cases} C^+ & \text{if } C\neq0\\ (1 + Q^T Q)^{-1}Q^T(D_M^{N'})^+ & \text{if } C = 0 \end{cases}\), and \(C=[Z_{N' + 1}|H_{M'}]-D_M^{N'}Q\). Through these formulas and methods, the paper demonstrates the efficiency and accuracy of BLS and its variants in processing large - scale datasets.

Analysis and Variants of Broad Learning System

Broad Learning System: An Effective and Efficient Incremental Learning System Without the Need for Deep Architecture

Stacked Broad Learning System: From Incremental Flatted Structure to Deep Model

Graph-based Broad Learning System for Classification

Incremental Bayesian broad learning system and its industrial application

Sparse Bayesian Approach to Fast Learning Network for Multiclassification.

SPRBF-ABLS: a novel attention-based broad learning systems with sparse polynomial-based radial basis function neural networks

Deep Optimized Broad Learning System for Applications in Tabular Data Recognition

TKS-BLS: Temporal Kernel Stationary Broad Learning System for Enhanced Modeling, Anomaly Detection, and Incremental Learning with Application to Ironmaking Processes

Universal Approximation Capability of Broad Learning System and Its Structural Variations

Broad Learning With Reinforcement Learning Signal Feedback: Theory and Applications

Two-Dimensional Broad Learning System for Data Analytic

Semi-Supervised Broad Learning System Based on Manifold Regularization and Broad Network

Incremental Multi-Layer Broad Learning System with Stochastic Configuration Algorithm for Regression

Factorization of Broad Expansion for Broad Learning System

Tree Broad Learning System for Small Data Modeling

Broad Learning System for Semi-Supervised Learning.

Broad learning system with manifold regularized sparse features for semi-supervised classification

Broad learning system: Feature extraction based on K-means clustering algorithm

ConvBLS: An Effective and Efficient Incremental Convolutional Broad Learning System for Image Classification

Cryogenic solvents inhibit the binding of the Escherichia coli heat-stable enterotoxin to intestinal brush border membranes.