Abstract:Deep neural networks have achieved breakthrough improvement in various application fields. Nevertheless, they usually suffer from a time-consuming training process because of the complicated structures of neural networks with a huge number of parameters. As an alternative, a fast and efficient discriminative broad learning system (BLS) is proposed, which takes the advantages of flat structure and incremental learning. The BLS has achieved outstanding performance in classification and regression problems. However, the previous studies ignored the reason why the BLS can generalize well. In this article, we focus on the interpretation from the viewpoint of the frequency domain. We discover the existence of the frequency principle in BLS, i.e., the BLS preferentially captures low-frequency components quickly and then fits the high frequencies during the incremental process of adding feature nodes and enhancement nodes. The frequency principle may be of great inspiration for expanding the application of BLS.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to explain why the Broad Learning System (BLS) can generalize well. Specifically, although BLS has achieved excellent performance in classification and regression problems, what previous studies have overlooked is the principle by which BLS can generalize well. Therefore, from the perspective of the frequency domain, this paper explores the working mechanism of BLS, especially the phenomenon of how it preferentially captures low - frequency components and then fits high - frequency components when adding feature nodes and enhancement nodes in the incremental process. ### Specific Problem Description 1. **Time Consumption in the Training Process**: Due to its complex structure and a large number of parameters, deep neural networks often take a long time in the training process. 2. **Theoretical Explanation of Generalization Ability**: Although BLS has demonstrated excellent generalization ability, there is a lack of theoretical explanation for its generalization mechanism. 3. **Stability of Incremental Learning**: As the number of feature nodes and enhancement nodes increases, the prediction ability of BLS is not always stable; sometimes it gets better, and sometimes it gets worse, and the reason for this phenomenon is still unclear. ### Research Motivation Researchers hope to reveal the principle behind the generalization ability of BLS by explaining its working mechanism from the perspective of the frequency domain. This not only helps to understand the reasons for the success of BLS but also can provide theoretical support for the improvement of other similar systems. ### Main Contributions The main contributions of this paper are as follows: - Provide an explanation of BLS from the perspective of the frequency domain and find that BLS preferentially captures low - frequency components and then gradually fits high - frequency components in the incremental process. - Reveal the instability that may occur when BLS fits high - frequency components, especially when the amplitude of high - frequency peaks is low. - Propose the potential inspiration of the frequency principle for expanding the application of BLS, such as its application in scientific computing problems. Through these studies, the author hopes to provide a theoretical basis for the further development and application of BLS and provide a new perspective for the research of other neural networks.

Frequency Principle in Broad Learning System

Extreme Fuzzy Broad Learning System: Algorithm, Frequency Principle, and Applications in Classification and Regression

Frequency Principle in deep learning: an overview

Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks

Broad Learning System: An Effective and Efficient Incremental Learning System Without the Need for Deep Architecture

Overview frequency principle/spectral bias in deep learning

Analysis and Variants of Broad Learning System

Theory of the Frequency Principle for General Deep Neural Networks

Training Behavior of Deep Neural Network in Frequency Domain.

Deep Frequency Principle Towards Understanding Why Deeper Learning Is Faster

Properties and Potential Applications of Random Functional-Linked Types of Neural Networks

Recurrent Broad Learning Systems for Time Series Prediction

TKS-BLS: Temporal Kernel Stationary Broad Learning System for Enhanced Modeling, Anomaly Detection, and Incremental Learning with Application to Ironmaking Processes

Sparse Bayesian Approach to Fast Learning Network for Multiclassification.

Incremental Bayesian broad learning system and its industrial application

Stacked Broad Learning System: From Incremental Flatted Structure to Deep Model

Broad learning system: Feature extraction based on K-means clustering algorithm

Frequency Principle in Deep Learning with General Loss Functions and Its Potential Application.

Broad Learning With Reinforcement Learning Signal Feedback: Theory and Applications

Frequency Principle in Deep Learning Beyond Gradient-descent-based Training

SPRBF-ABLS: a novel attention-based broad learning systems with sparse polynomial-based radial basis function neural networks