Abstract:A deep belief network (DBN) is effective to create a powerful generative model by using training data. However, it is difficult to fast determine its optimal structure given specific applications. In this paper, a growing DBN with transfer learning (TL-GDBN) is proposed to automatically decide its structure size, which can accelerate its learning process and improve model accuracy. First, a basic DBN structure with single hidden layer is initialized and then pretrained, and the learned weight parameters are frozen. Second, TL-GDBN uses TL to transfer the knowledge from the learned weight parameters to newly added neurons and hidden layers, which can achieve a growing structure until the stopping criterion for pretraining is satisfied. Third, the weight parameters derived from pretraining of TL-GDBN are further fine-tuned by using layer-by-layer partial least square regression from top to bottom, which can avoid many problems of traditional backpropagation algorithm-based fine-tuning. Moreover, the convergence analysis of the TL-GDBN is presented. Finally, TL-GDBN is tested on two benchmark data sets and a practical wastewater treatment system. The simulation results show that it has better modeling performance, faster learning speed, and more robust structure than existing models. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</italic> —Transfer learning (TL) aims to improve training effectiveness by transferring knowledge from a source domain to target domain. This paper presents a growing deep belief network (DBN) with TL to improve the training effectiveness and determine the optimal model size. Facing a complex process and real-world workflow, DBN tends to require long time for its successful training. The proposed growing DBN with TL (TL-GDBN) accelerates the learning process by instantaneously transferring the knowledge from a source domain to each new deeper or wider substructure. The experimental results show that the proposed TL-GDBN model has a great potential to deal with complex system, especially the systems with high nonlinearity. As a result, it can be readily applicable to some industrial nonlinear systems.

Guest Editorial: Special Issue on New Advances in Deep-Transfer Learning

Transfer Learning and Deep Domain Adaptation

A Review of Deep Transfer Learning and Recent Advancements

Adversarial Deep Transfer Learning in Fault Diagnosis: Progress, Challenges, and Future Prospects

DEEP LEARNING IN THE ERA OF BIG DATA: FOUNDATIONS, ADVANCES, APPLICATIONS, CHALLENGES, AND FUTURE DIRECTIONS

Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalization

A Survey on Deep Transfer Learning.

Advancing 3D Point Cloud Understanding through Deep Transfer Learning: A Comprehensive Survey

Deep Transfer Network: Unsupervised Domain Adaptation

Transfer Learning for Aided Target Recognition: Comparing Deep Learning to other Machine Learning Approaches

Special Issue:Deep Learning in Computer Graphics

Guest Editorial: Foreword to the Special Issue on Advanced Research and Applications of Deep Learning and Neural Network in Image Recognition

Rethinking Two Consensuses of the Transferability in Deep Learning

Deep Unsupervised Progressive Learning for Distant Domain Adaptation.

Deep Learning Techniques for Agronomy Applications

A Decade Survey of Transfer Learning (2010–2020)

TL-GDBN: Growing Deep Belief Network With Transfer Learning

Modern deep learning in bioinformatics

Transfer learning: a friendly introduction

Feature-based Distant Domain Transfer Learning.

Deep Transfer Learning Applications in Intrusion Detection Systems: A Comprehensive Review