Abstract:This tutorial addresses the advances in deep Bayesian mining and learning for natural language with ubiquitous applications ranging from speech recognition to document summarization, text classification, text segmentation, information extraction, image caption generation, sentence generation, dialogue control, sentiment classification, recommendation system, question answering and machine translation, to name a few. Traditionally, "deep learning" is taken to be a learning process where the inference or optimization is based on the real-valued deterministic model. The "semantic structure" in words, sentences, entities, actions and documents drawn from a large vocabulary may not be well expressed or correctly optimized in mathematical logic or computer programs. The "distribution function" in discrete or continuous latent variable model for natural language may not be properly decomposed or estimated. This tutorial addresses the fundamentals of statistical models and neural networks, and focus on a series of advanced Bayesian models and deep models including hierarchical Dirichlet process, Chinese restaurant process, hierarchical Pitman-Yor process, Indian buffet process, recurrent neural network, long short-term memory, sequence-to-sequence model, variational auto-encoder, generative adversarial network, attention mechanism, memory-augmented neural network, skip neural network, stochastic neural network, predictive state neural network, policy neural network. We present how these models are connected and why they work for a variety of applications on symbolic and complex patterns in natural language. The variational inference and sampling method are formulated to tackle the optimization for complicated models. The word and sentence embeddings, clustering and co-clustering are merged with linguistic and semantic constraints. A series of case studies are presented to tackle different issues in deep Bayesian mining, learning and understanding. At last, we will point out a number of directions and outlooks for future studies.

Application of Deep Belief Networks for natural language understanding

Deep Adversarial Learning for NLP.

Auditory Scene Classification with Deep Belief Network.

A Study on Different Deep Learning Algorithms Used in Deep Neural Nets: MLP SOM and DBN

Deep learning systems as complex networks

Deep belief network based CRF for spoken language understanding

Improved Bottleneck Feature Using Hierarchical Deep Belief Networks for Keyword Spotting in Continues Speech

Improved Classification Based on Deep Belief Networks

Image-based Natural Language Understanding Using 2D Convolutional Neural Networks

Application of Deep Belief Networks for opcode based malware detection

Deep Learning and Its Applications to Natural Language Processing

Deep Bayesian Mining, Learning and Understanding

Classification of Drug and Non-Drug Using Deep Belief Network Based on Data Preprocess

Recurrent Dirichlet Belief Networks for Interpretable Dynamic Relational Data Modelling

Learning Document Semantic Representation with Hybrid Deep Belief Network.

An Adaptive Deep Belief Network With Sparse Restricted Boltzmann Machines

How to Train Your Deep Neural Network with Dictionary Learning

Dissecting neural computations of the human auditory pathway using deep neural networks for speech

Neural language models for network configuration: Opportunities and reality check

Speech Recognition of Oral English Teaching Based on Deep Belief Network

General-to-Specialized Analysis Based on Deep Belief Network.