Abstract:Learning hidden topics from data streams has become absolutely necessary but posed challenging problems such as concept drift as well as short and noisy data. Using prior knowledge to enrich a topic model is one of potential solutions to cope with these challenges. Prior knowledge that is derived from human knowledge (e.g. Wordnet) or a pre-trained model (e.g. Word2vec) is very valuable and useful to help topic models work better. However, in a streaming environment where data arrives continually and infinitely, existing studies are limited to exploiting these resources effectively. Especially, a knowledge graph, that contains meaningful word relations, is ignored. In this paper, to aim at exploiting a knowledge graph effectively, we propose a novel graph convolutional topic model (GCTM) which integrates graph convolutional networks (GCN) into a topic model and a learning method which learns the networks and the topic model simultaneously for data streams. In each minibatch, our method not only can exploit an external knowledge graph but also can balance the external and old knowledge to perform well on new data. We conduct extensive experiments to evaluate our method with both a human knowledge graph (Wordnet) and a graph built from pre-trained word embeddings (Word2vec). The experimental results show that our method achieves significantly better performances than state-of-the-art baselines in terms of probabilistic predictive measure and topic coherence. In particular, our method can work well when dealing with short texts as well as concept drift. The implementation of GCTM is available at \url{<a class="link-external link-https" href="https://github.com/bachtranxuan/GCTM.git" rel="external noopener nofollow">this https URL</a>}.

Dynamic transformation of prior knowledge into Bayesian models for data streams

Deep Learning for Knowledge-Driven Ontology Stream Prediction.

Lifelong Bayesian Learning Machines for Streaming Industrial Big Data

Scalable Bayesian modeling, monitoring and analysis of dynamic network flow data

Knowledge Graph Embeddings for Dealing with Concept Drift in Machine Learning

A new online learning algorithm for streaming data and decision support with a Bayesian approach

Telescopic broad Bayesian learning for big data stream

T-SaS: Toward Shift-aware Dynamic Adaptation for Streaming Data

A Graph Convolutional Topic Model for Short and Noisy Text Streams

Transfer Entropy Bottleneck: Learning Sequence to Sequence Information Transfer

Toward a general, scaleable framework for Bayesian teaching with applications to topic models

Discovering an Evolutionary Classifier over a High-speed Nonstatic Stream

A Framework for On-Demand Classification of Evolving Data Streams

An ensemble-based online learning algorithm for streaming data

packetLSTM: Dynamic LSTM Framework for Streaming Data with Varying Feature Space

Transfer estimation of evolving class priors in data stream classification

A Unified Replay-based Continuous Learning Framework for Spatio-Temporal Prediction on Streaming Data

Discrete Key-Value Bottleneck

A Probabilistic Framework for Adapting to Changing and Recurring Concepts in Data Streams

Time Varying Dynamic Bayesian Network for Nonstationary Events Modeling and Online Inference

Streaming Bayes GFlowNets