Abstract:Estimating a sequence of dynamic undirected graphical models, in which adjacent graphs share similar structures, is of paramount importance in various social, financial, biological, and engineering systems, since the evolution of such networks can be utilized for example to spot trends, detect anomalies, predict vulnerability, and evaluate the impact of interventions. Existing methods for learning dynamic graphical models require the tuning parameters that control the graph sparsity and the temporal smoothness to be selected via brute-force grid search. Furthermore, these methods are computationally burdensome with time complexity O(NP3) for P variables and N time points. As a remedy, we propose a low-complexity tuning-free Bayesian approach, named BASS. Specifically, we impose temporally dependent spike and slab priors on the graphs such that they are sparse and varying smoothly across time. An efficient variational inference algorithm based on natural gradients is then derived to learn the graph structures from the data in an automatic manner. Owing to the pseudo-likelihood and the mean-field approximation, the time complexity of BASS is only O(NP2). To cope with the local maxima problem of variational inference, we resort to simulated annealing and propose a method based on bootstrapping of the observations to generate the annealing noise. We provide numerical evidence that BASS outperforms existing methods on synthetic data in terms of structure estimation, while being more efficient especially when the dimension P becomes high. We further apply the approach to the stock return data of 78 banks from 2005 to 2013 and find that the number of edges in the financial network as a function of time contains three peaks, in coincidence with the 2008 global financial crisis and the two subsequent European debt crisis. On the other hand, by identifying the frequency-domain resemblance to the time-varying graphical models, we show that BASS can be extended to learning frequency-varying inverse spectral density matrices, and further yields graphical models for multivariate stationary time series. As an illustration, we analyze scalp EEG signals of patients at the early stages of Alzheimer's disease (AD) and show that the brain networks extracted by BASS can better distinguish between the patients and the healthy controls.

Apprentissage d’un modele graphique non orienté hybride parcimonieux par utilisation du gradient proximal stochastique

Sparse pairwise Markov model learning for anomaly detection in heterogeneous data

STANLEY: Stochastic Gradient Anisotropic Langevin Dynamics for Learning Energy-Based Models

An adaptive Hessian approximated stochastic gradient MCMC method

Anomaly Detection and Localisation using Mixed Graphical Models

Gradient Estimation Using Stochastic Computation Graphs

Stochastic Proximal Gradient Algorithm with Minibatches. Application to Large Scale Learning Models

Efficient Learning of Discrete Graphical Models

Efficient Variational Bayes Learning of Graphical Models With Smooth Structural Changes

Probability Based Independence Sampler for Bayesian Quantitative Learning in Graphical Log-Linear Marginal Models

Learning Gaussian Graphical Models via Multiplicative Weights

Scalable Stochastic Gradient Riemannian Langevin Dynamics in Non-Diagonal Metrics

Convergent stochastic Expectation Maximization algorithm with efficient sampling in high dimension. Application to deformable template model estimation

A New Reliable & Parsimonious Learning Strategy Comprising Two Layers of Gaussian Processes, to Address Inhomogeneous Empirical Correlation Structures

A Pseudo-Gradient Approach for Model-free Markov Chain Optimization

Learning in graphical models

Stochastic Gradient Descent as Approximate Bayesian Inference

Inversion Bayesienne Et Algorithmes Stochastiques De Type Recuit. Application a La Restauration D'images Et a La Reconstruction 3-D a Partir D'un Nombre Très Limite De Projections

Active Learning Algorithms for Graphical Model Selection

Approximate Bayesian estimation in large coloured graphical Gaussian models