Abstract:The recent development of compressed sensing has led to spectacular advances in the understanding of sparse linear estimation problems as well as in algorithms to solve them. It has also triggered a new wave of developments in the related fields of generalized linear and bilinear inference problems, that have very diverse applications in signal processing and are furthermore a building block of deep neural networks. These problems have in common that they combine a linear mixing step and a nonlinear, probabilistic sensing step, producing indirect measurements of a signal of interest. Such a setting arises in problems as different as medical or astronomical imaging, clustering, matrix completion or blind source separation. The aim of this thesis is to propose efficient algorithms for this class of problems and to perform their theoretical analysis. To this end, it uses belief propagation, thanks to which high-dimensional distributions can be sampled efficiently, thus making a Bayesian approach to inference tractable. The resulting algorithms undergo phase transitions just as physical systems do. These phase transitions can be analyzed using the replica method, initially developed in statistical physics of disordered systems. The analysis reveals phases in which inference is easy, hard or impossible. These phases correspond to different energy landscapes of the problem. The main contributions of this thesis can be divided into three categories. First, the application of known algorithms to concrete problems: community detection, superposition codes and an innovative imaging system. Second, a new, efficient message-passing algorithm for a class of problems called blind sensor calibration. Third, a theoretical analysis of matrix compressed sensing and of instabilities in Bayesian bilinear inference algorithms.

Statistical physics, Bayesian inference and neural information processing

Information Theory and Statistical Physics - Lecture Notes

Sparse Representations, Inference and Learning

Kernels, data & physics

Lectures on Probability, Entropy, and Statistical Physics

TASI Lectures on Physics for Machine Learning

Introduction to Machine Learning: Class Notes 67577

Bayesian nonparametric statistics, St-Flour lecture notes

Machine learning with neural networks

Four lectures on computational statistical physics

Non-equilibrium physics: from spin glasses to machine and neural learning

Understanding Machine Learning Paradigms through the Lens of Statistical Thermodynamics: A tutorial

Statistical physics of complex systems: glasses, spin glasses, continuous constraint satisfaction problems, high-dimensional inference and neural networks

Lectures on Statistical Mechanics

High-Dimensional Non-Convex Landscapes and Gradient Descent Dynamics

Statistical physics of linear and bilinear inference problems

Statistical physics of inference: Thresholds and algorithms

Machine learning and the physical sciences

Information thermodynamics: from physics to neuroscience