Abstract:Bayesian estimation is a major and robust estimator for many advanced statistical models. Being able to incorporate prior knowledge in statistical inference, Bayesian methods have been successfully applied in many different fields such as business, computer science, economics, epidemiology, genetics, imaging, and political science. However, due to its high computational complexity, Bayesian estimation has been deemed difficult, if not impractical, for large-scale databases, stream data, data warehouses, and data in the cloud. In this paper, we propose a novel compression and aggregation schemes (C&A) that enables distributed, parallel, or incremental computation of Bayesian estimates. Assuming partitioning of a large dataset, the C&A scheme compresses each partition into a synopsis and aggregates the synopsis into an overall Bayesian estimate without accessing the raw data. Such a C&A scheme can find applications in OLAP for data cubes, stream data mining, and cloud computing. It saves tremendous computing time since it processes each partition only once, enabling fast incremental update, and allows parallel processing. We prove that the compression is asymptotically lossless in the sense that the aggregated estimator deviates from the true model by an error that is bounded and approaches to zero when the data size increases. The results show that the proposed C&A scheme can make feasible OLAP of Bayesian estimates in a data cube. Further, it supports real-time Bayesian analysis of stream data, which can only be scanned once and cannot be permanently retained. Experimental results validate our theoretical analysis and demonstrate that our method can dramatically save time and space costs with almost no degradation of the modeling accuracy.

Bayesian compression for dynamically expandable networks

Enhanced Bayesian Compression Via Deep Reinforcement Learning

Learned Scalable Image Compression with Bidirectional Context Disentanglement Network

On Compression Principle and Bayesian Optimization for Neural Networks

BaSIC: BayesNet Structure Learning for Computational Scalable Neural Image Compression

CBANet: Toward Complexity and Bitrate Adaptive Deep Image Compression Using a Single Network.

Efficient Model Compression for Bayesian Neural Networks

Multi-Resolution Model Compression for Deep Neural Networks: A Variational Bayesian Approach

Online Bayesian Learning for Remote-Sensing Imagery Compression

1-Bit Compressive Diffusion in Networks Based on Bayesian Inference.

Dynamically Expanded CNN Array for Video Coding

Learned Compression of Encoding Distributions

Learnable Embedding Space for Efficient Neural Architecture Compression

Compression with Bayesian Implicit Neural Representations

Variable-length image compression based on controllable learning network

Bayesian Compressed Regression

BCNet: Searching for Network Width with Bilaterally Coupled Network

Fast Conditional Network Compression Using Bayesian HyperNetworks

On Model Compression for Neural Networks: Framework, Algorithm, and Convergence Guarantee

Neural Network Coding of Difference Updates for Efficient Distributed Learning Communication

Compression and Aggregation of Bayesian Estimates for Data Intensive Computing