Luca Maestrini,Robert G. Aykroyd,Matt P. Wand
Abstract:A framework is presented for fitting inverse problem models via variational Bayes approximations. This methodology guarantees flexibility to statistical model specification for a broad range of applications, good accuracy and reduced model fitting times. The message passing and factor graph fragment approach to variational Bayes that is also described facilitates streamlined implementation of approximate inference algorithms and allows for supple inclusion of numerous response distributions and penalizations into the inverse problem model. Models for one- and two-dimensional response variables are examined and an infrastructure is laid down where efficient algorithm updates based on nullifying weak interactions between variables can also be derived for inverse problems in higher dimensions. An image processing application and a simulation exercise motivated by biomedical problems reveal the computational advantage offered by efficient implementation of variational Bayes over Markov chain Monte Carlo.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to fit the inverse problem model by the variational Bayes approximations within the Bayesian framework. Specifically, the paper proposes a variational inference framework, aiming to improve the fitting flexibility, accuracy and computational efficiency of the inverse problem model. Inverse problems usually involve inferring unknown parameters from observed data. Such problems are very common in statistical regression analysis, but are often difficult to be stably solved due to the large number of parameters and the high correlation of predictor variables.
### Main Problem Description
The paper defines a reference inverse problem model, which has the following form:
\[ y = Kx + \varepsilon, \quad \varepsilon \sim N(0, \sigma^2 I), \]
where:
- \( y \) is an \( m\times1 \) observed data vector.
- \( K \) is an \( m\times m \) matrix, serving as a linear operator.
- \( x \) is an \( m\times1 \) unknown parameter vector.
- \( \varepsilon \) is a normal error vector of length \( m \).
### Bayesian Model
The Bayesian model assumptions in the paper are as follows:
\[ y_i | x, \sigma^2_\varepsilon \sim N((Kx)_i, \sigma^2_\varepsilon), \quad i = 1, \ldots, m, \]
\[ (x\Delta)_j | b_j, \sigma^2_x \sim N(0, \sigma^2_x / b_j), \quad b_j \sim \text{Inverse - }\chi^2(2, 1), \quad j = 1, \ldots, d, \]
\[ \sigma^2_\varepsilon | a_\varepsilon \sim \text{Inverse - }\chi^2(1, 1/a_\varepsilon), \quad a_\varepsilon \sim \text{Inverse - }\chi^2(1, 1/A^2_\varepsilon), \]
\[ \sigma^2_x | a_x \sim \text{Inverse - }\chi^2(1, 1/a_x), \quad a_x \sim \text{Inverse - }\chi^2(1, 1/A^2_x), \]
### Variational Bayes Methods
The paper explores two variational Bayes methods: Mean Field Variational Bayes (MFVB) and Variational Message Passing (VMP). Both methods are based on the approximate posterior density function and are achieved by minimizing the Kullback - Leibler divergence between the approximate density function and the true posterior density function.
### Computational Advantages
The paper shows the computational advantages of the variational Bayes methods over the traditional Markov Chain Monte Carlo (MCMC) method through practical biomedical data applications and simulation experiments. In particular, the VMP method can efficiently implement the approximate inference algorithm through the use of Factor Graph Fragments, thereby significantly reducing the model fitting time.
### Application Examples
The paper mentions two main application examples:
1. **Medical Positron Emission Tomography (PET)**: Process the raw data to enhance the image quality.
2. **Two - dimensional deconvolution problems in archaeology**: Expand the basic model framework by changing the response and penalty distribution assumptions.
### Conclusion
The paper proposes a new...