Abstract:While mean-field models of cellular operations have identified dominant processes at the macroscopic scale, stochastic models may provide further insight into mechanisms at the molecular scale. In order to identify plausible stochastic models, quantitative comparisons between the models and the experimental data are required. The data for these systems have small sample sizes and time-evolving distributions. The aim of this study is to identify appropriate distance metrics for the quantitative comparison of stochastic model outputs and time-evolving stochastic measurements of a system. We identify distance metrics with features suitable for driving parameter inference, model comparison, and model validation, constrained by data from multiple experimental protocols. In this study, stochastic model outputs are compared to synthetic data across three scales: that of the data at the points the system is sampled during the time course of each type of experiment; a combined distance across the time course of each experiment; and a combined distance across all the experiments. Two broad categories of comparators at each point were considered, based on the empirical cumulative distribution function (ECDF) of the data and of the model outputs: discrete based measures such as the Kolmogorov-Smirnov distance, and integrated measures such as the Wasserstein-1 distance between the ECDFs. It was found that the discrete based measures were highly sensitive to parameter changes near the synthetic data parameters, but were largely insensitive otherwise, whereas the integrated distances had smoother transitions as the parameters approached the true values. The integrated measures were also found to be robust to noise added to the synthetic data, replicating experimental error. The characteristics of the identified distances provides the basis for the design of an algorithm suitable for fitting stochastic models to real world stochastic data.

Le cam theory on the comparison of statistical models

Asymptotic equivalence for density estimation and gaussian white noise: An extension

On the Equivalence of Statistical Distances for Isotropic Convex Measures

Cameron–Martin type theorem for a class of non-Gaussian measures

A Unifying Framework for Global Gaussianization: Asymptotic Equivalence of Locally Stationary Processes and Bivariate White Noise

Asymptotic comparison of negative multinomial and multivariate normal experiments

Limit Theorems for Dependent Bernoulli Variables with Statistical Inference

Dualizing Le Cam's method for functional estimation, with applications to estimating the unseens

Statistical inference for pairwise comparison models

Asymptotic equivalence of discretely observed diffusion processes and their Euler scheme: small variance case

Estimating the evidence -- a review

Large-sample theory for inferential models: a possibilistic Bernstein--von Mises theorem

The Distance Between: An Algorithmic Approach to Comparing Stochastic Models to Time-Series Data

The van Trees inequality in the spirit of Hajek and Le Cam

On lower bounds for the bias-variance trade-off

The Key Theorem and the Bounds on the Rate of Uniform Convergence of Statistical Learning Theory on Quasi-Probability Spaces

A Bayesian Approach To Histogram Comparison

Design and analysis of computer experiments with both numeral and distribution inputs

A Comparison of MLE for Some Index Distributions Based on Censored Samples

Geometric ergodicity of trans-dimensional Markov chain Monte Carlo algorithms

Gaussian quasi-likelihood analysis for non-Gaussian linear mixed-effects model with system noise