Abstract:While mean-field models of cellular operations have identified dominant processes at the macroscopic scale, stochastic models may provide further insight into mechanisms at the molecular scale. In order to identify plausible stochastic models, quantitative comparisons between the models and the experimental data are required. The data for these systems have small sample sizes and time-evolving distributions. The aim of this study is to identify appropriate distance metrics for the quantitative comparison of stochastic model outputs and time-evolving stochastic measurements of a system. We identify distance metrics with features suitable for driving parameter inference, model comparison, and model validation, constrained by data from multiple experimental protocols. In this study, stochastic model outputs are compared to synthetic data across three scales: that of the data at the points the system is sampled during the time course of each type of experiment; a combined distance across the time course of each experiment; and a combined distance across all the experiments. Two broad categories of comparators at each point were considered, based on the empirical cumulative distribution function (ECDF) of the data and of the model outputs: discrete based measures such as the Kolmogorov-Smirnov distance, and integrated measures such as the Wasserstein-1 distance between the ECDFs. It was found that the discrete based measures were highly sensitive to parameter changes near the synthetic data parameters, but were largely insensitive otherwise, whereas the integrated distances had smoother transitions as the parameters approached the true values. The integrated measures were also found to be robust to noise added to the synthetic data, replicating experimental error. The characteristics of the identified distances provides the basis for the design of an algorithm suitable for fitting stochastic models to real world stochastic data.

The bisimdist library: efficient computation of bisimilarity distances for markovian models

Bisimulation Metrics are Optimal Transport Distances, and Can be Computed Efficiently

Behavioural Pseudometrics for Nondeterministic Probabilistic Systems

A Bi-metric Framework for Fast Similarity Search

Logical, Metric, and Algorithmic Characterisations of Probabilistic Bisimulation

Approximating Bisimilarity for Markov Processes

On Metrics for Probabilistic Systems: Definitions and Algorithms

A distance function for stochastic matrices

Minimising the Probabilistic Bisimilarity Distance

A Parzen-based distance between probability measures as an alternative of summary statistics in Approximate Bayesian Computation

Probabilistic Bisimilarity As Testing Equivalence

A Spectrum of Approximate Probabilistic Bisimulations

Efficient pseudometrics for data-driven comparisons of nonlinear dynamical systems

A Local Algorithm for Checking Probabilistic Bisimilarity

A practical guide to pseudo-marginal methods for computational inference in systems biology

The Distance Between: An Algorithmic Approach to Comparing Stochastic Models to Time-Series Data

A quasi-local algorithm for checking bisimilarity

Marathon: An Open Source Software Library for the Analysis of Markov-Chain Monte Carlo Algorithms

On Behavioral Metric for Probabilistic Systems: Definition and Approximation Algorithm

Distribution-based Bisimulation and Bisimulation Metric in Probabilistic Automata