Abstract:While mean-field models of cellular operations have identified dominant processes at the macroscopic scale, stochastic models may provide further insight into mechanisms at the molecular scale. In order to identify plausible stochastic models, quantitative comparisons between the models and the experimental data are required. The data for these systems have small sample sizes and time-evolving distributions. The aim of this study is to identify appropriate distance metrics for the quantitative comparison of stochastic model outputs and time-evolving stochastic measurements of a system. We identify distance metrics with features suitable for driving parameter inference, model comparison, and model validation, constrained by data from multiple experimental protocols. In this study, stochastic model outputs are compared to synthetic data across three scales: that of the data at the points the system is sampled during the time course of each type of experiment; a combined distance across the time course of each experiment; and a combined distance across all the experiments. Two broad categories of comparators at each point were considered, based on the empirical cumulative distribution function (ECDF) of the data and of the model outputs: discrete based measures such as the Kolmogorov-Smirnov distance, and integrated measures such as the Wasserstein-1 distance between the ECDFs. It was found that the discrete based measures were highly sensitive to parameter changes near the synthetic data parameters, but were largely insensitive otherwise, whereas the integrated distances had smoother transitions as the parameters approached the true values. The integrated measures were also found to be robust to noise added to the synthetic data, replicating experimental error. The characteristics of the identified distances provides the basis for the design of an algorithm suitable for fitting stochastic models to real world stochastic data.

Statistical Log Differencing

Using Finite-State Models for Log Differencing

Detecting abnormal situations using the Kullback-Leibler divergence.

Understanding Differencing Algorithms for Mobile Application Updates

The Logarithmic Super Divergence and its use in Statistical Inference

Datalog-based Scalable Semantic Diffing of Concurrent Programs

EvLog: Identifying Anomalous Logs over Software Evolution

Efficient Differential Dependency Discovery

The Distance Between: An Algorithmic Approach to Comparing Stochastic Models to Time-Series Data

Analyzing Logs of Large-Scale Software Systems using Time Curves Visualization

ModelDiff: A Framework for Comparing Learning Algorithms

SemDiff: Binary Similarity Detection by Diffing Key-Semantics Graphs

Log-Determinant Divergences Revisited: Alpha-Beta and Gamma Log-Det Divergences

Development of Anomaly Detection System Based on Distributed Log Tracing

A Kernel Method for the Two-Sample Problem

Toward Interactive Optimization of Source Code Differences: An Empirical Study of Its Performance

Detecting Process Concept Drifts from Event Logs.

Computing Marginal and Conditional Divergences between Decomposable Models with Applications

DiffGen: Automated Unit Test Generation for Regression Testing

Are you using test log-likelihood correctly?

Testing the martingale difference hypothesis using martingale difference divergence function