Risk Assessment for Mobile Devices

Thomas Lederm,N. Clarke

DOI: https://doi.org/10.1007/978-3-642-22890-2_18

2011-08-29

Abstract:

What problem does this paper attempt to address?

Automated deep abstractions for stochastic chemical reaction networks

Denis Repin,Tatjana Petrov

DOI: https://doi.org/10.1016/j.ic.2021.104788

IF: 1.24

2021-12-01

Information and Computation

Abstract:Predicting stochastic cellular dynamics emerging from chemical reaction networks (CRNs) is a long-standing challenge in systems biology. Deep learning was recently used to abstract the CRN dynamics by a mixture density neural network, trained with traces of the original process. Such abstraction is dramatically cheaper to execute, yet it preserves the statistical features of the training data. However, in practice, the modeller has to take care of finding the suitable neural network architecture manually, for each given CRN, through a trial-and-error cycle. In this paper, we propose to further automatise deep abstractions for stochastic CRNs, through learning the neural network architecture along with learning the transition kernel of the stochastic process. The method is applicable to any given CRN, time-saving for deep learning experts and crucial for non-specialists. We demonstrate performance over a number of CRNs with multi-modal phenotypes and a multi-scale scenario where CRNs interact across a spatial grid.

computer science, theory & methods,mathematics, applied
Post-distillation Via Neural Resuscitation

Zhiqiang Bao,Zihao Chen,Chang-Dong Wang,Wei-Shi Zheng,Zhenhua Huang,Yunwen Chen

DOI: https://doi.org/10.1109/tmm.2023.3306601

IF: 7.3

2023-01-01

IEEE Transactions on Multimedia

Abstract:Knowledge distillation, a widely adopted model compression technique, distils knowledge from a large teacher model to a smaller student model, with the goal of reducing the computational resources required for the student model. However, most existing distillation approaches focus on the types of knowledge and how to distil them, which neglect the student model's neuronal responses to the knowledge. In this paper, we demonstrate that the kullback-leibler loss inhibits the neuronal responses in the opposite gradient direction, which injures the student model's potential during distilling. To address this problem, we introduce a principled dual-stage distillation scheme to rejuvenate all inhibited neurons at the neuronal level. In the first stage, we detect all the neurons in the student model during the standard distillation period and divide them into two parts according to their responses. In the second stage, we propose three strategies to resuscitate the neurons differently, which allows us to exploit the full potential of the student model. Through the experiments in various aspects of knowledge distillation, it is verified that the proposed approach outperforms the current state-of-the-art approaches. Our work provides a neuronal perspective for studying the response of the student model to the knowledge from the teacher model.
Learning chemical reaction networks from trajectory data

Wei Zhang,Stefan Klus,Tim Conrad,Christof Schütte

DOI: https://doi.org/10.1137/19M1265880

2019-09-29

Abstract:We develop a data-driven method to learn chemical reaction networks from trajectory data. Modeling the reaction system as a continuous-time Markov chain and assuming the system is fully observed, our method learns the propensity functions of the system with predetermined basis functions by maximizing the likelihood function of the trajectory data under $l^1$ sparse regularization. We demonstrate our method with numerical examples using synthetic data and carry out an asymptotic analysis of the proposed learning procedure in the infinite-data limit.

Optimization and Control
Probing reaction channels via reinforcement learning

Senwei Liang,Aditya N. Singh,Yuanran Zhu,David T. Limmer,Chao Yang

2023-05-28

Abstract:We propose a reinforcement learning based method to identify important configurations that connect reactant and product states along chemical reaction paths. By shooting multiple trajectories from these configurations, we can generate an ensemble of configurations that concentrate on the transition path ensemble. This configuration ensemble can be effectively employed in a neural network-based partial differential equation solver to obtain an approximation solution of a restricted Backward Kolmogorov equation, even when the dimension of the problem is very high. The resulting solution, known as the committor function, encodes mechanistic information for the reaction and can in turn be used to evaluate reaction rates.

Chemical Physics,Artificial Intelligence,Machine Learning,Numerical Analysis
Autonomous Discovery of Unknown Reaction Pathways from Data by Chemical Reaction Neural Network

Weiqi Ji,Sili Deng

DOI: https://doi.org/10.1021/acs.jpca.0c09316

2021-01-09

Abstract:Chemical reactions occur in energy, environmental, biological, and many other natural systems, and the inference of the reaction networks is essential to understand and design the chemical processes in engineering and life sciences. Yet, revealing the reaction pathways for complex systems and processes is still challenging due to the lack of knowledge of the involved species and reactions. Here, we present a neural network approach that autonomously discovers reaction pathways from the time-resolved species concentration data. The proposed Chemical Reaction Neural Network (CRNN), by design, satisfies the fundamental physics laws, including the Law of Mass Action and the Arrhenius Law. Consequently, the CRNN is physically interpretable such that the reaction pathways can be interpreted, and the kinetic parameters can be quantified simultaneously from the weights of the neural network. The inference of the chemical pathways is accomplished by training the CRNN with species concentration data via stochastic gradient descent. We demonstrate the successful implementations and the robustness of the approach in elucidating the chemical reaction pathways of several chemical engineering and biochemical systems. The autonomous inference by the CRNN approach precludes the need for expert knowledge in proposing candidate networks and addresses the curse of dimensionality in complex systems. The physical interpretability also makes the CRNN capable of not only fitting the data for a given system but also developing knowledge of unknown pathways that could be generalized to similar chemical systems.

Molecular Networks,Machine Learning,Chemical Physics
A Scalable Computational Framework for Establishing Long-Term Behavior of Stochastic Reaction Networks

Ankit Gupta,Corentin Briat,Mustafa Khammash

DOI: https://doi.org/10.1371/journal.pcbi.1003669

2014-06-26

PLoS Computational Biology

Abstract:Reaction networks are systems in which the populations of a finite number of species evolve through predefined interactions. Such networks are found as modeling tools in many biological disciplines such as biochemistry, ecology, epidemiology, immunology, systems biology and synthetic biology. It is now well-established that, for small population sizes, stochastic models for biochemical reaction networks are necessary to capture randomness in the interactions. The tools for analyzing such models, however, still lag far behind their deterministic counterparts. In this paper, we bridge this gap by developing a constructive framework for examining the long-term behavior and stability properties of the reaction dynamics in a stochastic setting. In particular, we address the problems of determining ergodicity of the reaction dynamics, which is analogous to having a globally attracting fixed point for deterministic dynamics. We also examine when the statistical moments of the underlying process remain bounded with time and when they converge to their steady state values. The framework we develop relies on a blend of ideas from probability theory, linear algebra and optimization theory. We demonstrate that the stability properties of a wide class of biological networks can be assessed from our sufficient theoretical conditions that can be recast as efficient and scalable linear programs, well-known for their tractability. It is notably shown that the computational complexity is often linear in the number of species. We illustrate the validity, the efficiency and the wide applicability of our results on several reaction networks arising in biochemistry, systems biology, epidemiology and ecology. The biological implications of the results as well as an example of a non-ergodic biological network are also discussed.

biochemical research methods,mathematical & computational biology
Exploiting network topology for large-scale inference of nonlinear reaction models

Nikhil Galagali,Youssef M Marzouk

DOI: https://doi.org/10.1098/rsif.2018.0766

2019-03-29

Abstract:The development of chemical reaction models aids understanding and prediction in areas ranging from biology to electrochemistry and combustion. A systematic approach to building reaction network models uses observational data not only to estimate unknown parameters but also to learn model structure. Bayesian inference provides a natural approach to this data-driven construction of models. Yet traditional Bayesian model inference methodologies that numerically evaluate the evidence for each model are often infeasible for nonlinear reaction network inference, as the number of plausible models can be combinatorially large. Alternative approaches based on model-space sampling can enable large-scale network inference, but their realization presents many challenges. In this paper, we present new computational methods that make large-scale nonlinear network inference tractable. First, we exploit the topology of networks describing potential interactions among chemical species to design improved 'between-model' proposals for reversible-jump Markov chain Monte Carlo. Second, we introduce a sensitivity-based determination of move types which, when combined with network-aware proposals, yields significant additional gains in sampling performance. These algorithms are demonstrated on inference problems drawn from systems biology, with nonlinear differential equation models of species interactions.
Knowledge Representing: Efficient, Sparse Representation of Prior Knowledge for Knowledge Distillation

Junjie Liu,Dongchao Wen,Hongxing Gao,Wei Tao,Tse-Wei Chen,Kinya Osa,Masami Kato

DOI: https://doi.org/10.48550/arXiv.1911.05329

2019-11-13

Abstract:Despite the recent works on knowledge distillation (KD) have achieved a further improvement through elaborately modeling the decision boundary as the posterior knowledge, their performance is still dependent on the hypothesis that the target network has a powerful capacity (representation ability). In this paper, we propose a knowledge representing (KR) framework mainly focusing on modeling the parameters distribution as prior knowledge. Firstly, we suggest a knowledge aggregation scheme in order to answer how to represent the prior knowledge from teacher network. Through aggregating the parameters distribution from teacher network into more abstract level, the scheme is able to alleviate the phenomenon of residual accumulation in the deeper layers. Secondly, as the critical issue of what the most important prior knowledge is for better distilling, we design a sparse recoding penalty for constraining the student network to learn with the penalized gradients. With the proposed penalty, the student network can effectively avoid the over-regularization during knowledge distilling and converge faster. The quantitative experiments exhibit that the proposed framework achieves the state-ofthe-arts performance, even though the target network does not have the expected capacity. Moreover, the framework is flexible enough for combining with other KD methods based on the posterior knowledge.

Computer Vision and Pattern Recognition
Efficient and scalable prediction of stochastic reaction-diffusion processes using graph neural networks

Zhixing Cao,Rui Chen,Libin Xu,Xinyi Zhou,Xiaoming Fu,Weimin Zhong,Ramon Grima

DOI: https://doi.org/10.1016/j.mbs.2024.109248

Abstract:The dynamics of locally interacting particles that are distributed in space give rise to a multitude of complex behaviours. However the simulation of reaction-diffusion processes which model such systems is highly computationally expensive, the cost increasing rapidly with the size of space. Here, we devise a graph neural network based approach that uses cheap Monte Carlo simulations of reaction-diffusion processes in a small space to cast predictions of the dynamics of the same processes in a much larger and complex space, including spaces modelled by networks with heterogeneous topology. By applying the method to two biological examples, we show that it leads to accurate results in a small fraction of the computation time of standard stochastic simulation methods. The scalability and accuracy of the method suggest it is a promising approach for studying reaction-diffusion processes in complex spatial domains such as those modelling biochemical reactions, population evolution and epidemic spreading.
Autonomous Learning of Generative Models with Chemical Reaction Network Ensembles

William Poole,Thomas E. Ouldridge,Manoj Gopalkrishnan

2023-11-07

Abstract:Can a micron sized sack of interacting molecules autonomously learn an internal model of a complex and fluctuating environment? We draw insights from control theory, machine learning theory, chemical reaction network theory, and statistical physics to develop a general architecture whereby a broad class of chemical systems can autonomously learn complex distributions. Our construction takes the form of a chemical implementation of machine learning's optimization workhorse: gradient descent on the relative entropy cost function. We show how this method can be applied to optimize any detailed balanced chemical reaction network and that the construction is capable of using hidden units to learn complex distributions. This result is then recast as a form of integral feedback control. Finally, due to our use of an explicit physical model of learning, we are able to derive thermodynamic costs and trade-offs associated to this process.

Molecular Networks,Emerging Technologies,Machine Learning,Neural and Evolutionary Computing,Systems and Control,Biological Physics
Mating patterns and rates of biological invasion.

I. Parker

DOI: https://doi.org/10.1073/PNAS.0405787101

2004-09-21

Abstract:Invasive nonnative plants are in the news because of their dramatic effects on our national agriculture, aquaculture, rangelands, water supplies, and other important resources (1, 2). Invasive plants also cause the decline of local biodiversity worldwide, with clear effects on native species and ecosystems (3, 4). For economic and ecological reasons, these invaders have been a primary focus of conservation and restoration efforts, although control programs seldom have the resources to keep up with the spread of new invaders (5). One of the conundrums of controlling invasive plants is that eradication is feasible only very early on, before a species has become widespread. However, a new nonnative plant is unlikely to attract attention or funding when it is only found here and there, especially when its populations are growing slowly and therefore seem to pose little immediate threat. Initially slow “lag phases” in population growth are suspected to be common in invasions (e.g., ref. 6), yet evidence that they occur has been mostly anecdotal. Demonstrating the occurrence of lag phases has been impeded by the nature of exponential growth. When numbers are small, population growth can be imperceptible, even if the growth rate is quite large. A lag phase implies that the rate itself increases over time. We need hard evidence that rates of population growth change over time, along with experiments to reveal the mechanism behind that change. In this issue of PNAS, Davis …
Conditional Monte Carlo for Reaction Networks

David F. Anderson,Kurt W. Ehlert

DOI: https://doi.org/10.48550/arXiv.1906.05353

2022-01-05

Abstract:Reaction networks are often used to model interacting species in fields such as biochemistry and ecology. When the counts of the species are sufficiently large, the dynamics of their concentrations are typically modeled via a system of differential equations. However, when the counts of some species are small, the dynamics of the counts are typically modeled stochastically via a discrete state, continuous time Markov chain. A key quantity of interest for such models is the probability mass function of the process at some fixed time. Since paths of such models are relatively straightforward to simulate, we can estimate the probabilities by constructing an empirical distribution. However, the support of the distribution is often diffuse across a high-dimensional state space, where the dimension is equal to the number of species. Therefore generating an accurate empirical distribution can come with a large computational cost. We present a new Monte Carlo estimator that fundamentally improves on the "classical" Monte Carlo estimator described above. It also preserves much of classical Monte Carlo's simplicity. The idea is basically one of conditional Monte Carlo. Our conditional Monte Carlo estimator has two parameters, and their choice critically affects the performance of the algorithm. Hence, a key contribution of the present work is that we demonstrate how to approximate optimal values for these parameters in an efficient manner. Moreover, we provide a central limit theorem for our estimator, which leads to approximate confidence intervals for its error.

Numerical Analysis,Probability,Molecular Networks,Quantitative Methods
Recurrent neural chemical reaction networks that approximate arbitrary dynamics

Alexander Dack,Benjamin Qureshi,Thomas E. Ouldridge,Tomislav Plesa

2024-06-06

Abstract:Many important phenomena in chemistry and biology are realized via dynamical features such as multi-stability, oscillations, and chaos. Construction of novel chemical systems with such finely-tuned dynamics is a challenging problem central to the growing field of synthetic biology. In this paper, we address this problem by putting forward a molecular version of a recurrent artificial neural network, which we call a recurrent neural chemical reaction network (RNCRN). We prove that the RNCRN, with sufficiently many auxiliary chemical species and suitable fast reactions, can be systematically trained to achieve any dynamics. This approximation ability is shown to hold independent of the initial conditions for the auxiliary species, making the RNCRN more experimentally feasible. To demonstrate the results, we present a number of relatively simple RNCRNs trained to display a variety of biologically-important dynamical features.

Molecular Networks,Dynamical Systems
Learning Interactions in Reaction Diffusion Equations by Neural Networks

Sichen Chen,Nicolas J-B Brunel,Xin Yang,Xinping Cui

DOI: https://doi.org/10.3390/e25030489

2023-03-11

Abstract:Partial differential equations are common models in biology for predicting and explaining complex behaviors. Nevertheless, deriving the equations and estimating the corresponding parameters remains challenging from data. In particular, the fine description of the interactions between species requires care for taking into account various regimes such as saturation effects. We apply a method based on neural networks to discover the underlying PDE systems, which involve fractional terms and may also contain integration terms based on observed data. Our proposed framework, called Frac-PDE-Net, adapts the PDE-Net 2.0 by adding layers that are designed to learn fractional and integration terms. The key technical challenge of this task is the identifiability issue. More precisely, one needs to identify the main terms and combine similar terms among a huge number of candidates in fractional form generated by the neural network scheme due to the division operation. In order to overcome this barrier, we set up certain assumptions according to realistic biological behavior. Additionally, we use an L2-norm based term selection criterion and the sparse regression to obtain a parsimonious model. It turns out that the method of Frac-PDE-Net is capable of recovering the main terms with accurate coefficients, allowing for effective long term prediction. We demonstrate the interest of the method on a biological PDE model proposed to study the pollen tube growth problem.
A Comprehensive Survey on Knowledge Distillation of Diffusion Models

Weijian Luo

2023-04-09

Abstract:Diffusion Models (DMs), also referred to as score-based diffusion models, utilize neural networks to specify score functions. Unlike most other probabilistic models, DMs directly model the score functions, which makes them more flexible to parametrize and potentially highly expressive for probabilistic modeling. DMs can learn fine-grained knowledge, i.e., marginal score functions, of the underlying distribution. Therefore, a crucial research direction is to explore how to distill the knowledge of DMs and fully utilize their potential. Our objective is to provide a comprehensible overview of the modern approaches for distilling DMs, starting with an introduction to DMs and a discussion of the challenges involved in distilling them into neural vector fields. We also provide an overview of the existing works on distilling DMs into both stochastic and deterministic implicit generators. Finally, we review the accelerated diffusion sampling algorithms as a training-free method for distillation. Our tutorial is intended for individuals with a basic understanding of generative models who wish to apply DM's distillation or embark on a research project in this field.

Machine Learning
Harmonizing knowledge Transfer in Neural Network with Unified Distillation

Yaomin Huang,Zaomin Yan,Chaomin Shen,Faming Fang,Guixu Zhang

2024-09-27

Abstract:Knowledge distillation (KD), known for its ability to transfer knowledge from a cumbersome network (teacher) to a lightweight one (student) without altering the architecture, has been garnering increasing attention. Two primary categories emerge within KD methods: feature-based, focusing on intermediate layers' features, and logits-based, targeting the final layer's logits. This paper introduces a novel perspective by leveraging diverse knowledge sources within a unified KD framework. Specifically, we aggregate features from intermediate layers into a comprehensive representation, effectively gathering semantic information from different stages and scales. Subsequently, we predict the distribution parameters from this representation. These steps transform knowledge from the intermediate layers into corresponding distributive forms, thereby allowing for knowledge distillation through a unified distribution constraint at different stages of the network, ensuring the comprehensiveness and coherence of knowledge transfer. Numerous experiments were conducted to validate the effectiveness of the proposed method.

Computer Vision and Pattern Recognition
Stochastic reaction networks within interacting compartments

David F. Anderson,Aidan S. Howells

2023-06-30

Abstract:Stochastic reaction networks, which are usually modeled as continuous-time Markov chains on $\mathbb Z^d_{\ge 0}$, and simulated via a version of the "Gillespie algorithm," have proven to be a useful tool for the understanding of processes, chemical and otherwise, in homogeneous environments. There are multiple avenues for generalizing away from the assumption that the environment is homogeneous, with the proper modeling choice dependent upon the context of the problem being considered. One such generalization was recently introduced in (Duso and Zechner, PNAS, 2020), where the proposed model includes a varying number of interacting compartments, or cells, each of which contains an evolving copy of the stochastic reaction system. The novelty of the model is that these compartments also interact via the merging of two compartments (including their contents), the splitting of one compartment into two, and the appearance and destruction of compartments. In this paper we begin a systematic exploration of the mathematical properties of this model. We (i) obtain basic/foundational results pertaining to explosivity, transience, recurrence, and positive recurrence of the model, (ii) explore a number of examples demonstrating some possible non-intuitive behaviors of the model, and (iii) identify the limiting distribution of the model in a special case that generalizes three formulas from an example in (Duso and Zechner, PNAS, 2020).

Probability,Quantitative Methods
State and parameter estimation from exact partial state observation in stochastic reaction networks

Muruhan Rathinam,Mingkai Yu

DOI: https://doi.org/10.1063/5.0032539

2020-12-10

Abstract:We consider chemical reaction networks modeled by a discrete state and continuous in time Markov process for the vector copy number of the species and provide a novel particle filter method for state and parameter estimation based on exact observation of some of the species in continuous time. The conditional probability distribution of the unobserved states is shown to satisfy a system of differential equations with jumps. We provide a method of simulating a process that is a proxy for the vector copy number of the unobserved species along with a weight. The resulting weighted Monte Carlo simulation is then used to compute the conditional probability distribution of the unobserved species. We also show how our algorithm can be adapted for a Bayesian estimation of parameters and for the estimation of a past state value based on observations up to a future time.

Molecular Networks,Probability
Stochastic Filtering of Reaction Networks Partially Observed in Time Snapshots

Muruhan Rathinam,Mingkai Yu

DOI: https://doi.org/10.1016/j.jcp.2024.113265

2024-07-25

Abstract:Stochastic reaction network models arise in intracellular chemical reactions, epidemiological models and other population process models, and are a class of continuous time Markov chains which have the nonnegative integer lattice as state space. We consider the problem of estimating the conditional probability distribution of a stochastic reaction network given exact partial state observations in time snapshots. We propose a particle filtering method called the targeting method. Our approach takes into account that the reaction counts in between two observation snapshots satisfy linear constraints and also uses inhomogeneous Poisson processes as proposals for the reaction counts to facilitate exact interpolation. We provide rigorous analysis as well as numerical examples to illustrate our method and compare it with other alternatives.

Numerical Analysis,Probability
Quantifying Rare Events in Stochastic Reaction-Diffusion Dynamics Using Tensor Networks

Schuyler B. Nicholson,Todd R. Gingrich

DOI: https://doi.org/10.1103/PhysRevX.13.041006

2023-07-21

Abstract:The interplay between stochastic chemical reactions and diffusion can generate rich spatiotemporal patterns. While the timescale for individual reaction or diffusion events may be very fast, the timescales for organization can be much longer. That separation of timescales makes it particularly challenging to anticipate how the rapid microscopic dynamics gives rise to macroscopic rates in the non-equilibrium dynamics of many reacting and diffusing chemical species. Within the regime of stochastic fluctuations, the standard approach is to employ Monte Carlo sampling to simulate realizations of random trajectories. Here, we present an alternative numerically tractable approach to extract macroscopic rates from the full ensemble evolution of many-body reaction diffusion problems. The approach leverages the Doi-Peliti second-quantized representation of reaction-diffusion master equations along with compression and evolution algorithms from tensor networks. By focusing on a Schlögl model with one-dimensional diffusion between $L$ otherwise well-mixed sites, we illustrate the potential of the tensor network approach to compute rates from many-body systems, here with approximately $3 \times 10^{15}$ microstates. Specifically, we compute the rate for switching between metastable macrostates, with the expense for computing those rates growing subexponentially in $L$. Because we directly work with ensemble evolutions, we crucially bypass many of the difficulties encountered by rare event sampling techniques$\unicode{x2013}$detailed balance and reaction coordinates are not needed.

Statistical Mechanics

Risk Assessment for Mobile Devices

Automated deep abstractions for stochastic chemical reaction networks

Post-distillation Via Neural Resuscitation

Learning chemical reaction networks from trajectory data

Probing reaction channels via reinforcement learning

Autonomous Discovery of Unknown Reaction Pathways from Data by Chemical Reaction Neural Network

A Scalable Computational Framework for Establishing Long-Term Behavior of Stochastic Reaction Networks

Exploiting network topology for large-scale inference of nonlinear reaction models

Knowledge Representing: Efficient, Sparse Representation of Prior Knowledge for Knowledge Distillation

Efficient and scalable prediction of stochastic reaction-diffusion processes using graph neural networks

Autonomous Learning of Generative Models with Chemical Reaction Network Ensembles

Mating patterns and rates of biological invasion.

Conditional Monte Carlo for Reaction Networks

Recurrent neural chemical reaction networks that approximate arbitrary dynamics

Learning Interactions in Reaction Diffusion Equations by Neural Networks

A Comprehensive Survey on Knowledge Distillation of Diffusion Models

Harmonizing knowledge Transfer in Neural Network with Unified Distillation

Stochastic reaction networks within interacting compartments

State and parameter estimation from exact partial state observation in stochastic reaction networks

Stochastic Filtering of Reaction Networks Partially Observed in Time Snapshots

Quantifying Rare Events in Stochastic Reaction-Diffusion Dynamics Using Tensor Networks