Abstract:In the past years, many computational methods have been developed to infer the structure of gene regulatory networks from time-series data. However, the applicability and accuracy presumptions of such algorithms remain unclear due to experimental heterogeneity. This paper assesses the performance of recent and successful network inference strategies under a novel, multifactorial evaluation framework in order to highlight pragmatic tradeoffs in experimental design. The effects of data quantity and systems perturbations are addressed, thereby formulating guidelines for efficient resource management. Realistic data were generated from six widely used benchmark models of rhythmic and non-rhythmic gene regulatory systems with random perturbations mimicking the effect of gene knock-out or chemical treatments. Then, time-series data of increasing lengths were provided to five state-of-the-art network inference algorithms representing distinctive mathematical paradigms. The performances of such network reconstruction methodologies are uncovered under various experimental conditions. We report that the algorithms do not benefit equally from data increments. Furthermore, for rhythmic systems, it is more profitable for network inference strategies to be run on long time-series rather than short time-series with multiple perturbations. By contrast, for the non-rhythmic systems, increasing the number of perturbation experiments yielded better results than increasing the sampling frequency. We expect that future benchmark and algorithm design would integrate such multifactorial considerations to promote their widespread and conscientious usage.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to evaluate the impact of different experimental designs on the performance of network inference strategies in the reconstruction of gene regulatory networks (GRNs). Specifically, the paper focuses on the following aspects: 1. **Impact of data volume**: Research the impact of different data volumes (i.e., the length of time - series data) on the accuracy of network reconstruction. 2. **Impact of system perturbations**: Explore the impact of different types of system perturbations (such as gene knockout or chemical treatment) on the performance of network reconstruction. 3. **Trade - offs in experimental design**: Through a multi - factor evaluation framework, reveal how to select the optimal experimental design to improve the accuracy of network reconstruction under resource - limited conditions. ### Main research content - **Background and motivation**: - Genes do not work in isolation in organisms but interact through complex regulatory networks. Understanding the structure of these regulatory networks is crucial for the study of disease mechanisms and drug development. - Although many computational methods based on time - series data have been developed to infer the structure of gene regulatory networks, the applicability and accuracy of these methods under different experimental conditions remain unclear. - **Research methods**: - **Data generation**: Generate realistic time - series data from six widely used benchmark models (including one periodic system and five non - periodic systems) and simulate external interventions (such as gene knockout and chemical treatment). - **Network inference algorithms**: Use five state - of - the - art network inference algorithms (All - to - all, GPDM, dynGENIE3, ARNI, iCheMA), which represent different mathematical paradigms. - **Performance evaluation**: Evaluate the performance of each algorithm under different experimental conditions through standard classification algorithm techniques (such as the area under the ROC curve and the PR curve). ### Key findings - **Periodic systems**: - For periodic systems (such as the circadian rhythm system in plants), long - time - series data is more helpful for network inference than multiple short - time - series data. - Some algorithms (such as GPDM) perform better when dealing with short - time - series data, while other algorithms (such as ATA) perform better in long - time - series data. - **Non - periodic systems**: - For non - periodic systems, increasing the number of perturbation experiments can improve the accuracy of network reconstruction more than increasing the sampling frequency. - GPDM performs best in most cases, followed by dynGENIE3 and ARNI. ### Experimental design recommendations - **Resource management**: - Under resource - limited conditions, for periodic systems, priority should be given to extending the time of a single experiment; for non - periodic systems, the number of perturbation experiments should be increased. - By rationally selecting the experimental design, higher network reconstruction accuracy can be obtained under limited resources. ### Conclusion This study systematically evaluates the impact of different experimental designs on the performance of gene regulatory network reconstruction through a multi - factor evaluation framework, providing valuable guidance for future experimental design and algorithm development. Future research can further integrate prior knowledge, optimize experimental design, and improve the precision and efficiency of network reconstruction.

A multifactorial evaluation framework for gene regulatory network reconstruction

Gene Regulatory Network Inference from Multifactorial Perturbation Data

A computational framework for gene regulatory network inference that combines multiple methods and datasets

Gene Regulatory Network Inference from Multifactorial Perturbation Data Using Both Regression and Correlation Analyses

Improving gene regulatory network inference and assessment: The importance of using network structure

Inferring Regulatory Networks by Combining Perturbation Screens and Steady State Gene Expression Profiles

Supervised, semi-supervised and unsupervised inference of gene regulatory networks

Reconstructing Gene Regulatory Networks by Integrating Knockout and Multifactorial Perturbation Data

Gene regulatory network inference using fused LASSO on multiple data sets

Multi-Omics Regulatory Network Inference in the Presence of Missing Data

Improvements in the reconstruction of time-varying gene regulatory networks: dynamic programming and regularization by information sharing among genes

A Novel Computational Framework to Reconstruct Gene Regulatory Networks from Gene Sets

Practical Guidelines for Incorporating Knowledge-Based and Data-Driven Strategies into the Inference of Gene Regulatory Networks

Using Knowledge Driven Matrix Factorization to Reconstruct Modular Gene Regulatory Network.

Comparison and evaluation of methods to infer gene regulatory networks from multimodal single-cell data

A novel mutual information-based Boolean network inference method from time-series gene expression data

Gene Regulatory Network Reconstruction Using Conditional Mutual Information

A comprehensive overview and critical evaluation of gene regulatory network inference technologies

Quantifying the Multi-Scale Performance of Network Inference Algorithms

Gene expression complex networks: synthesis, identification, and analysis

Recovering Genetic Regulatory Networks by Integrating Multiple Data Sources