Abstract:Computational experiments are an important method for carrying out the quantitative analysis of complex systems and play a major role in mapping the real world to the virtual world. However, the flexibility of computational experiments leads to arbitrary modeling processes and unconvincing results, which greatly hinder the large-scale application of this method. In this context, the verification of computational models has become an urgent problem in this field. Currently, model evaluation is still in its infancy and the existing evaluation methods are not mature enough. Thus, we took epidemic models as the research object and proposed a capability maturity evaluation framework for computational models of artificial society. The framework differs from previous assessment methods that focus on the validity of results, but instead provides a comprehensive evaluation from two perspectives: 1) evaluation of the model itself—by comparing the expectation with the final implementation, we can obtain whether the model meets the expectation and 2) comparison between different models—by evaluating the implementation process of each model and comparing the results, we can identify more mature models. The implementation of the model is evaluated from input, process, and output. Further, specific analyses and evaluations are conducted for several representative COVID-19 models to verify the validity of this evaluation framework. The results of the case study show that the proposed evaluation framework can help decision-makers identify more mature and referential models, and point out the directions where modelers can improve their models.

Computational Experiments: A New Analysis Method for Cyber-Physical-Social Systems

Computational Experiments: Past, Present and Future

Computational Experiments for Complex Social Systems: Experiment Design and Generative Explanation

Computational Experiments for Complex Social Systems: Integrated Design of Experiment System

Computational Experiments for Complex Social Systems-Part III: the Docking of Domain Models

Optimal Coupling Pattern of Cyber-Physical Systems

A Co-Simulation Interface for Cyber-Physical Systems

Computational Experiments for Complex Social Systems--Part I: the Customization of Computational Model

Data-driven Discovery of Cyber-Physical Systems

Evaluating of Dynamic Service Matching Strategy for Social Manufacturing in Cloud Environment

Advances in Cyber-Physical Systems Research.

Survey of Modeling Methods in Cyber-Physical System

Analysis and Controlling of Manufacturing Service Ecosystem: A Research Framework Based on the Parallel System Theory

A System-Level Modeling and Design for Cyber-Physical-Social Systems.

Cyber-physical Systems Modeling Method Based on Modelica

Characteristic, Architecture, Technology, and Design Methodology of Cyber-Physical Systems

Composition Challenges and Approaches for Cyber Physical Systems.

Cyber-Physical-Social Systems: The State of the Art and Perspectives.

Research on Modeling and Analysis of CPS.

Computational Experiments for Complex Social Systems—Part II: the Evaluation of Computational Models

Study on Integrated Modeling Methods Toward Co-simulation of Cyber-Physical System