Abstract:Computational experiments are an important method for carrying out the quantitative analysis of complex systems and play a major role in mapping the real world to the virtual world. However, the flexibility of computational experiments leads to arbitrary modeling processes and unconvincing results, which greatly hinder the large-scale application of this method. In this context, the verification of computational models has become an urgent problem in this field. Currently, model evaluation is still in its infancy and the existing evaluation methods are not mature enough. Thus, we took epidemic models as the research object and proposed a capability maturity evaluation framework for computational models of artificial society. The framework differs from previous assessment methods that focus on the validity of results, but instead provides a comprehensive evaluation from two perspectives: 1) evaluation of the model itself—by comparing the expectation with the final implementation, we can obtain whether the model meets the expectation and 2) comparison between different models—by evaluating the implementation process of each model and comparing the results, we can identify more mature models. The implementation of the model is evaluated from input, process, and output. Further, specific analyses and evaluations are conducted for several representative COVID-19 models to verify the validity of this evaluation framework. The results of the case study show that the proposed evaluation framework can help decision-makers identify more mature and referential models, and point out the directions where modelers can improve their models.

Artificial Societies, Computational Experiments, and Parallel Systems: An Investigation on a Computational Theory for Complex Socioeconomic Systems

ACP-based Social Computing and Parallel Intelligence: Societies 5.0 and Beyond

The Application of Multi-paradigm Modeling in Social Computation

Computational Experiments for Complex Social Systems: Experiment Design and Generative Explanation

Methodological innovation and its practical application on complex social and economic systems

Computational Experiments for Complex Social Systems: Integrated Design of Experiment System

Computational Experiments: Past, Present and Future

Agent-based computational economics: growing economies from the bottom up

A Software Framework For Acp In Social Computing

Computational socioeconomics

System Paradigm in the Research of Social Sciences: The Overview of Agent Computational Economics

Multi-Paradigm Artificial Society Modeling and Multi-Agent Simulation Platform Framework

Research and Implement on Dynamic Simulation Engine for Society Computing Experiments

Intelligent ACP Based Experimental Approach

An ACP-based approach for complex social network marketing system

Representative Methods of Computational Socioeconomics

Frontier Development of Agent-based Computational Economics:A Survey

Computational Experiments for Complex Social Systems—Part II: the Evaluation of Computational Models

Agent-based and macroscopic modeling of the complex socio-economic systems

KD-ACP: A Software Framework for Social Computing in Emergency Management

A self-evolving network-based artificial society model for the experiment analysis of complex social system