A Numerical Model Oriented Large-scale Parallel I/O Optimization Method.
Lei Zhang,Tao Wu,Zheng Zhou,Dong Wang,Xi Chen
DOI: https://doi.org/10.1109/DSDE58527.2023.00015
2023-01-01
Abstract:Numerical models usually contain a large number of data reading and writing, which generally takes a long time. At present, massively parallel technology has accelerated the speed of computing in model, but due to the limitation of computer I/O ability, the acceleration effect is limited, and there is a serious load imbalance during the model operation process, resulting in a waste of computing resources. To deal with that, this paper proposes a new I/O scheme, which can dynamically optimize task allocation during program operation, make full use of computing resources, and increase the mutual coverage of computing and I/O processes, so as to effectively shorten the model running time. Finally, the test in the earth simulator shows that the new I/O scheme improves the operation speed of the regional high-precision air pollution model by 25.45%.