Enabling High‐Performance Cloud Computing for Earth Science Modeling on Over a Thousand Cores: Application to the GEOS‐Chem Atmospheric Chemistry Model

Jiawei Zhuang,Daniel J. Jacob,Haipeng Lin,Elizabeth W. Lundgren,Robert M. Yantosca,Judit Flo Gaya,Melissa P. Sulprizio,Sebastian D. Eastham

DOI: https://doi.org/10.1029/2020ms002064

IF: 8.469

2020-05-01

Journal of Advances in Modeling Earth Systems

Abstract:Cloud computing platforms can facilitate the use of Earth science models by providing immediate access to fully configured software, massive computing power, and large input data sets. However, slow internode communication performance has previously discouraged the use of cloud platforms for massively parallel simulations. Here we show that recent advances in the network performance on the Amazon Web Services cloud enable efficient model simulations with over a thousand cores. The choices of Message Passing Interface library configuration and internode communication protocol are critical to this success. Application to the Goddard Earth Observing System (GEOS)‐Chem global 3‐D chemical transport model at 50‐km horizontal resolution shows efficient scaling up to at least 1,152 cores, with performance and cost comparable to the National Aeronautics and Space Administration Pleiades supercomputing cluster. Earth science model simulations are computationally expensive, typically requiring the use of high‐end supercomputing clusters that are managed by universities or national laboratories. Commercial cloud computing offers an alternative. However, past work found that cloud computing platforms were not efficient for large‐scale simulations on over 100 CPU cores, because the network communication performance on the cloud was slow compared to local clusters. Here we show that recent advances in the cloud network performance enable efficient model simulations with over a thousand cores, and cloud platforms can now serve as a viable alternative to local clusters for simulations at large scale. Computing on the cloud has extensive advantages, such as providing immediate access to fully configured model code and large data sets for any users, allowing full reproducibility of model simulation results, offering quick access to novel hardware that might not be available on local clusters, and being able to scale to virtually unlimited amounts of compute and storage resources. Those benefits will help advance Earth science modeling research. Recent advances in network performance enable efficient model simulations with over a thousand cores on the Amazon Web Services (AWS) cloud Performance and cost of cloud can be comparable to local supercomputing cluster The GEOS‐Chem chemical transport model is now available and documented for massively parallel simulations on the AWS cloud

meteorology & atmospheric sciences