Abstract:We propose a parallel (distributed) version of the spectral proper orthogonal decomposition (SPOD) technique. The parallel SPOD algorithm distributes the spatial dimension of the dataset preserving time. This approach is adopted to preserve the non-distributed fast Fourier transform of the data in time, thereby avoiding the associated bottlenecks. The parallel SPOD algorithm is implemented in the PySPOD library and makes use of the standard message passing interface (MPI) library, implemented in Python via mpi4py . An extensive performance evaluation of the parallel package is provided, including strong and weak scalability analyses. The open-source library allows the analysis of large datasets of interest across the scientific community. Here, we present applications in fluid dynamics and geophysics, that are extremely difficult (if not impossible) to achieve without a parallel algorithm. This work opens the path toward modal analyses of big quasi-stationary data, helping to uncover new unexplored spatio-temporal patterns. Program summary Program Title: PySPOD CPC Library link to program files: https://doi.org/10.17632/jf5bf26jcj.1 Developer's repository link: https://github.com/MathEXLab/PySPOD Licensing provisions: MIT License Programming language: Python Nature of problem: Large spatio-temporal datasets may contain coherent patterns that can be leveraged to better understand, model, and possibly predict the behaviour of complex dynamical systems. To this end, modal decomposition methods, such as the proper orthogonal decomposition (POD) and its spectral counterpart (SPOD), constitute powerful tools. The SPOD algorithm allows the systematic identification of space-time coherent patterns. This can be used to understand better the physics of the process of interest, and provide a path for mathematical modeling, including reduced order modeling. The SPOD algorithm has been successfully applied to fluid dynamics, geophysics and other domains. However, the existing open-source implementations are serial, and they prevent running on the increasingly large datasets that are becoming available, especially in computational physics. The inability to analyse via SPOD large dataset in turn prevents unlocking novel mechanisms and dynamical behaviours in complex systems. Solution method: We provide an open-source parallel (MPI distributed) code, namely PySPOD, that is able to run on large datasets (the ones considered in the present paper reach about 200 Terabytes). The code is built on the previous serial open-source code PySPOD that was published in https://joss.theoj.org/papers/10.21105/joss.02862.pdf . The new parallel implementation is able to scale on several nodes (we show both weak and strong scalability) and solve some of the bottlenecks that are commonly found at the I/O stage. The current parallel code allows running on datasets that was not easy or possible to analyse with serial SPOD algorithms, hence providing a path towards unlocking novel findings in computational physics. Additional comments including restrictions and unusual features: The code comes with a set of built-in postprocessing tools, for visualizing the results. It also comes with extensive continuous integration, documentation, and tutorials, as well as a dedicated website in addition to the associated GiHub repository. Within the package we also provide a parallel implementation of the proper orthogonal decomposition (POD), that leverages the I/O parallel capabilities of the SPOD algorithm.

Paicos: A Python package for analysis of (cosmological) simulations performed with Arepo

cortecs: A Python package for compressing opacities

Pyleoclim: Paleoclimate Timeseries Analysis and Visualization with Python

CosmoFlow: Python package for cosmological correlators

sympy2c: from symbolic expressions to fast C/C++ functions and ODE solvers in Python

The cosipy library: COSI's high-level analysis software

pysersic: A Python package for determining galaxy structural properties via Bayesian inference, accelerated with jax

PySCo: A fast Particle-Mesh $N$-body code for modified gravity simulations in Python

AsaPy: A Python Library for Aerospace Simulation Analysis

Predicting Cosmological Observables with PyCosmo

Unlocking massively parallel spectral proper orthogonal decompositions in the PySPOD package

Sherpa: An Open Source Python Fitting Package

ana_cont: Python package for analytic continuation

Sherpa: An Open-source Python Fitting Package

iCosmo: an interactive cosmology package

A Python package for the local multiscale analysis of spatial point processes (LomPy)

COPASutils: An R Package for Reading, Processing, and Visualizing Data from COPAS Large-Particle Flow Cytometers

AMEP: The Active Matter Evaluation Package for Python

Parcels v0.9: prototyping a Lagrangian ocean analysis framework for the petascale age

Gammapy: A Python package for gamma-ray astronomy