Abstract:We propose a parallel (distributed) version of the spectral proper orthogonal decomposition (SPOD) technique. The parallel SPOD algorithm distributes the spatial dimension of the dataset preserving time. This approach is adopted to preserve the non-distributed fast Fourier transform of the data in time, thereby avoiding the associated bottlenecks. The parallel SPOD algorithm is implemented in the PySPOD library and makes use of the standard message passing interface (MPI) library, implemented in Python via mpi4py . An extensive performance evaluation of the parallel package is provided, including strong and weak scalability analyses. The open-source library allows the analysis of large datasets of interest across the scientific community. Here, we present applications in fluid dynamics and geophysics, that are extremely difficult (if not impossible) to achieve without a parallel algorithm. This work opens the path toward modal analyses of big quasi-stationary data, helping to uncover new unexplored spatio-temporal patterns. Program summary Program Title: PySPOD CPC Library link to program files: https://doi.org/10.17632/jf5bf26jcj.1 Developer's repository link: https://github.com/MathEXLab/PySPOD Licensing provisions: MIT License Programming language: Python Nature of problem: Large spatio-temporal datasets may contain coherent patterns that can be leveraged to better understand, model, and possibly predict the behaviour of complex dynamical systems. To this end, modal decomposition methods, such as the proper orthogonal decomposition (POD) and its spectral counterpart (SPOD), constitute powerful tools. The SPOD algorithm allows the systematic identification of space-time coherent patterns. This can be used to understand better the physics of the process of interest, and provide a path for mathematical modeling, including reduced order modeling. The SPOD algorithm has been successfully applied to fluid dynamics, geophysics and other domains. However, the existing open-source implementations are serial, and they prevent running on the increasingly large datasets that are becoming available, especially in computational physics. The inability to analyse via SPOD large dataset in turn prevents unlocking novel mechanisms and dynamical behaviours in complex systems. Solution method: We provide an open-source parallel (MPI distributed) code, namely PySPOD, that is able to run on large datasets (the ones considered in the present paper reach about 200 Terabytes). The code is built on the previous serial open-source code PySPOD that was published in https://joss.theoj.org/papers/10.21105/joss.02862.pdf . The new parallel implementation is able to scale on several nodes (we show both weak and strong scalability) and solve some of the bottlenecks that are commonly found at the I/O stage. The current parallel code allows running on datasets that was not easy or possible to analyse with serial SPOD algorithms, hence providing a path towards unlocking novel findings in computational physics. Additional comments including restrictions and unusual features: The code comes with a set of built-in postprocessing tools, for visualizing the results. It also comes with extensive continuous integration, documentation, and tutorials, as well as a dedicated website in addition to the associated GiHub repository. Within the package we also provide a parallel implementation of the proper orthogonal decomposition (POD), that leverages the I/O parallel capabilities of the SPOD algorithm.

ObsPy: a bridge for seismology into the scientific Python ecosystem

An ObsPy library for event detection and seismic attribute calculation: preparing waveforms for automated analysis

opseestools: A Python library to streamline OpenSeesPy workflows

Pyseistr: A Python Package for Structural Denoising and Interpolation of Multichannel Seismic Data

Pyleoclim: Paleoclimate Timeseries Analysis and Visualization with Python

DiffraPy: An open-source Python software for seismic diffraction imaging

SAIPy: A Python Package for single station Earthquake Monitoring using Deep Learning

Refrapy: A Python program for seismic refraction data analysis

surfQuake: A new Python toolbox for the workflow process of seismic sources

Squidpy: a scalable framework for spatial omics analysis

osl-ephys: A Python toolbox for the analysis of electrophysiology data

Streamlining spatial omics data analysis with Pysodb

PyRefra – Refraction seismic data treatment and inversion

Unlocking massively parallel spectral proper orthogonal decompositions in the PySPOD package

Systems Neuroscience Computing in Python (SyNCoPy): A Python Package for Large-scale Analysis of Electrophysiological Data

SSA2py: A High-Performance Python Implementation of the Source-Scanning Algorithm for Spatiotemporal Seismic Source Imaging

Echopype: A Python library for interoperable and scalable processing of water column sonar data for biological information

SciPy 1.0: fundamental algorithms for scientific computing in Python

The SunPy Project: An Interoperable Ecosystem for Solar Data Analysis

MetPy: A Meteorological Python Library for Data Analysis and Visualization

Modern Python at the Large Synoptic Survey Telescope