Abstract:Abstract Background Resolution estimation is the main evaluation criteria for the reconstruction of macromolecular 3D structure in the field of cryoelectron microscopy (cryo-EM). At present, there are many methods to evaluate the 3D resolution for reconstructed macromolecular structures from Single Particle Analysis (SPA) in cryo-EM and subtomogram averaging (SA) in electron cryotomography (cryo-ET). As global methods, they measure the resolution of the structure as a whole, but they are inaccurate in detecting subtle local changes of reconstruction. In order to detect the subtle changes of reconstruction of SPA and SA, a few local resolution methods are proposed. The mainstream local resolution evaluation methods are based on local Fourier shell correlation (FSC), which is computationally intensive. However, the existing resolution evaluation methods are based on multi-threading implementation on a single computer with very poor scalability. Results This paper proposes a new fine-grained 3D array partition method by key-value format in Spark. Our method first converts 3D images to key-value data (K-V). Then the K-V data is used for 3D array partitioning and data exchange in parallel. So Spark-based distributed parallel computing framework can solve the above scalability problem. In this distributed computing framework, all 3D local FSC tasks are simultaneously calculated across multiple nodes in a computer cluster. Through the calculation of experimental data, 3D local resolution evaluation algorithm based on Spark fine-grained 3D array partition has a magnitude change in computing speed compared with the mainstream FSC algorithm under the condition that the accuracy remains unchanged, and has better fault tolerance and scalability. Conclusions In this paper, we proposed a K-V format based fine-grained 3D array partition method in Spark to parallel calculating 3D FSC for getting a 3D local resolution density map. 3D local resolution density map evaluates the three-dimensional density maps reconstructed from single particle analysis and subtomogram averaging. Our proposed method can significantly increase the speed of the 3D local resolution evaluation, which is important for the efficient detection of subtle variations among reconstructed macromolecular structures.

Large-scale Parallel Design for Cryo-EM Structure Determination on Heterogeneous Many-core Architectures.

Parallelism for cryo-EM 3D reconstruction on CPU-GPU heterogeneous system

A Multi-GPU Design for Large Size Cryo-EM 3D Reconstruction

Parallelizing the Cryo‐em Structure Determination in THUNDER Using GPU Cluster

Parallelizing cryo-EM 3D reconstruction on GPU cluster with a partitioned and streamed model

Single-Particle 3d Reconstruction From Cryo-Electron Microscopy Images On Gpu

Acceleration of cryo-EM Flexible Fitting for Large Biomolecular Systems by Efficient Space Partitioning

Accelerating the Cryo-Em Structure Determination in RELION on GPU Cluster.

High-Performance Three-Dimensional Image Reconstruction for Molecular Structure Determination.

Leveraging the Hardware Resources to Accelerate Cryo-Em Reconstruction of RELION on the New Sunway Supercomputer

Experience Of Parallelizing Cryo-Em 3d Reconstruction On A Cpu-Gpu Heterogeneous System

Protein complex structure modeling by cross-modal alignment between cryo-EM maps and protein sequences

Accounting Conformational Dynamics into Structural Modeling Reflected by Cryo-EM with Deep Learning

Low cost, high performance GPU computing solution for atomic resolution cryoEM single-particle reconstruction.

SuperDragon: A Heterogeneous Parallel System for Accelerating 3D Reconstruction of Cryo-Electron Microscopy Images

Spark-based parallel calculation of 3D fourier shell correlation for macromolecule structure local resolution estimation

Large-Scale Molecular Dynamics Simulation Based on Heterogeneous Many-Core Architecture.

Massively Parallel Unsupervised Single-Particle Cryo-Em Data Clustering Via Statistical Manifold Learning

Multiscale analysis for 3D lattice structures based on parallel computing

A Model-Based Parallel Origin And For Cryotem And Its Application Orientation Refinement Algorithm To The Study Of Virus Structures

Computational Methods for in Situ Structural Studies with Cryogenic Electron Tomography.