Abstract:We propose a framework to jointly determine the deformation parameters and reconstruct the unknown volume in electron cryotomography (CryoET). CryoET aims to reconstruct three-dimensional biological samples from two-dimensional projections. A major challenge is that we can only acquire projections for a limited range of tilts, and that each projection undergoes an unknown deformation during acquisition. Not accounting for these deformations results in poor reconstruction. The existing CryoET software packages attempt to align the projections, often in a workflow which uses manual feedback. Our proposed method sidesteps this inconvenience by automatically computing a set of undeformed projections while simultaneously reconstructing the unknown volume. We achieve this by learning a continuous representation of the undeformed measurements and deformation parameters. We show that our approach enables the recovery of high-frequency details that are destroyed without accounting for deformations.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to simultaneously determine the deformation parameters and reconstruct the unknown volume in electron cryo - tomography (CryoET). Specifically, CryoET aims to reconstruct three - dimensional biological samples from two - dimensional projections, but it faces two main challenges: 1. **Limited tilt - angle range**: In practical operations, CryoET can only obtain tilt - angles within a limited range (usually between - 70 degrees and + 70 degrees), which leads to the "missing wedge" problem in the measurement data. 2. **Projection deformation**: Due to the sample movement caused by mechanical stage drift and beam current, each projection will experience an unknown deformation (such as translation, shearing and rotation) during the acquisition process. If these deformations are not considered, the quality of the reconstructed volume will be significantly degraded. Existing CryoET software packages usually align the projections by means of manual feedback to correct the degradation caused by deformation. However, these methods are often based on geometric heuristics and cannot guarantee the optimal reconstruction effect. To solve these problems, the authors propose a framework that uses neural deformation fields to simultaneously recover the deformation parameters and the unknown volume. By learning the undeformed measurements and deformation parameters with continuous representations, this method can automatically optimize these parameters without alternating optimization of deformation and volume. In addition, this method also uses implicit neural networks to parameterize various types of deformations and optimize these parameters through automatic differentiation techniques. ### Formula Analysis - **Measurement model**: \[ y_m = D(\phi_m) P R(\theta_m) \rho+\eta_m, \quad m = 1, \ldots, M \] where: - \(y_m\) is the \(m\)-th projection image. - \(D(\phi_m)\) is the deformation operator with parameter \(\phi_m\). - \(P\) is the projection operator from \(\mathbb{R}^{N\times N\times N}\) to \(\mathbb{R}^{N\times N}\), that is, summing over the last coordinate. - \(R(\theta_m)\) is the rotation matrix with rotation (tilt) angle \(\theta_m\). - \(\rho\) is the unknown three - dimensional volume. - \(\eta_m\) is the independently and identically distributed Gaussian noise. - **Loss function**: - **Data consistency loss**: \[ L_{\text{data}}(\phi, \gamma)=\sum_{m = 1}^M \|D(\phi_m) f_\gamma(\theta_m, \cdot)-y_m\|_{\ell_2(X)}^2 \] - **Operation loss**: \[ L_{\text{op}}(\gamma)=\|f_\gamma - A(\theta) A^\dagger(\theta) f_\gamma\|_{\ell_2(X)}^2 \] where \(A(\theta)=[a(\theta_1), \ldots, a(\theta_M)]\), \(a(\theta_m)=P R(\theta_m)\), and \(A^\dagger(\theta)\) represents filtered back - projection. - **Regularization term**: \[ L_{\text{reg}}(\gamma)=\lambda_\theta \|\nabla_\theta f_\gamma\|_{\ell_1(X)}+\lambda_x \|\nabla_x f_\gamma\|_{\ell_1(X)} \]

Joint Cryo-ET Alignment and Reconstruction with Neural Deformation Fields

Implicit Reconstructions from Deformed Projections for CryoET

Ice-Tide: Implicit Cryo-ET Imaging and Deformation Estimation

CryoFormer: Continuous Heterogeneous Cryo-EM Reconstruction using Transformer-based Neural Representations

CryoFormer: Continuous Reconstruction of 3D Structures from Cryo-EM Data using Transformer-based Neural Representations

SparseAlign: A Super-Resolution Algorithm for Automatic Marker Localization and Deformation Estimation in Cryo-Electron Tomography

CryoAI: Amortized Inference of Poses for Ab Initio Reconstruction of 3D Molecular Volumes from Real Cryo-EM Images

Missing Wedge Completion via Unsupervised Learning with Coordinate Networks

Isotropic reconstruction for electron tomography with deep learning

A deep learning approach to the automatic detection of alignment errors in cryo-electron tomographic reconstructions

Reconstructing continuous distributions of 3D protein structure from cryo-EM images

Automated fiducial-based alignment of cryo-electron tomography tilt series in Dynamo

Rapid Synthesis of Cryo-ET Data for Training Deep Learning Models

cryoTIGER: Deep-Learning Based Tilt Interpolation Generator for Enhanced Reconstruction in Cryo Electron Tomography

A joint alignment and reconstruction algorithm for electron tomography to visualize in-depth cell-to-cell interactions

End-to-end neural ab initio reconstruction of challenging cryo-EM and cryo-ET datasets

Amortized Inference for Heterogeneous Reconstruction in Cryo-EM

A Bayesian approach to single-particle electron cryo-tomography in RELION-4.0

Adaptive Tilt-Series Alignment with Feature Resampling in Cryo-Electron Tomography

Finding Nano-Ötzi: Semi-Supervised Volume Visualization for Cryo-Electron Tomography