3D Reconstruction of Protein Structures from Multi-view AFM Images using Neural Radiance Fields (NeRFs)

Jaydeep Rade,Ethan Herron,Soumik Sarkar,Anwesha Sarkar,Adarsh Krishnamurthy

2024-08-12

Abstract:Recent advancements in deep learning for predicting 3D protein structures have shown promise, particularly when leveraging inputs like protein sequences and Cryo-Electron microscopy (Cryo-EM) images. However, these techniques often fall short when predicting the structures of protein complexes (PCs), which involve multiple proteins. In our study, we investigate using atomic force microscopy (AFM) combined with deep learning to predict the 3D structures of PCs. AFM generates height maps that depict the PCs in various random orientations, providing a rich information for training a neural network to predict the 3D structures. We then employ the pre-trained UpFusion model (which utilizes a conditional diffusion model for synthesizing novel views) to train an instance-specific NeRF model for 3D reconstruction. The performance of UpFusion is evaluated through zero-shot predictions of 3D protein structures using AFM images. The challenge, however, lies in the time-intensive and impractical nature of collecting actual AFM images. To address this, we use a virtual AFM imaging process that transforms a `PDB' protein file into multi-view 2D virtual AFM images via volume rendering techniques. We extensively validate the UpFusion architecture using both virtual and actual multi-view AFM images. Our results include a comparison of structures predicted with varying numbers of views and different sets of views. This novel approach holds significant potential for enhancing the accuracy of protein complex structure predictions with further fine-tuning of the UpFusion network.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to predict the 3D structures of protein complexes (PCs) using multi - view atomic force microscopy (AFM) images. Although deep learning has made remarkable progress in predicting 3D protein structures in recent years, especially when using protein sequences and cryo - electron microscopy (Cryo - EM) images as inputs, these techniques often perform poorly when predicting the structures of protein complexes. The main reason is that protein complexes are composed of multiple proteins and have more complex structures, and existing methods are difficult to accurately capture this complexity.

To overcome this challenge, the authors propose a new method that combines the height maps generated by atomic force microscopy (AFM) and deep - learning techniques, especially using neural radiance fields (NeRFs) for 3D reconstruction. Specifically, they use the pre - trained UpFusion model, which synthesizes new views through a conditional diffusion model, and train an instance - specific NeRF model for 3D reconstruction.

In addition, because the cost and time consumption of actually collecting a large number of AFM images are relatively high, the authors also develop a virtual AFM imaging process, which converts the PDB files of proteins into multi - view 2D virtual AFM images through volume rendering techniques, so that extensive verification and testing can be carried out without actual AFM images.

In summary, the main goal of this paper is to improve the accuracy of predicting the 3D structures of protein complexes, especially in the case of limited data, by combining AFM images and deep - learning techniques.