Abstract:While the recent development of cryogenic electron tomography (CryoET) makes it possible to identify various macromolecules inside cells and determine their structure at near-atomic resolution, it remains challenging to visualize the complex cellular environment at the atomic level. One of the main hurdles in cell visualization is to render the millions of molecules in real time computationally. Here, using a video game engine, we demonstrate the capability of rendering massive biological macromolecules at the atomic level within their native environment. To facilitate the visualization, we also provide tools that help the interactive navigation inside the cells, as well as software that converts protein structures identified using CryoET to a scene that can be explored with the game engine.

What problem does this paper attempt to address?

This paper presents a solution to the challenge of rendering protein structures inside cells at the atomic level. While current techniques can identify macromolecules inside cells and determine their near-atomic resolution structures through cryo-electron tomography (CryoET), it is still difficult to visualize the complex cellular environment with millions of molecules in real-time. In this paper, the authors demonstrate the ability to render large biomolecular structures at the atomic level in their native environment using the video game engine, Unreal Engine. They developed tools to assist with interactive navigation and provided software to convert protein structures identified through CryoET into scenes that can be explored using the game engine. By leveraging the new virtual geometry system, Nanite, in Unreal Engine 5, the paper achieves rendering that automatically adjusts the level of detail based on the distance between the observer and the object, thereby reducing the number of polygons that need to be displayed in real-time. This enables the interactive rendering of a large number of high-resolution protein structures on consumer-grade GPUs. The paper showcases the rendering capabilities of Unreal Engine 5 with two biological systems: a salmonella ella minicell infected with the P22 bacteriophage and a partial eukaryotic cell, including parts of the Golgi apparatus and chloroplasts. The authors also developed blueprints and scripts to help users place and position protein structures in the game scene and provided interactive annotation and protein dynamic display functionalities. While the current system primarily presents static structures, the authors plan to incorporate cryo-electron microscopy heterogeneity analysis methods in the future to achieve rendering of dynamic changes in protein structures at the atomic level. Furthermore, the paper discusses support for virtual reality displays, including generating stereoscopic panoramic videos for VR devices to overcome resource limitations of real-time rendering of large molecular scenes in VR.

Rendering protein structures inside cells at the atomic level with Unreal Engine

Nanouniverse: Virtual Instancing of Structural Detail and Adaptive Shell Mapping

Game on, science - how video game technology may help biologists tackle visualization challenges

Nanomatrix: Scalable Construction of Crowded Biological Environments

CryoDRGN-ET: deep reconstructing generative networks for visualizing dynamic biomolecules inside cells

Mesoscale explorer: Visual exploration of large‐scale molecular models

Journey to the centre of the cell: Virtual reality immersion into scientific data

Mesoscale Explorer - Visual Exploration of Large-Scale Molecular Models

Virtual Bio-Molecular Modeling

Molecular Dynamics Visualization (MDV): Stereoscopic 3D Display of Biomolecular Structure and Interactions Using the Unity Game Engine

Touching proteins with virtual bare hands: how to visualize protein-drug complexes and their dynamics in virtual reality

Game Engines for Immersive Visualization: Using Unreal Engine Beyond Entertainment

Bridging structural and cell biology with cryo-electron microscopy

Visualizing Conformational Space of Functional Biomolecular Complexes by Deep Manifold Learning

Visualizing the translation landscape in human cells at high resolution

An idea to explore: Use of augmented reality for teaching three-dimensional biomolecular structures

Cellular and Structural Studies of Eukaryotic Cells by Cryo-Electron Tomography

Cell studio: A platform for interactive, 3D graphical simulation of immunological processes

Resolving individual atoms of protein complex by cryo-electron microscopy

Simulating the cellular context in synthetic datasets for cryo-electron tomography

Mesoscopic Rigid Body Modelling of the Extracellular Matrix Self-Assembly