Abstract:Spatial structures in the 3D space are important to determine molecular properties. Recent papers use geometric deep learning to represent molecules and predict properties. These papers, however, are computationally expensive in capturing long-range dependencies of input atoms; and more importantly, they have not considered the non-uniformity of interatomic distances, thus failing to learn context-dependent representations at different scales. To deal with such issues, we introduce 3D-Transformer, a variant of the Transformer for molecular representations that incorporates 3D spatial information. 3D-Transformer operates on a fully-connected graph with direct connections between atoms. To cope with the non-uniformity of interatomic distances, we develop a multi-scale self-attention module that exploits local fine-grained patterns with increasing contextual scales. As molecules of different sizes rely on different kinds of spatial features, we design an adaptive position encoding module that adopts different position encoding methods for small and large molecules. Finally, to attain the molecular representation from atom embeddings, we propose an attentive farthest point sampling algorithm that selects a portion of atoms with the assistance of attention scores, overcoming handicaps of the virtual node and previous distance-dominant downsampling methods. We validate 3D-Transformer across three important scientific domains: quantum chemistry, material science,and proteomics. Our experiments show significant improvements over state-of-the-art models on the crystal property prediction task and the protein-ligand binding affinity prediction task, and show better or competitive performance in quantum chemistry molecular datasets. This work provides clear evidence that biochemical tasks can gain consistent benefits from 3D molecular representations and different tasks require different position encoding methods.

3D Mitochondria Instance Segmentation with Spatio-Temporal Transformers

Advanced Deep Networks for 3D Mitochondria Instance Segmentation

GCTransNet: 3D mitochondrial instance segmentation based on Global Context Vision Transformers

Semi-Supervised Segmentation of Mitochondria from Electron Microscopy Images Using Spatial Continuity

Automatic Mitochondria Segmentation for EM Data Using a 3D Supervised Convolutional Network

Electron Microscopy Images As Set of Fragments for Mitochondrial Segmentation

Adaptive Template Transformer for Mitochondria Segmentation in Electron Microscopy Images

A Novel 3D Connection Algorithm of Mitochondria From ATUM-SEM Stacks Based on Segmentation Information in Context

Depth Swin Transformer Unet for Serial Section Biomedical Image Segmentation

Current Progress and Challenges in Large-scale 3D Mitochondria Instance Segmentation

Mask Rearranging Data Augmentation for 3D Mitochondria Segmentation

EM-NET: Centerline-Aware Mitochondria Segmentation in EM Images via Hierarchical View-Ensemble Convolutional Network

An Effective Fully Deep Convolutional Neural Network for Mitochondria Segmentation Based on ATUM-SEM.

Stable Deep Neural Network Architectures for Mitochondria Segmentation on Electron Microscopy Volumes

Automatic Reconstruction of Mitochondria and Endoplasmic Reticulum in Electron Microscopy Volumes by Deep Learning

SegFormer3D: an Efficient Transformer for 3D Medical Image Segmentation

UNetFormer: A Unified Vision Transformer Model and Pre-Training Framework for 3D Medical Image Segmentation

Instance segmentation of mitochondria in electron microscopy images with a generalist deep learning model

3D-Transformer: Molecular Representation with Transformer in 3D Space

A Robust Volumetric Transformer for Accurate 3D Tumor Segmentation

HIVE-Net: Centerline-Aware HIerarchical View-Ensemble Convolutional Network for Mitochondria Segmentation in EM Images