Abstract:Deep learning is changing many areas in molecular physics, and it has shown great potential to deliver new solutions to challenging molecular modeling problems. Along with this trend arises the increasing demand of expressive and versatile neural network architectures which are compatible with molecular systems. A new deep neural network architecture, Molecular Configuration Transformer (Molecular CT), is introduced for this purpose. Molecular CT is composed of a relation-aware encoder module and a computationally universal geometry learning unit, thus able to account for the relational constraints between particles meanwhile scalable to different particle numbers and invariant with respect to the trans-rotational transforms. The computational efficiency and universality make Molecular CT versatile for a variety of molecular learning scenarios and especially appealing for transferable representation learning across different molecular systems. As examples, we show that Molecular CT enables representational learning for molecular systems at different scales, and achieves comparable or improved results on common benchmarks using a more light-weighted structure compared to baseline models.

What problem does this paper attempt to address?

The main goal of this paper is to propose a new deep neural network architecture—Molecular Configuration Transformer (Molecular CT)—to address the problem of unified geometric and relational representation learning for molecular systems at different scales. Specifically, the paper attempts to solve the following core issues: 1. **Unified handling of atomic and molecular systems**: Traditionally, quantum mechanics (QM) models at the atomic level and molecular mechanics (MM) models that consider intermolecular interactions and structural constraints are treated separately. This paper introduces a dual-representation framework to unify the handling of these two systems. 2. **Designing a general and efficient neural network architecture**: To effectively handle the aforementioned types of molecular systems, the authors developed a new architecture called the Molecular Configuration Transformer. This architecture consists of a relation-aware encoder module and a computationally universal geometric learning unit, which can simultaneously consider relational constraints between particles and can be extended to different numbers of particles while maintaining translational and rotational invariance. 3. **Achieving cross-scale representation learning**: The model aims to enable representation learning across different molecular scales, meaning it can flexibly handle molecular systems of varying complexity, from single atoms to entire proteins. 4. **Improving model expressiveness and transferability**: The method introduced in the paper not only improves the model's performance in predicting molecular properties but also enhances the model's transfer learning capability across different molecular systems, which is crucial for model generalization in practical applications. In summary, the main contribution of this paper is the proposal of a new model that can effectively and uniformly handle various molecular systems from the atomic level to the molecular level, while also improving the model's representation learning ability and adaptability across different systems.

Molecular CT: Unifying Geometry and Representation Learning for Molecules at Different Scales

3D-Transformer: Molecular Representation with Transformer in 3D Space

GeoMFormer: A General Architecture for Geometric Molecular Representation Learning

One Transformer Can Understand Both 2D & 3D Molecular Data

Unified 2D and 3D Pre-Training of Molecular Representations

Molecular geometric deep learning

Pretraining Graph Transformer for Molecular Representation with Fusion of Multimodal Information

CTAGE: Curvature-Based Topology-Aware Graph Embedding for Learning Molecular Representations

MOL-Mamba: Enhancing Molecular Representation with Structural & Electronic Insights

Molecular set representation learning

Deep Molecular Representation Learning via Fusing Physical and Chemical Information

Molecular Geometry-aware Transformer for accurate 3D Atomic System modeling

Learning Universal and Robust 3D Molecular Representations with Graph Convolutional Networks

Geometry-Augmented Molecular Representation Learning for Property Prediction

3DMol-Net: Learn 3D Molecular Representation using Adaptive Graph Convolutional Network Based on Rotation Invariance

Deep Learning for Multi-Scale Molecular Modeling

GTAM: A Molecular Pretraining Model with Geometric Triangle Awareness

A Universal Framework for Accurate and Efficient Geometric Deep Learning of Molecular Systems