Abstract:In recent work we reported the vibrational spectrum of more than 100 000 known protein structures, and a self-consistent sonification method to render the spectrum in the audible range of frequencies (Qin and Buehler 2019 <i>Extreme Mech. Lett</i>. 100460). Here we present a method to transform these molecular vibrations into materialized vibrations of thin water films using acoustic actuators, leading to complex patterns of surface waves, and using the resulting macroscopic images in further processing using deep convolutional neural networks. Specifically, the patterns of water surface waves for each protein structure is used to build training sets for neural networks, aimed to classify and further process the patterns. Once trained, the neural network model is capable of discerning different proteins solely by analyzing the macroscopic surface wave patterns in the water film. Not only can the method distinguish different types of proteins (e.g. alpha-helix vs. hybrids of alpha-helices and beta-sheets), but it is also capable of determining different folding states of the same protein, or the binding events of proteins to ligands. Using the DeepDream algorithm, instances of key features of the deep neural network can be made visible in a range of images, allowing us to explore the inner workings of protein surface wave patter neural networks, as well as the creation of new images by finding and highlighting features of protein molecular spectra in a range of photographic input. The integration of the water-focused realization of cymatics, combined with neural networks and especially generative methods, offer a new direction to realize materiomusical 'Protein Inceptionism' as a possible direction in nano-inspired art. The method could have applications for detecting different protein structures, the effect of mutations, or uses in medical imaging and diagnostics, with broad impact in nano-to-macro transitions.

AI Panorama Art of Proteins with Image Blending Deep Learning

Taming Vector-Wise Quantization for Wide-Range Image Blending with Smooth Transition

Global-Context Aware Generative Protein Design

3D Structure from 2D Microscopy images using Deep Learning

DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization

GlueGAN: Gluing Two Images As a Panorama with Adversarial Learning

Novel 3D-Aware Composition Images Synthesis for Object Display with Diffusion Model.

SmartPaint: a Co-Creative Drawing System Based on Generative Adversarial Networks

Liquified protein vibrations, classification and cross-paradigm de novo image generation using deep neural networks

A new age in protein design empowered by deep learning

Structure-based protein design with deep learning

GP-GAN: Towards Realistic High-Resolution Image Blending

Illuminating protein space with a programmable generative model

Protein design via deep learning

Bridging the Visual Gap: Wide-Range Image Blending

Interactive Image Synthesis with Panoptic Layout Generation

An Artificial Intelligence-Based Stacked Ensemble Approach for Prediction of Protein Subcellular Localization in Confocal Microscopy Images

Deep Learning in Protein Structural Modeling and Design

Advances of Deep Learning in Protein Science: A Comprehensive Survey

3D Reconstruction of Protein Complex Structures Using Synthesized Multi-View AFM Images

ASAP: Interpretable Analysis and Summarization of AI-generated Image Patterns at Scale