Abstract:Molecular Simulation (MS) is a powerful tool for studying physical/chemical features of large systems and has seen applications in many scientific and engineering domains. During the simulation process, the experiments generate a very large number of atoms and intend to observe their spatial and temporal relationships for scientific analysis. The sheer data volumes and their intensive interactions impose significant challenges for data accessing, managing, and analysis. To date, existing MS software systems fall short on storage and handling of MS data, mainly because of the missing of a platform to support applications that involve intensive data access and analytical process. In this paper, we present the database-centric molecular simulation (DCMS) system our team developed in the past few years. The main idea behind DCMS is to store MS data in a relational database management system (DBMS) to take advantage of the declarative query interface (i.e., SQL), data access methods, query processing, and optimization mechanisms of modern DBMSs. A unique challenge is to handle the analytical queries that are often compute-intensive. For that, we developed novel indexing and query processing strategies (including algorithms running on modern co-processors) as integrated components of the DBMS. As a result, researchers can upload and analyze their data using efficient functions implemented inside the DBMS. Index structures are generated to store analysis results that may be interesting to other users, so that the results are readily available without duplicating the analysis. We have developed a prototype of DCMS based on the PostgreSQL system and experiments using real MS data and workload show that DCMS significantly outperforms existing MS software systems. We also used it as a platform to test other data management issues such as security and compression.

Chimera: a Virtual Data System for Representing, Querying, and Automating Data Derivation

Applying Chimera Virtual Data Concepts to Cluster Finding in the Sloan Sky Survey

Virtual Data in CMS Analysis

Virtual Data in CMS Production

Virtual Data Grid Middleware Services for Data-Intensive Science.

A virtual data language and system for scientific workflow management in data grid environments

Grid Middleware Services for Virtual Data Discovery, Composition, and Integration

Applying the Virtual Data Provenance Model

UCSF ChimeraX: Meeting modern challenges in visualization and analysis

Enhancing UCSF Chimera through web services

UCSF ChimeraX: Structure visualization for researchers, educators, and developers

Specifying and Iterating over Virtual Datasets

Data provenance tracking as the basis for a biomedical virtual research environment

Mesh Data Management in Large-Scale Scientific Computing

A Data Management Tool for Virtual Screening on Grid

Prototyping Virtual Data Technologies in ATLAS Data Challenge 1 Production

DCMS: A data analytics and management system for molecular simulation

The Galactica database: an open, generic and versatile tool for the dissemination of simulation data in astrophysics

A Semantic Cross-Species Derived Data Management Application

A Virtual Dataspaces Model for Large-Scale Materials Scientific Data Access

Virtual Experiment Services