Abstract:The role of scalable high-performance workflows and flexible workflow management systems that can support multiple simulations will continue to increase in importance. For example, with the end of Dennard scaling, there is a need to substitute a single long running simulation with multiple repeats of shorter simulations, or concurrent replicas. Further, many scientific problems involve ensembles of simulations in order to solve a higher-level problem or produce statistically meaningful results. However most supercomputing software development and performance enhancements have focused on optimizing single- simulation performance. On the other hand, there is a strong inconsistency in the definition and practice of workflows and workflow management systems. This inconsistency often centers around the difference between several different types of workflows, including modeling and simulation, grid, uncertainty quantification, and purely conceptual workflows. This work explores this phenomenon by examining the different types of workflows and workflow management systems, reviewing the perspective of a large supercomputing facility, examining the common features and problems of workflow management systems, and finally presenting a proposed solution based on the concept of common building blocks. The implications of the continuing proliferation of workflow management systems and the lack of interoperability between these systems are discussed from a practical perspective. In doing so, we have begun an investigation of the design and implementation of open workflow systems for supercomputers based upon common components.

A Component Based Approach to Scientific Workflow Management

An Architecture Of Workflow In Grid Environment

Managing Evolving Business Workflows through the Capture of Descriptive Information

An Architectural Model for a Grid based Workflow Management Platform in Scientific Applications

Facilitating Evolution during Design and Implementation

CRISTAL : A Practical Study in Designing Systems to Cope with Change

Scientific Workflow Management and the Kepler System.

Detector Construction Management and Quality Control: Establishing and Using a CRISTAL System

Toward Common Components for Open Workflow Systems

A Workflow System Based on Software Agent for CAPP

The Deployment of an Enhanced Model-Driven Architecture for Business Process Management

Complex Workflow Management and Integration of Distributed Computing Resources by Science Gateway Portal for Molecular Dynamics Simulations in Materials Science

Towards a Grid Platform for Scientific Workflows Management

A toolbox for improving the workflow of NMR crystallography

Component based development of workflow system

An Agent-Based Approach to Component Management

A data capture and production management tool for the assembly and construction of the CMS detectors

A workflow management system for reproducible and interoperable high-throughput self-driving experiments

From Design to Production Control Through the Integration of Engineering Data Management and Workflow Management Systems

A Context- and Role-Driven Scientific Workflow Development Pattern

Crystal Toolkit: A Web App Framework to Improve Usability and Accessibility of Materials Science Research Algorithms