Abstract:The data volumes stored in telescope archives is constantly increasing due to the development and improvements in the instrumentation. Often the archives need to be stored over a distributed storage architecture, provided by independent compute centres. Such a distributed data archive requires overarching data management orchestration. Such orchestration comprises of tools which handle data storage and cataloguing, and steering transfers integrating different storage systems and protocols, while being aware of data policies and locality. In addition, it needs a common Authorisation and Authentication Infrastructure (AAI) layer which is perceived as a single entity by end users and provides transparent data access. The scientific domain of particle physics also uses complex and distributed data management systems. The experiments at the Large Hadron Collider\,(LHC) accelerator at CERN generate several hundred petabytes of data per year. This data is globally distributed to partner sites and users using national compute facilities. Several innovative tools were developed to successfully address the distributed computing challenges in the context of the Worldwide LHC Computing Grid (WLCG). The work being carried out in the ESCAPE project and in the Data Infrastructure for Open Science (DIOS) work package is to prototype a Scientific Data Lake using the tools developed in the context of the WLCG, harnessing different physics scientific disciplines addressing FAIR standards and Open Data. We present how the Scientific Data Lake prototype is applied to address astronomical data use cases. We introduce the software stack and also discuss some of the differences between the domains.

Opportunity of Science Data Services of Library and Information Institutions Based on Data-as-a-Service

A Scalable, Adaptive, Self-management and Fault-Tolerant Architecture for Digital Library

Dataspace and its application in digital libraries

The student staffing advantage: Data Science Consulting Service at NC State University Libraries

Towards Realization of Open Cloud Services Architecture for Data Service in Materials Field ⋆

Dataspace:A New Data Organization and Management Model

Welcome from DSAA 2014 Chairs

Data Mining Service Architecture - an Exploration on Self-organizing Software.

Data science in data librarianship: Core competencies of a data librarian

Scientific Data Management and Service in University Library——A Case Study of Wuhan University Library

Empowering Dataspace 4.0: Unveiling Promise of Decentralized Data-Sharing

Social Scientific Data Sharing and Services in the United States

Toward Democratizing Access to Facilities Data: A Framework for Intelligent Data Discovery and Delivery

Internet of Data:A Solution for Dataspace Infrastructure and Its Technical Challenges

Big Data Construction and Sharing from the Perspective of Library

Research and Implementation of a Science Data Cloud and Services

DaaS - Internet-perception Big Data Systems Based on AI.

Astronomical data organization, management and access in Scientific Data Lakes

The Intelligent Service Mode of University Library Based on Internet

Data as a Research Infrastructure - CDS, the Virtual Observatory, Astronomy, and beyond

Data Infrastructure for Remote Sensing Big Data:Integration ,management and On-Demand Service