Abstract:Data lakes are a fundamental building block for many industrial data analysis solutions and becoming increasingly popular in research. Often associated with big data use cases, data lakes are, for example, used as central data management systems of research institutions or as the core entity of machine learning pipelines. The basic underlying idea of retaining data in its native format within a data lake facilitates a large range of use cases and improves data reusability, especially when compared to the schema-on-write approach applied in data warehouses, where data is transformed prior to the actual storage to fit a predefined schema. Storing such massive amounts of raw data, however, has its very own challenges, spanning from the general data modeling, and indexing for concise querying to the integration of suitable and scalable compute capabilities. In this contribution, influential papers of the last decade have been selected to provide a comprehensive overview of developments and obtained results. The papers are analyzed with regard to the applicability of their input to data lakes that serve as central data management systems of research institutions. To achieve this, contributions to data lake architectures, metadata models, data provenance, workflow support, and FAIR principles are investigated. Last, but not least, these capabilities are mapped onto the requirements of two common research personae to identify open challenges. With that, potential research topics are determined, which have to be tackled toward the applicability of data lakes as central building blocks for research data management.

Self-supervised data lakes discovery through unsupervised metadata-driven weighted similarity

Metadata Systems for Data Lakes: Models and Features

U^2F^2S^2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection

Analytic Processing in Data Lakes: A Semantic Query-Driven Discovery Approach

Searching Data Lakes for Nested and Joined Data

On data lake architectures and metadata management

Metadata Management for Textual Documents in Data Lakes

Semantic Data Management in Data Lakes

Retrieve, Merge, Predict: Augmenting Tables with Data Lakes

A Zone-Based Data Lake Architecture for IoT, Small and Big Data

Minimally-Supervised Attribute Fusion for Data Lakes

Cross Modal Data Discovery over Structured and Unstructured Data Lakes

FREYJA: Efficient Join Discovery in Data Lakes

Coining goldMEDAL: A New Contribution to Data Lake Generic Metadata Modeling

Toward data lakes as central building blocks for data management and analysis

DatAasee -- A Metadata-Lake as Metadata Catalog for a Virtual Data-Lake

LakeBench: Benchmarks for Data Discovery over Data Lakes

Navigating the Data Lake with Datamaran: Automatically Extracting Structure from Log Datasets

Relaxed Functional Dependency Discovery in Heterogeneous Data Lakes

Joint Management and Analysis of Textual Documents and Tabular Data within the AUDAL Data Lake

Semantics-Aware Dataset Discovery from Data Lakes with Contextualized Column-Based Representation Learning