Abstract:Scientific progress increasingly depends on data management, particularly to clean and curate data so that it can be systematically analyzed and reused. A wealth of techniques for managing and curating data (and its provenance) have been proposed, largely in the database community. In particular, a number of influential papers have proposed collecting provenance information explaining where a piece of data was copied from, or what other records were used to derive it. Most of these techniques, however, exist only as research prototypes and are not available in mainstream database systems. This means scientists must either implement such techniques themselves or (all too often) go without. This is essentially a code reuse problem: provenance techniques currently cannot be implemented reusably, only as ad hoc, usually unmaintained extensions to standard databases. An alternative, relatively unexplored approach is to support such techniques at a higher abstraction level, using metaprogramming or reflection techniques. Can advanced programming techniques make it easier to transfer provenance research results into practice? We build on a recent approach called language-integrated provenance, which extends language-integrated query techniques with source-to-source query translations that record provenance. In previous work, a proof of concept was developed in a research programming language called Links, which supports sophisticated Web and database programming. In this paper, we show how to adapt this approach to work in Haskell building on top of the Database-Supported Haskell (DSH) library. Even though it seemed clear in principle that Haskell's rich programming features ought to be sufficient, implementing language-integrated provenance in Haskell required overcoming a number of technical challenges due to interactions between these capabilities. Our implementation serves as a proof of concept showing how this combination of metaprogramming features can, for the first time, make data provenance facilities available to programmers as a library in a widely-used, general-purpose language. In our work we were successful in implementing forms of provenance known as where-provenance and lineage. We have tested our implementation using a simple database and query set and established that the resulting queries are executed correctly on the database. Our implementation is publicly available on GitHub. Our work makes provenance tracking available to users of DSH at little cost. Although Haskell is not widely used for scientific database development, our work suggests which languages features are necessary to support provenance as library. We also highlight how combining Haskell's advanced type programming features can lead to unexpected complications, which may motivate further research into type system expressiveness.

Open Source Prover in the Attic

Open Geometry Prover Community Project

SAT Heritage: a community-driven effort for archiving, building and running more than thousand SAT solvers

Improving the efficiency of work in Open Source projects based on architectural analysis (using the example of the Sakhan project)

Trust Challenges in Reusing Open Source Software: An Interview-based Initial Study

Language-integrated provenance in Haskell

Source Code Archiving to the Rescue of Reproducible Deployment

LeanDojo: Theorem Proving with Retrieval-Augmented Language Models

Experiences from Exporting Major Proof Assistant Libraries

Open Source in Lab Management

Truth, Proof, and Reproducibility: There's no counter-attack for the codeless

Software Resurrection: Discovering Programming Pearls by Showing Modernity to Historical Software

Known Vulnerabilities of Open Source Projects: Where Are the Fixes?

READ-EVAL-PRINT in Parallel and Asynchronous Proof-checking

Beyond Notations: Hygienic Macro Expansion for Theorem Proving Languages

LiveDocs: Crafting Interactive Development Environments From Research Findings

TellTable Spreadsheet Audit: from Technical Possibility to Operating Prototype

Automated Theorem Proving in GeoGebra: Current Achievements

Modularization of Research Software for Collaborative Open Source Development

Reversible Language Extensions and their Application in Debugging

Proving and Disproving Programs with Shared Mutable Data