General Data Management Workflow to Process Tabular Data in Automated and High throughput Heterogeneous Catalysis Research

Erwin Lam,Tanguy Maury,Sebastian Preiss,Yuhui Hou,Hannes Frey,Caterina Barillari,Paco Laveille
DOI: https://doi.org/10.26434/chemrxiv-2024-91vb5
2024-11-01
Abstract:Data management and processing are crucial steps to implement streamlined and standardized data workflows for automated and high-throughput laboratories. Electronic laboratory notebooks (ELNs) have proven to be effective to manage data in combination with a laboratory information management system (LIMS) to connect data and inventory. However, streamlined data processing does still pose a challenge on an ELN especially with large data. Herein we present a Python library that allows to streamline and automate data management of tabular data generated within a data-driven, automated high-throughput laboratory with a focus on heterogeneous catalysis R&D. This approach speeds up data processing and avoids errors introduced by manual data processing. Through the Python library, raw data from individual instruments related to a project are downloaded from an ELN, merged in a relational database fashion, processed and re-uploaded back to the ELN. Straightforward data merging is especially important, since information stemming from multiple devices needs to be processed together. By providing a configuration file that contain all the data management information, data merging and processing of individual data sources is executed. Having established streamlined data management workflows allows to standardize data handling and contributes to the implementation and use of open research data following Findable, Accessible, Interoperable and Reusable (FAIR) principles in the field of heterogeneous catalysis.
Chemistry
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to efficiently manage and process tabular data in automated and high - throughput heterogeneous catalysis research. Specifically, the authors are concerned with how a large amount of data generated during the experimental process can be effectively managed, merged, and processed to support the rapid progress of scientific research. The paper mentions that although electronic laboratory notebooks (ELN) and laboratory information management systems (LIMS) have proven their effectiveness in data management and inventory connection, streamlining data flow in ELN, especially when dealing with large - scale data sets, remains a challenge. Therefore, the paper proposes a Python - based library (PyCatDat), aiming to achieve automated management, merging, and processing of tabular data generated in heterogeneous catalysis research by integrating with openBIS (an open - source ELN/LIMS system), thereby accelerating data processing speed, reducing errors that may be introduced when handling data manually, and promoting the implementation and use of open research data (following the FAIR principles: findable, accessible, interoperable, and reusable).