Open computational materials science

Aron Walsh

DOI: https://doi.org/10.1038/s41563-023-01699-7

IF: 41.2

2024-01-01

Nature Materials

Abstract:The materials modelling community is emerging as a champion for reproducible and reusable science. Aron Walsh discusses how FAIR databases, collaborative codes and transparent workflows are advancing this movement.

materials science, multidisciplinary,chemistry, physical,physics, applied, condensed matter

What problem does this paper attempt to address?

Reproducibility in Computational Materials Science: Lessons from 'A General-Purpose Machine Learning Framework for Predicting Properties of Inorganic Materials'

Daniel Persaud,Logan Ward,Jason Hattrick-Simpers

2023-10-11

Abstract:The integration of machine learning techniques in materials discovery has become prominent in materials science research and has been accompanied by an increasing trend towards open-source data and tools to propel the field. Despite the increasing usefulness and capabilities of these tools, developers neglecting to follow reproducible practices creates a significant barrier for researchers looking to use or build upon their work. In this study, we investigate the challenges encountered while attempting to reproduce a section of the results presented in "A general-purpose machine learning framework for predicting properties of inorganic materials." Our analysis identifies four major categories of challenges: (1) reporting computational dependencies, (2) recording and sharing version logs, (3) sequential code organization, and (4) clarifying code references within the manuscript. The result is a proposed set of tangible action items for those aiming to make code accessible to, and useful for the community.

Materials Science
Materials Databases: The Need for Open, Interoperable Databases with Standardized Data and Rich Metadata

François‐Xavier Coudert

DOI: https://doi.org/10.1002/adts.201900131

2019-09-18

Advanced Theory and Simulations

Abstract:Driven by the recent rapid increase in the number of materials databases published (open and commercial), some perspectives on the growing need for standardized, interoperable, open databases are discussed here. The field of computational materials discovery is quickly expanding, and recent advances in data mining, high‐throughput screening, and machine learning highlight the potential of open databases. Driven by the recent rapid increase in the number of materials databases published (both open and commercial), there is a growing need in the community for standardized, interoperable, open databases. The field of computational materials discovery is quickly expanding, and recent advances in data mining, high‐throughput screening, and machine learning highlight the potential of open databases.

multidisciplinary sciences
Community Action on FAIR Data will Fuel a Revolution in Materials Research

LC Brinson,LM Bartolo,B Blaiszik,D Elbert,I Foster,A Strachan,PW Voorhees

DOI: https://doi.org/10.48550/arXiv.2204.02881

2023-02-24

Abstract:Data - arguably the most important product of worldwide materials research investment - are rarely shared. The small and biased proportion of results published are buried in plots and text licensed by journals. This situation wastes resources, hinders innovation, and, in the current era of data-driven discovery, is no longer tenable. In this comment, we identify opportunities for synergistic, collaborative, and global actions to assemble large quantities of FAIR (Findable, Accessible, Interoperable, Reusable) (1) materials data.

Materials Science
FAIR and Open Computer Science Research Software

Wilhelm Hasselbring,Leslie Carr,Simon Hettrick,Heather Packer,Thanassis Tiropanis

DOI: https://doi.org/10.48550/arXiv.1908.05986

2019-08-16

Software Engineering

Abstract:In computational science and in computer science, research software is a central asset for research. Computational science is the application of computer science and software engineering principles to solving scientific problems, whereas computer science is the study of computer hardware and software design. The Open Science agenda holds that science advances faster when we can build on existing results. Therefore, research software has to be reusable for advancing science. Thus, we need proper research software engineering for obtaining reusable and sustainable research software. This way, software engineering methods may improve research in other disciplines. However, research in software engineering and computer science itself will also benefit from reuse when research software is involved. For good scientific practice, the resulting research software should be open and adhere to the FAIR principles (findable, accessible, interoperable and repeatable) to allow repeatability, reproducibility, and reuse. Compared to research data, research software should be both archived for reproducibility and actively maintained for reusability. The FAIR data principles do not require openness, but research software should be open source software. Established open source software licenses provide sufficient licensing options, such that it should be the rare exception to keep research software closed. We review and analyze the current state in this area in order to give recommendations for making computer science research software FAIR and open. We observe that research software publishing practices in computer science and in computational science show significant differences.
Open Materials 2024 (OMat24) Inorganic Materials Dataset and Models

Luis Barroso-Luque,Muhammed Shuaibi,Xiang Fu,Brandon M. Wood,Misko Dzamba,Meng Gao,Ammar Rizvi,C. Lawrence Zitnick,Zachary W. Ulissi

2024-10-17

Abstract:The ability to discover new materials with desirable properties is critical for numerous applications from helping mitigate climate change to advances in next generation computing hardware. AI has the potential to accelerate materials discovery and design by more effectively exploring the chemical space compared to other computational methods or by trial-and-error. While substantial progress has been made on AI for materials data, benchmarks, and models, a barrier that has emerged is the lack of publicly available training data and open pre-trained models. To address this, we present a Meta FAIR release of the Open Materials 2024 (OMat24) large-scale open dataset and an accompanying set of pre-trained models. OMat24 contains over 110 million density functional theory (DFT) calculations focused on structural and compositional diversity. Our EquiformerV2 models achieve state-of-the-art performance on the Matbench Discovery leaderboard and are capable of predicting ground-state stability and formation energies to an F1 score above 0.9 and an accuracy of 20 meV/atom, respectively. We explore the impact of model size, auxiliary denoising objectives, and fine-tuning on performance across a range of datasets including OMat24, MPtraj, and Alexandria. The open release of the OMat24 dataset and models enables the research community to build upon our efforts and drive further advancements in AI-assisted materials science.

Materials Science,Artificial Intelligence,Computational Physics
Big Data-Driven Materials Science and Its FAIR Data Infrastructure

Claudia Draxl,Matthias Scheffler

DOI: https://doi.org/10.1007/978-3-319-42913-7_104-1

2019-01-01

Abstract:This chapter addresses the forth paradigm of materials research – big data-driven materials science. Its concepts and state of the art are described, and its challenges and chances are discussed. For furthering the field, open data and an all-embracing sharing, an efficient data infrastructure, and the rich ecosystem of computer codes used in the community are of critical importance. For shaping this forth paradigm and contributing to the development or discovery of improved and novel materials, data must be what is now called FAIR – Findable, Accessible, Interoperable, and Repurposable/Reusable. This sets the stage for advances of methods from artificial intelligence that operate on large data sets to find trends and patterns that cannot be obtained from individual calculations and not even directly from high-throughput studies. Recent progress is reviewed and demonstrated, and the chapter is concluded by a forward-looking perspective, addressing important not yet solved challenges.
FAIR data enabling new horizons for materials research

Matthias Scheffler,Martin Aeschlimann,Martin Albrecht,Tristan Bereau,Hans-Joachim Bungartz,Claudia Felser,Mark Greiner,Axel Groß,Christoph T. Koch,Kurt Kremer,Wolfgang E. Nagel,Markus Scheidgen,Christof Wöll,Claudia Draxl

DOI: https://doi.org/10.1038/s41586-022-04501-x

2022-04-28

Abstract:The prosperity and lifestyle of our society are very much governed by achievements in condensed matter physics, chemistry and materials science, because new products for sectors such as energy, the environment, health, mobility and information technology (IT) rely largely on improved or even new materials. Examples include solid-state lighting, touchscreens, batteries, implants, drug delivery and many more. The enormous amount of research data produced every day in these fields represents a gold mine of the twenty-first century. This gold mine is, however, of little value if these data are not comprehensively characterized and made available. How can we refine this feedstock; that is, turn data into knowledge and value? For this, a FAIR (findable, accessible, interoperable and reusable) data infrastructure is a must. Only then can data be readily shared and explored using data analytics and artificial intelligence (AI) methods. Making data 'findable and AI ready' (a forward-looking interpretation of the acronym) will change the way in which science is carried out today. In this Perspective, we discuss how we can prepare to make this happen for the field of materials science.

Materials Science,Chemical Physics
Materials Cloud, a platform for open computational science

Leopold Talirz,Snehal Kumbhar,Elsa Passaro,Aliaksandr V. Yakutovich,Valeria Granata,Fernando Gargiulo,Marco Borelli,Martin Uhrin,Sebastiaan P. Huber,Spyros Zoupanos,Carl S. Adorf,Casper W. Andersen,Ole Schütt,Carlo A. Pignedoli,Daniele Passerone,Joost VandeVondele,Thomas C. Schulthess,Berend Smit,Giovanni Pizzi,Nicola Marzari

DOI: https://doi.org/10.1038/s41597-020-00637-5

2020-03-28

Abstract:Materials Cloud is a platform designed to enable open and seamless sharing of resources for computational science, driven by applications in materials modelling. It hosts 1) archival and dissemination services for raw and curated data, together with their provenance graph, 2) modelling services and virtual machines, 3) tools for data analytics, and pre-/post-processing, and 4) educational materials. Data is citable and archived persistently, providing a comprehensive embodiment of the FAIR principles that extends to computational workflows. Materials Cloud leverages the AiiDA framework to record the provenance of entire simulation pipelines (calculations performed, codes used, data generated) in the form of graphs that allow to retrace and reproduce any computed result. When an AiiDA database is shared on Materials Cloud, peers can browse the interconnected record of simulations, download individual files or the full database, and start their research from the results of the original authors. The infrastructure is agnostic to the specific simulation codes used and can support diverse applications in computational science that transcend its initial materials domain.

Materials Science,Computational Physics
Shared Metadata for Data-Centric Materials Science

Luca M. Ghiringhelli,Carsten Baldauf,Tristan Bereau,Sandor Brockhauser,Christian Carbogno,Javad Chamanara,Stefano Cozzini,Stefano Curtarolo,Claudia Draxl,Shyam Dwaraknath,Ádám Fekete,James Kermode,Christoph T. Koch,Markus Kühbach,Alvin Noe Ladines,Patrick Lambrix,Maja-Olivia Lenz-Himmer,Sergey Levchenko,Micael Oliveira,Adam Michalchuk,Ron Miller,Berk Onat,Pasquale Pavone,Giovanni Pizzi,Benjamin Regler,Gian-Marco Rignanese,Jörg Schaarschmidt,Markus Scheidgen,Astrid Schneidewind,Tatyana Sheveleva,Chuanxun Su,Denis Usvyat,Omar Valsson,Christof Wöll,Matthias Scheffler

2023-08-23

Abstract:The expansive production of data in materials science, their widespread sharing and repurposing requires educated support and stewardship. In order to ensure that this need helps rather than hinders scientific work, the implementation of the FAIR-data principles (Findable, Accessible, Interoperable, and Reusable) must not be too narrow. Besides, the wider materials-science community ought to agree on the strategies to tackle the challenges that are specific to its data, both from computations and experiments. In this paper, we present the result of the discussions held at the workshop on "Shared Metadata and Data Formats for Big-Data Driven Materials Science". We start from an operative definition of metadata, and what features a FAIR-compliant metadata schema should have. We will mainly focus on computational materials-science data and propose a constructive approach for the FAIRification of the (meta)data related to ground-state and excited-states calculations, potential-energy sampling, and generalized workflows. Finally, challenges with the FAIRification of experimental (meta)data and materials-science ontologies are presented together with an outlook of how to meet them.

Materials Science
Data-driven materials science: status, challenges and perspectives

Lauri Himanen,Amber Geurts,Adam S. Foster,Patrick Rinke

DOI: https://doi.org/10.1002/advs.201900808

2019-08-19

Abstract:Data-driven science is heralded as a new paradigm in materials science. In this field, data is the new resource, and knowledge is extracted from materials data sets that are too big or complex for traditional human reasoning - typically with the intent to discover new or improved materials or materials phenomena. Multiple factors, including the open science movement, national funding, and progress in information technology, have fueled its development. Such related tools as materials databases, machine learning, and high-throughput methods are now established as parts of the materials research toolset. However, there are a variety of challenges that impede progress in data-driven materials science: data veracity, integration of experimental and computational data, data longevity, standardization, and the gap between industrial interests and academic efforts. In this perspective article, we discuss the historical development and current state of data-driven materials science, building from the early evolution of open science to the rapid expansion of materials data infrastructures. We also review key successes and challenges so far, providing a perspective on the future development of the field.

Computational Physics,Materials Science
A Community Contribution Framework for Sharing Materials Data with Materials Project

Patrick Huck,Anubhav Jain,Dan Gunter,Donald Winston,Kristin Persson

DOI: https://doi.org/10.1109/eScience.2015.75

2015-10-17

Abstract:As scientific discovery becomes increasingly data-driven, software platforms are needed to efficiently organize and disseminate data from disparate sources. This is certainly the case in the field of materials science. For example, Materials Project has generated computational data on over 60,000 chemical compounds and has made that data available through a web portal and REST interface. However, such portals must seek to incorporate community submissions to expand the scope of scientific data sharing. In this paper, we describe MPContribs, a computing/software infrastructure to integrate and organize contributions of simulated or measured materials data from users. Our solution supports complex submissions and provides interfaces that allow contributors to share analyses and graphs. A RESTful API exposes mechanisms for book-keeping, retrieval and aggregation of submitted entries, as well as persistent URIs or DOIs that can be used to reference the data in publications. Our approach isolates contributed data from a host project's quality-controlled core data and yet enables analyses across the entire dataset, programmatically or through customized web apps. We expect the developed framework to enhance collaborative determination of material properties and to maximize the impact of each contributor's dataset. In the long-term, MPContribs seeks to make Materials Project an institutional, and thus community-wide, memory for computational and experimental materials science.

Software Engineering
Data‐Driven Materials Science: Status, Challenges, and Perspectives

Lauri Himanen,Amber Geurts,Adam Stuart Foster,Patrick Rinke

DOI: https://doi.org/10.1002/advs.201900808

IF: 15.1

2019-09-01

Advanced Science

Abstract:<p>Data‐driven science is heralded as a new paradigm in materials science. In this field, data is the new resource, and knowledge is extracted from materials datasets that are too big or complex for traditional human reasoning—typically with the intent to discover new or improved materials or materials phenomena. Multiple factors, including the open science movement, national funding, and progress in information technology, have fueled its development. Such related tools as materials databases, machine learning, and high‐throughput methods are now established as parts of the materials research toolset. However, there are a variety of challenges that impede progress in data‐driven materials science: data veracity, integration of experimental and computational data, data longevity, standardization, and the gap between industrial interests and academic efforts. In this perspective article, the historical development and current state of data‐driven materials science, building from the early evolution of open science to the rapid expansion of materials data infrastructures are discussed. Key successes and challenges so far are also reviewed, providing a perspective on the future development of the field.</p>

materials science, multidisciplinary,nanoscience & nanotechnology,chemistry
The open-science movement for sharing laboratory materials gains momentum

Andy Tay

DOI: https://doi.org/10.1038/d41586-024-00172-y

IF: 64.8

2024-01-23

Nature

Abstract:Many researchers support open science, but how can they translate this view into behaviours to boost sharing? Many researchers support open science, but how can they translate this view into behaviours to boost sharing?

multidisciplinary sciences
Why big data and compute are not necessarily the path to big materials science

Naohiro Fujinuma,Naohiro Fujinuma,Brian DeCost,Jason Hattrick-Simpers,Samuel E. Lofland

DOI: https://doi.org/10.1038/s43246-022-00283-x

2022-08-30

Communications Materials

Abstract:Applied machine learning has rapidly spread throughout the physical sciences. In fact, machine learning-based data analysis and experimental decision-making have become commonplace. Here, we reflect on the ongoing shift in the conversation from proving that machine learning can be used, to how to effectively implement it for advancing materials science. In particular, we advocate a shift from a big data and large-scale computations mentality to a model-oriented approach that prioritizes the use of machine learning to support the ecosystem of computational models and experimental measurements. We also recommend an open conversation about dataset bias to stabilize productive research through careful model interrogation and deliberate exploitation of known biases. Further, we encourage the community to develop machine learning methods that connect experiments with theoretical models to increase scientific understanding rather than incrementally optimizing materials. Moreover, we envision a future of radical materials innovations enabled by computational creativity tools combined with online visualization and analysis tools that support active outside-the-box thinking within the scientific knowledge feedback loop.

English Else
A Workflow-Centric Approach to Generating FAIR Data Objects for Computationally Generated Microstructure-Sensitive Mechanical Data

Ronak Shoghi,Alexander Hartmaier

DOI: https://doi.org/10.1002/adem.202401876

2024-11-18

Abstract:From a data perspective, the materials mechanics field is characterized by sparsity of available data, mainly due to the strong microstructure-sensitivity of properties like strength, fracture toughness, and fatigue limit. This requires testing specimens with different thermo-mechanical histories, even when the composition is similar. Experimental data on mechanical behavior is rare, as mechanical testing is destructive and requires significant material and effort. Furthermore, mechanical behavior is typically characterized in simplified tests under uniaxial loading conditions, whereas a complete characterization requires multiaxial testing. To address this data sparsity, simulation methods like micromechanical modeling can contribute to microstructure-sensitive data collections. This work introduces a novel data schema integrating both metadata and mechanical data, following the workflows of the material modeling processes by which the data has been generated. Each workflow run produces unique data objects by incorporating user, system, and job-specific information correlated with mechanical properties. This approach can be applied to any type of workflow as long as it is well-defined. This integrated format provides a sustainable way of generating Findable, Accessible, Interoperable, and Reusable (FAIR) data objects. The metadata elements focus on key features required to characterize microstructure-specific data, simplifying the collection of purpose-specific datasets by search algorithms.

Computational Physics,Materials Science
The case for open computer programs

Darrel C. Ince,Leslie Hatton,John Graham-Cumming

DOI: https://doi.org/10.1038/nature10836

IF: 64.8

2012-02-01

Nature

Abstract:Scientific reproducibility now very often depends on the computational method being available to duplicate, so here it is argued that all source code should be freely available.

multidisciplinary sciences
Sustainable computational science: the ReScience initiative

Nicolas P Rougier,Konrad Hinsen,Frédéric Alexandre,Thomas Arildsen,Lorena A Barba,Fabien C Y Benureau,C Titus Brown,Pierre de Buyl,Ozan Caglayan,Andrew P Davison,Marc-André Delsuc,Georgios Detorakis,Alexandra K Diem,Damien Drix,Pierre Enel,Benoît Girard,Olivia Guest,Matt G Hall,Rafael N Henriques,Xavier Hinaut,Kamil S Jaron,Mehdi Khamassi,Almar Klein,Tiina Manninen,Pietro Marchesi,Daniel McGlinn,Christoph Metzner,Owen Petchey,Hans Ekkehard Plesser,Timothée Poisot,Karthik Ram,Yoav Ram,Etienne Roesch,Cyrille Rossant,Vahid Rostami,Aaron Shifman,Jemma Stachelek,Marcel Stimberg,Frank Stollmeier,Federico Vaggi,Guillaume Viejo,Julien Vitay,Anya E Vostinar,Roman Yurchak,Tiziano Zito

DOI: https://doi.org/10.7717/peerj-cs.142

2017-12-18

Abstract:Computer science offers a large set of tools for prototyping, writing, running, testing, validating, sharing and reproducing results; however, computational science lags behind. In the best case, authors may provide their source code as a compressed archive and they may feel confident their research is reproducible. But this is not exactly true. James Buckheit and David Donoho proposed more than two decades ago that an article about computational results is advertising, not scholarship. The actual scholarship is the full software environment, code, and data that produced the result. This implies new workflows, in particular in peer-reviews. Existing journals have been slow to adapt: source codes are rarely requested and are hardly ever actually executed to check that they produce the results advertised in the article. ReScience is a peer-reviewed journal that targets computational research and encourages the explicit replication of already published research, promoting new and open-source implementations in order to ensure that the original research can be replicated from its description. To achieve this goal, the whole publishing chain is radically different from other traditional scientific journals. ReScience resides on GitHub where each new implementation of a computational study is made available together with comments, explanations, and software tests.
Semantics-Enabled Data Federation: Bringing Materials Scientists Closer to FAIR Data

Kareem S. Aggour,Vijay S. Kumar,Vipul K. Gupta,Alfredo Gabaldon,Paul Cuddihy,Varish Mulwad

DOI: https://doi.org/10.1007/s40192-024-00348-4

2024-04-09

Integrating Materials and Manufacturing Innovation

Abstract:Abstract The development and discovery of new materials can be significantly enhanced through the adoption of FAIR (Findable, Accessible, Interoperable, and Reusable) data principles and the establishment of a robust data infrastructure in support of materials informatics. A FAIR data infrastructure and associated best practices empower materials scientists to access and make the most of a wealth of information on materials properties, structures, and behaviors, allowing them to collaborate effectively, and enable data-driven approaches to material discovery. To make data findable, accessible, interoperable, and reusable to materials scientists, we developed and are in the process of expanding a materials data infrastructure to capture, store, and link data to enable a variety of analytics and visualizations. Our infrastructure follows three key architectural design philosophies: (i) capture data across a federated storage layer to minimize the storage footprint and maximize the query performance for each data type, (ii) use a knowledge graph-based data fusion layer to provide a single logical interface above the federated data repositories, and (iii) provide an ensemble of FAIR data access and reuse services atop the knowledge graph to make it easy for materials scientists and other domain experts to explore, use, and derive value from the data. This paper details our architectural approach, open-source technologies used to build the capabilities and services, and describes two applications through which we have successfully demonstrated its use. In the first use case, we created a system to enable additive manufacturing data storage and process parameter optimization with a range of user-friendly visualizations. In the second use case, we created a system for exploring data from cathodic arc deposition experiments to develop a new steam turbine coating material, fusing a combination of materials data with physics-based equations to enable advanced reasoning over the combined knowledge using a natural language chatbot-like user interface.

materials science, multidisciplinary,engineering, manufacturing
Open Science and the impact of Open Access, Open Data, and FAIR publishing principles on data-driven academic research: Towards ever more transparent, accessible, and reproducible academic output?

Gaby Umbach

DOI: https://doi.org/10.3233/sji-240021

2024-03-02

Statistical Journal of the IAOS

Abstract:Contemporary evidence-informed policy-making (EIPM) and societies require openly accessible high-quality knowledge as input into transparent and accountable decision-making and informed societal action. Open Science1 supports this requirement. As both enablers and logical consequences of the paradigm of Open Science, the ideas of Open Access, Open Data, and FAIR publishing principles revolutionise how academic research needs to be conceptualised, conducted, disseminated, published, and used. This 'academic openness quartet' is especially relevant for the ways in which research data are created, annotated, curated, managed, shared, reproduced, (re-)used, and further developed in academia. Greater accessibility of scientific output and scholarly data also aims at increasing the transparency and reproducibility of research results and the quality of research itself. In the applied 'academic openness quartet' perspective, they also function as remedies for academic malaises, like missing replicability of results or secrecy around research data. Against this backdrop, the present article offers a conceptual discussion on the four academic openness paradigms, their meanings, interrelations, as well as potential benefits and challenges arising from their application in data-driven research.
Computational Reproducibility in Computational Social Science

David Schoch,Chung-hong Chan,Claudia Wagner,Arnim Bleier

2023-10-04

Abstract:Replication crises have shaken the scientific landscape during the last decade. As potential solutions, open science practices were heavily discussed and have been implemented with varying success in different disciplines. We argue that computational-x disciplines such as computational social science, are also susceptible for the symptoms of the crises, but in terms of reproducibility. We expand the binary definition of reproducibility into a tier system which allows increasing levels of reproducibility based on external verfiability to counteract the practice of open-washing. We provide solutions for barriers in Computational Social Science that hinder researchers from obtaining the highest level of reproducibility, including the use of alternate data sources and considering reproducibility proactively.

Computers and Society

Open computational materials science

Reproducibility in Computational Materials Science: Lessons from 'A General-Purpose Machine Learning Framework for Predicting Properties of Inorganic Materials'

Materials Databases: The Need for Open, Interoperable Databases with Standardized Data and Rich Metadata

Community Action on FAIR Data will Fuel a Revolution in Materials Research

FAIR and Open Computer Science Research Software

Open Materials 2024 (OMat24) Inorganic Materials Dataset and Models

Big Data-Driven Materials Science and Its FAIR Data Infrastructure

FAIR data enabling new horizons for materials research

Materials Cloud, a platform for open computational science

Shared Metadata for Data-Centric Materials Science

Data-driven materials science: status, challenges and perspectives

A Community Contribution Framework for Sharing Materials Data with Materials Project

Data‐Driven Materials Science: Status, Challenges, and Perspectives

The open-science movement for sharing laboratory materials gains momentum

Why big data and compute are not necessarily the path to big materials science

A Workflow-Centric Approach to Generating FAIR Data Objects for Computationally Generated Microstructure-Sensitive Mechanical Data

The case for open computer programs

Sustainable computational science: the ReScience initiative

Semantics-Enabled Data Federation: Bringing Materials Scientists Closer to FAIR Data

Open Science and the impact of Open Access, Open Data, and FAIR publishing principles on data-driven academic research: Towards ever more transparent, accessible, and reproducible academic output?

Computational Reproducibility in Computational Social Science