Abstract:Accurate and comprehensive material databases extracted from research papers are crucial for materials science and engineering, but their development requires significant human effort. With large language models (LLMs) transforming the way humans interact with text, LLMs provide an opportunity to revolutionize data extraction. In this study, we demonstrate a simple and efficient method for extracting materials data from full-text research papers leveraging the capabilities of LLMs combined with human supervision. This approach is particularly suitable for mid-sized databases and requires minimal to no coding or prior knowledge about the extracted property. It offers high recall and nearly perfect precision in the resulting database. The method is easily adaptable to new and superior language models, ensuring continued utility. We show this by evaluating and comparing its performance on GPT-3 and GPT-3.5/4 (which underlie ChatGPT), as well as free alternatives such as BART and DeBERTaV3. We provide a detailed analysis of the method's performance in extracting sentences containing bulk modulus data, achieving up to 90% precision at 96% recall, depending on the amount of human effort involved. We further demonstrate the method's broader effectiveness by developing a database of critical cooling rates for metallic glasses over twice the size of previous human curated databases.

What problem does this paper attempt to address?

This paper attempts to address the issue of extracting accurate and comprehensive materials data from research papers in the field of materials science. Specifically, the authors propose a flexible and model-agnostic approach that leverages large language models (LLMs) combined with human supervision to efficiently extract materials data from full-text research papers. This method is particularly suitable for medium-sized databases, requiring minimal coding or prior knowledge of the extraction attributes, and is capable of achieving high recall rates and near-perfect precision. ### Main Issues: 1. **Challenges of Manual Data Extraction**: Currently, obtaining reliable and comprehensive materials data is crucial for many research and industrial applications. However, if the necessary information is not available through existing databases, researchers often need to manually extract data from research papers, a process that is time-consuming and labor-intensive. 2. **Limitations of Existing Automated Methods**: While fully automated data extraction methods are very useful for handling large-scale data, these methods often require complex software, training schemes, and knowledge of the extraction attributes. Additionally, these methods may fall short in terms of high integrity and accuracy. 3. **Need for Medium-Sized Databases**: For medium-sized databases (containing hundreds to thousands of data points), existing methods are either too complex or inefficient. Therefore, a new approach is needed to efficiently extract high-quality data. ### Solution: The authors propose a method based on large language models (LLMs) combined with human supervision for extracting materials data from medium-sized research papers. The main features of this method include: - **Flexibility**: It can easily adapt to new and better language models, ensuring continued practicality. - **High Precision and High Recall**: It can achieve up to 90% precision and 96% recall when extracting bulk modulus data. - **Low Human Labor Cost**: Compared to fully manual data extraction, the required human time is significantly reduced. - **Wide Applicability**: It is not only suitable for bulk modulus data but can also be extended to other types of materials data extraction tasks. ### Experimental Validation: The authors demonstrate the effectiveness of this method by evaluating and comparing the performance of different language models (such as GPT-3, GPT-3.5/4, BART, and DeBERTaV3). Additionally, they developed a database of critical cooling rates for metallic glasses, which is more than twice the size of previously manually curated databases, further proving the broad applicability and effectiveness of this method.

Flexible, Model-Agnostic Method for Materials Data Extraction from Text Using General Purpose Language Models

Flexible, Model-Agnostic Method for Materials Data Extraction from Text Using General Purpose Language Models

Extracting accurate materials data from research papers with conversational language models and prompt engineering

Mining experimental data from Materials Science literature with Large Language Models: an evaluation study

From Text to Insight: Large Language Models for Materials Science Data Extraction

Materials science in the era of large language models: a perspective

Large Language Models as Master Key: Unlocking the Secrets of Materials Science with GPT

Evaluating the Performance and Robustness of LLMs in Materials Science Q&A and Property Predictions

Beyond designer's knowledge: Generating materials design hypotheses via large language models

Towards Development of Automated Knowledge Maps and Databases for Materials Engineering using Large Language Models

Large Language Models for Material Property Predictions: elastic constant tensor prediction and materials design

MaScQA: Investigating Materials Science Knowledge of Large Language Models

AlchemBERT: Exploring Lightweight Language Models for Materials Informatics

AMGPT: a Large Language Model for Contextual Querying in Additive Manufacturing

Evaluation of Open-Source Large Language Models for Metal-Organic Frameworks Research

Polymetis:Large Language Modeling for Multiple Material Domains

Materials Informatics Transformer: A Language Model for Interpretable Materials Properties Prediction

A general-purpose material property data extraction pipeline from large polymer corpora using Natural Language Processing

Dynamic In-context Learning with Conversational Models for Data Extraction and Materials Property Prediction

Are LLMs Ready for Real-World Materials Discovery?