Image and Data Mining in Reticular Chemistry Using GPT-4V

Zhiling Zheng,Zhiguo He,Omar Khattab,Nakul Rampal,Matei A. Zaharia,Christian Borgs,Jennifer T. Chayes,Omar M. Yaghi

2023-12-09

Abstract:The integration of artificial intelligence into scientific research has reached a new pinnacle with GPT-4V, a large language model featuring enhanced vision capabilities, accessible through ChatGPT or an API. This study demonstrates the remarkable ability of GPT-4V to navigate and obtain complex data for metal-organic frameworks, especially from graphical sources. Our approach involved an automated process of converting 346 scholarly articles into 6240 images, which represents a benchmark dataset in this task, followed by deploying GPT-4V to categorize and analyze these images using natural language prompts. This methodology enabled GPT-4V to accurately identify and interpret key plots integral to MOF characterization, such as nitrogen isotherms, PXRD patterns, and TGA curves, among others, with accuracy and recall above 93%. The model's proficiency in extracting critical information from these plots not only underscores its capability in data mining but also highlights its potential in aiding the creation of comprehensive digital databases for reticular chemistry. In addition, the extracted nitrogen isotherm data from the selected literature allowed for a comparison between theoretical and experimental porosity values for over 200 compounds, highlighting certain discrepancies and underscoring the importance of integrating computational and experimental data. This work highlights the potential of AI in accelerating scientific discovery and innovation, bridging the gap between computational tools and experimental research, and paving the way for more efficient, inclusive, and comprehensive scientific inquiry.

Artificial Intelligence,Materials Science,Computer Vision and Pattern Recognition,Information Retrieval

What problem does this paper attempt to address?

The paper attempts to address the problem of how to efficiently mine and analyze complex data in the field of Reticular Chemistry using the large language model GPT-4V with enhanced visual capabilities, particularly extracting key information from graphical sources. Specifically, the research team aims to automate the process of converting academic articles into images and use GPT-4V to classify and analyze these images to identify and interpret key characterization spectra of Metal-Organic Frameworks (MOFs), such as nitrogen adsorption-desorption isotherms, X-ray powder diffraction (PXRD) patterns, thermogravimetric analysis (TGA) curves, etc.

The main objectives include:
1. **Data Mining and Information Extraction**: Automatically identify and extract key spectra from scientific literature using GPT-4V, improving the efficiency and accuracy of data mining.
2. **Database Creation**: Create a comprehensive digital database using the extracted data to support further research and discoveries in the field of Reticular Chemistry.
3. **Comparison of Experimental and Theoretical Data**: Reveal the differences between experimentally measured porosity values and theoretical calculations by comparing them, providing a more comprehensive basis for material selection.
4. **Generality and Extensibility**: Explore the potential applications of GPT-4V in other scientific fields, demonstrating its generality and adaptability across different disciplines.

Through these efforts, the research team hopes to accelerate scientific discovery and innovation, bridge the gap between computational tools and experimental research, and promote more efficient, inclusive, and comprehensive scientific research.