Generative retrieval-augmented ontologic graph and multi-agent strategies for interpretive large language model-based materials design

Markus J. Buehler

2023-10-31

Abstract:Transformer neural networks show promising capabilities, in particular for uses in materials analysis, design and manufacturing, including their capacity to work effectively with both human language, symbols, code, and numerical data. Here we explore the use of large language models (LLMs) as a tool that can support engineering analysis of materials, applied to retrieving key information about subject areas, developing research hypotheses, discovery of mechanistic relationships across disparate areas of knowledge, and writing and executing simulation codes for active knowledge generation based on physical ground truths. When used as sets of AI agents with specific features, capabilities, and instructions, LLMs can provide powerful problem solution strategies for applications in analysis and design problems. Our experiments focus on using a fine-tuned model, MechGPT, developed based on training data in the mechanics of materials domain. We first affirm how finetuning endows LLMs with reasonable understanding of domain knowledge. However, when queried outside the context of learned matter, LLMs can have difficulty to recall correct information. We show how this can be addressed using retrieval-augmented Ontological Knowledge Graph strategies that discern how the model understands what concepts are important and how they are related. Illustrated for a use case of relating distinct areas of knowledge - here, music and proteins - such strategies can also provide an interpretable graph structure with rich information at the node, edge and subgraph level. We discuss nonlinear sampling strategies and agent-based modeling applied to complex question answering, code generation and execution in the context of automated force field development from actively learned Density Functional Theory (DFT) modeling, and data analysis.

Computation and Language,Disordered Systems and Neural Networks,Mesoscale and Nanoscale Physics,Materials Science,Applied Physics

What problem does this paper attempt to address?

The paper aims to address the following issues:

1. **Improving the accuracy of large language models (LLMs) in material analysis and design**: By using a strategy of Retrieval-Augmented Ontological Knowledge Graphs, the accuracy of information recall and interpretability of LLMs in handling material science-related problems is enhanced.

2. **Extending the capabilities of LLMs to handle interdisciplinary knowledge**: The paper demonstrates how these strategies can be used to link knowledge from different fields (such as music and proteins) and generate rich, interpretable graph structures.

3. **Addressing the limitations of LLMs in factual recall**: By introducing Retrieval-Augmented Generation (RAG) technology, especially when dealing with specific materials (such as molybdenene), the paper overcomes factual errors or hallucinations caused by limited training data in LLMs.

4. **Developing multi-agent systems to solve complex problems**: The paper explores how to use multi-agent modeling methods, enabling multiple LLMs to collaborate and solve problems through group chat interactions.

5. **Exploring interpretability strategies based on ontological knowledge graphs**: The paper compares traditional RAG methods with those based on ontological knowledge graphs in handling complex problems, with the latter providing more detailed and mechanistic answers.

6. **Achieving task-related ontological knowledge graph visualization**: By constructing task-related ontological knowledge graphs for specific tasks (such as defect tolerance in protein networks), the transparency and interpretability of LLM output results are improved.

7. **Proposing non-linear sampling strategies**: The paper introduces advanced sampling methods such as Chain-of-Thought prompting and tree-like thinking strategies to improve the quality and detail of LLM-generated content.

Through the above research, the paper aims to enhance the application effectiveness of LLMs in scientific research, particularly in the field of material design, and explore their potential development directions in future research.