MechGPT, a language-based strategy for mechanics and materials modeling that connects knowledge across scales, disciplines and modalities

Markus J. Buehler

2023-10-16

Abstract:For centuries, researchers have sought out ways to connect disparate areas of knowledge. While early scholars (Galileo, da Vinci, etc.) were experts across fields, specialization has taken hold later. With the advent of Artificial Intelligence, we can now explore relationships across areas (e.g., mechanics-biology) or disparate domains (e.g., failure mechanics-art). To achieve this, we use a fine-tuned Large Language Model (LLM), here for a subset of knowledge in multiscale materials failure. The approach includes the use of a general-purpose LLM to distill question-answer pairs from raw sources followed by LLM fine-tuning. The resulting MechGPT LLM foundation model is used in a series of computational experiments to explore its capacity for knowledge retrieval, various language tasks, hypothesis generation, and connecting knowledge across disparate areas. While the model has some ability to recall knowledge from training, we find that LLMs are particularly useful to extract structural insights through Ontological Knowledge Graphs. These interpretable graph structures provide explanatory insights, frameworks for new research questions, and visual representations of knowledge that also can be used in retrieval-augmented generation. Three versions of MechGPT are discussed, featuring different sizes from 13 billion to 70 billion parameters, and reaching context lengths of more than 10,000 tokens. This provides ample capacity for sophisticated retrieval augmented strategies, as well as agent-based modeling where multiple LLMs interact collaboratively and/or adversarially, the incorporation of new data from the literature or web searches, as well as multimodality.

Computation and Language,Materials Science

What problem does this paper attempt to address?

The main objective of this paper is to develop a strategic approach based on language models—MechGPT—for mechanics and materials modeling, to connect knowledge across different scales, disciplines, and modalities. Specifically, the paper aims to address the following key issues:

1. **Interdisciplinary Knowledge Connection**: Explore connections between different fields (such as mechanics and biology) or different disciplinary areas (such as failure mechanics and art) using large language models (LLMs).

2. **Knowledge Retrieval and Generation**: Utilize fine-tuned LLMs for knowledge retrieval, various language tasks, hypothesis generation, and connecting knowledge across different domains.

3. **Ontological Knowledge Graphs Generation**: Generate interpretable knowledge graphs by extracting structured insights, which can provide explanatory insights, frameworks for new research questions, and visual representations of knowledge to enhance retrieval-generation strategies.

4. **Multimodal and Cross-Domain Applications**: Explore the potential of LLMs in connecting different knowledge domains, such as applying mechanical principles to new areas like failure analysis of leaves.

5. **Model Capability Evaluation**: Assess the performance of different scales of MechGPT models (ranging from 13 billion to 70 billion parameters) on various tasks, including knowledge retrieval and hypothesis generation, and explore their applications in cross-domain research.

6. **Agent-based Modeling**: Investigate how multiple LLMs interact, collaborate, or compete to form in-depth insights on specific topics or problem-solving.

7. **Impact of Temperature Parameter**: Explore the effect of sampling temperature on the randomness and diversity of the text generated by the model, and how adjusting the temperature parameter can control the model's behavior.

8. **Role of System Prompts**: Analyze how different system prompts influence the overall behavior of the model to better control the model's output.

9. **Future Research Directions**: Propose new research directions and hypotheses based on existing research outcomes, particularly in the integration of interdisciplinary fields such as material failure, philosophy, and mathematics.

In summary, this paper aims to demonstrate how the development of fine-tuned language models specifically for material failure and related multi-scale approaches can promote the connection of interdisciplinary knowledge and innovative applications.