Abstract:Large language models (LLMs) such as ChatGPT, Gemini, LlaMa, and Claude are trained on massive quantities of text parsed from the internet and have shown a remarkable ability to respond to complex prompts in a manner often indistinguishable from humans. We present a LLM fine-tuned on up to 40,000 data that can predict electromagnetic spectra over a range of frequencies given a text prompt that only specifies the metasurface geometry. Results are compared to conventional machine learning approaches including feed-forward neural networks, random forest, linear regression, and K-nearest neighbor (KNN). Remarkably, the fine-tuned LLM (FT-LLM) achieves a lower error across all dataset sizes explored compared to all machine learning approaches including a deep neural network. We also demonstrate the LLM's ability to solve inverse problems by providing the geometry necessary to achieve a desired spectrum. LLMs possess some advantages over humans that may give them benefits for research, including the ability to process enormous amounts of data, find hidden patterns in data, and operate in higher-dimensional spaces. We propose that fine-tuning LLMs on large datasets specific to a field allows them to grasp the nuances of that domain, making them valuable tools for research and analysis.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to explore whether large - language models (LLMs) can learn and predict the physical properties of metamaterials, especially the electromagnetic spectrum, with a small amount of training data. Specifically, the paper compares the performance of a fine - tuned large - language model (FT - LLM) with traditional machine - learning methods (such as feed - forward neural networks, random forests, linear regression, and K - nearest - neighbor algorithms) in predicting the electromagnetic spectrum of metamaterials with a given geometric structure. In addition, the paper also explores the performance of these models in inverse design, that is, generating the corresponding geometric structure according to the required electromagnetic spectrum. ### Main Research Questions 1. **Prediction Performance**: How does the FT - LLM perform in predicting the electromagnetic spectrum of metamaterials under different dataset sizes? How does the FT - LLM perform compared with traditional machine - learning methods? 2. **Inverse Design Ability**: Can the FT - LLM generate the geometric structure of metamaterials with a specific electromagnetic spectrum through inverse design? 3. **Temperature Influence**: What is the influence of the temperature parameter on the prediction performance of the FT - LLM? 4. **Influence of Prompt Templates**: Do different prompt templates affect the prediction performance of the FT - LLM? 5. **Interpretability**: To what extent does the FT - LLM understand the physical properties of metamaterials? ### Research Background - **Metamaterial Research**: Metamaterials are artificially designed materials, and their electromagnetic properties can be controlled by adjusting the geometric structure. Although traditional numerical simulation methods are accurate, they are computationally expensive, while deep - learning methods (such as deep neural networks) can accelerate this process, but they require a large amount of training data. - **Large - Language Models**: LLMs such as ChatGPT, Gemini, LlaMa, and Claude, trained with large - scale text data, have demonstrated the ability to handle complex tasks. These models perform well in natural - language - processing tasks, but their application potential in the scientific field has not been fully explored. ### Methods - **Data Transformation**: Convert the geometric parameters of metamaterials into text descriptions for input into the LLM. - **Model Fine - Tuning**: Use OPENAI's API to fine - tune GPT - 3.5 to adapt it to the regression task of metamaterials. - **Evaluation Metrics**: Use the mean - squared error (MSE), the mean - absolute error (MAE), and the mean - absolute - relative error (MARE) to evaluate the model performance. ### Experimental Results - **Prediction Performance**: In the case of a low amount of data, the FT - LLM performs better than other models in the MARE metric, but performs worse in the MSE metric. As the amount of data increases, the performance of the FT - LLM improves significantly, becoming the best model when there are 40,000 training samples. - **Temperature Influence**: A low - temperature setting helps to reduce the MSE, especially on large - scale datasets. A high - temperature setting will degrade the model performance. - **Prompt Template Influence**: Different prompt templates have little impact on the model performance, indicating that the LLM is not sensitive to the representation form of the input data. - **Inverse Design**: The FT - LLM performs poorly in the inverse - design task, probably because the training data lacks diversity. - **Interpretability**: The FT - LLM has a limited understanding of the physical properties of metamaterials and cannot provide detailed physical explanations. ### Conclusion This study shows the potential of the FT - LLM in the task of predicting the electromagnetic spectrum of metamaterials, especially performing excellently on large - scale datasets. However, the LLM still has limitations in inverse design and interpretability, and future research can further optimize these aspects.

Can Large Language Models Learn the Physics of Metamaterials? An Empirical Study with ChatGPT

Large Language Models as Master Key: Unlocking the Secrets of Materials Science with GPT

Exploring large language models for microstructure evolution in materials

Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physics

What is the Role of Large Language Models in the Evolution of Astronomy Research?

A Prompt-Engineered Large Language Model, Deep Learning Workflow for Materials Classification

Large language models, physics-based modeling, experimental measurements: the trinity of data-scarce learning of polymer properties

Perspective: Large Language Models in Applied Mechanics

Materials science in the era of large language models: a perspective

What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks

How Can Large Language Models Help Humans in Design and Manufacturing?

Large Language Models: A Survey

Comparative Study of Large Language Model Architectures on Frontier

Assessing Large Language Models in Mechanical Engineering Education: A Study on Mechanics-Focused Conceptual Understanding

Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation

Physics simulation capabilities of LLMs

Regression with Large Language Models for Materials and Molecular Property Prediction

14 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model Hackathon

Mining experimental data from Materials Science literature with Large Language Models: an evaluation study

Large Language Models Meet NLP: A Survey

Assessment of Fine-Tuned Large Language Models for Real-World Chemistry and Material Science Applications