Abstract:The integration of experimental technologies with large language models (LLMs) is transforming scientific research, positioning AI as a versatile research assistant rather than a mere problem-solving tool. In the field of power systems, however, managing simulations -- one of the essential experimental technologies -- remains a challenge for LLMs due to their limited domain-specific knowledge, restricted reasoning capabilities, and imprecise handling of simulation parameters. To address these limitations, we propose a feedback-driven, multi-agent framework that incorporates three proposed modules: an enhanced retrieval-augmented generation (RAG) module, an improved reasoning module, and a dynamic environmental acting module with an error-feedback mechanism. Validated on 69 diverse tasks from Daline and MATPOWER, this framework achieves success rates of 93.13% and 96.85%, respectively, significantly outperforming the latest LLMs (ChatGPT 4o and o1-preview), which achieved a 27.77% success rate on standard simulation tasks and 0% on complex tasks. Additionally, our framework also supports rapid, cost-effective task execution, completing each simulation in approximately 30 seconds at an average cost of 0.014 USD for tokens. Overall, this adaptable framework lays a foundation for developing intelligent LLM-based assistants for human researchers, facilitating power system research and beyond.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in power system simulation, the capabilities of large - language models (LLMs) are limited. Specifically, the paper points out that although LLMs show strong potential in many fields, when dealing with power system simulation tasks, they have the following limitations: 1. **Limited domain - specific knowledge**: Due to the low frequency of power - system - specific knowledge in the LLMs training datasets, especially in the long - tail part of the data, this restricts the model's effective generalization ability for professional simulation tasks. 2. **Restricted reasoning ability**: The multi - step reasoning in power system simulation is a challenge for LLMs, especially when the representation of power system simulation in the model learning mode is sparse or ambiguous. 3. **Imprecise application of functions and options**: Accurately identifying simulation parameters, functions and their logical relationships poses high demands on LLMs. Especially when the knowledge of LLMs is incomplete or fragmented, it may lead to semantic drift, causing the code generation to gradually deviate from the correct version. To overcome these limitations, the paper proposes a feedback - based multi - agent framework that combines three modules: - **Enhanced retrieval - augmented generation (RAG) module**: Through an adaptive query planning strategy and triple - structure design, it expands the accessible knowledge of LLMs and improves their ability to identify and interpret simulation functions, options and their logical relationships. - **Improved reasoning module**: Utilizing professional knowledge in the simulation field, chain - of - thought prompting (CoT) and few - shot example prompting, it strengthens the understanding and reasoning ability of LLMs when generating simulation code. - **Dynamic environment interaction module**: Through an error - feedback mechanism, it realizes direct interaction with the simulation environment, provides execution feedback, and automatically corrects errors, thereby improving the reliability of simulation results. The paper verifies the effectiveness of this framework by conducting tests in multiple strategy combinations, different simulation environments (such as DALINE and MATPOWER), and diverse simulation tasks. The results show that this framework significantly improves the success rate of LLMs in power system simulation and enables fast and cost - effective task execution.

Enhancing LLMs for Power System Simulations: A Feedback-driven Multi-agent Framework

Enabling Large Language Models to Perform Power System Simulations with Previously Unseen Tools: A Case of Daline

LLM-based Frameworks for Power Engineering from Routine to Novel Tasks

STRIDE: A Tool-Assisted LLM Agent Framework for Strategic and Interactive Decision-Making

Domain-specific ReAct for physics-integrated iterative modeling: A case study of LLM agents for gas path analysis of gas turbines

LLM experiments with simulation: Large Language Model Multi-Agent System for Simulation Model Parametrization in Digital Twins

MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration

Small LLMs Are Weak Tool Learners: A Multi-LLM Agent

Synergistic Simulations: Multi-Agent Problem Solving with Large Language Models

Optimizing Collaboration of LLM based Agents for Finite Element Analysis

A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration

Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System

Towards Fully Autonomous Research Powered by LLMs: Case Study on Simulations

MetaAgents: Simulating Interactions of Human Behaviors for LLM-based Task-oriented Coordination via Collaborative Generative Agents

Research on the role of LLM in multi-agent systems: A survey

MetaOpenFOAM: an LLM-based multi-agent framework for CFD

Exploring the capabilities and limitations of large language models in the electric energy sector

LMAgent: A Large-scale Multimodal Agents Society for Multi-user Simulation

Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents

CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models