Abstract:Recent advancements in large language models (LLMs) have driven a revolutionary paradigm shift in process automation from Robotic Process Automation to Agentic Process Automation by automating the workflow orchestration procedure based on LLMs. However, existing LLMs (even the advanced OpenAI GPT-4o) are confined to achieving satisfactory capability in workflow orchestration. To address this limitation, we present WorkflowLLM, a data-centric framework elaborately designed to enhance the capability of LLMs in workflow orchestration. It first constructs a large-scale fine-tuning dataset WorkflowBench with 106,763 samples, covering 1,503 APIs from 83 applications across 28 categories. Specifically, the construction process can be divided into three phases: (1) Data Collection: we collect real-world workflow data from Apple Shortcuts and RoutineHub, transcribing them into Python-style code. We further equip them with generated hierarchical thought via ChatGPT. (2) Query Expansion: we prompt ChatGPT to generate more task queries to enrich the diversity and complexity of workflows. (3) Workflow Generation: we leverage an annotator model trained on collected data to generate workflows for synthesized queries. Finally, we merge the synthetic samples that pass quality confirmation with the collected samples to obtain the WorkflowBench. Based on WorkflowBench, we fine-tune Llama-3.1-8B to obtain WorkflowLlama. Our experiments show that WorkflowLlama demonstrates a strong capacity to orchestrate complex workflows, while also achieving notable generalization performance on previously unseen APIs. Additionally, WorkflowBench exhibits robust zero-shot generalization capabilities on an out-of-distribution task planning dataset, T-Eval. Our data and code are available at <a class="link-external link-https" href="https://github.com/OpenBMB/WorkflowLLM" rel="external noopener nofollow">this https URL</a>.

LLM-for-X: Application-agnostic Integration of Large Language Models to Support Personal Writing Workflows

OverleafCopilot: Empowering Academic Writing in Overleaf with Large Language Models

Low-code LLM: Graphical User Interface over Large Language Models

X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languages

LLM-based Smart Reply (LSR): Enhancing Collaborative Performance with ChatGPT-mediated Smart Reply System

WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models

When Young Scholars Cooperate with LLMs in Academic Tasks: The Influence of Individual Differences and Task Complexities

Harnessing LLMs for API Interactions: A Framework for Classification and Synthetic Data Generation

WebLLM: A High-Performance In-Browser LLM Inference Engine

Human-Centered LLM-Agent User Interface: A Position Paper

LLMs + Persona-Plug = Personalized LLMs

Task Supportive and Personalized Human-Large Language Model Interaction: A User Study

Beyond ChatBots: ExploreLLM for Structured Thoughts and Personalized Model Responses

Sketch: A Toolkit for Streamlining LLM Operations

Towards a Middleware for Large Language Models

LLM4Workflow: An LLM-based Automated Workflow Model Generation Tool

LLM With Tools: A Survey

Domain Specific Adaptation of an Open-Source LLM (Large Language Model)

The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development

A General-Purpose Device for Interaction with LLMs