Effort and Size Estimation in Software Projects with Large Language Model-based Intelligent Interfaces

Claudionor N. Coelho Jr,Hanchen Xiong,Tushar Karayil,Sree Koratala,Rex Shang,Jacob Bollinger,Mohamed Shabar,Syam Nair

2024-06-28

Abstract:The advancement of Large Language Models (LLM) has also resulted in an equivalent proliferation in its applications. Software design, being one, has gained tremendous benefits in using LLMs as an interface component that extends fixed user stories. However, inclusion of LLM-based AI agents in software design often poses unexpected challenges, especially in the estimation of development efforts. Through the example of UI-based user stories, we provide a comparison against traditional methods and propose a new way to enhance specifications of natural language-based questions that allows for the estimation of development effort by taking into account data sources, interfaces and algorithms.

Software Engineering,Machine Learning

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is: **How to accurately estimate the development effort and scale in software projects that use large language models (LLMs) as user interface components**.

Specifically, the paper focuses on the fact that with the development of LLM technology, the traditional user - story design method based on UI/UX is gradually being replaced by natural - language interfaces. Although this transformation improves user accessibility and software adaptability, it also introduces new challenges, especially when estimating development effort. Due to the flexibility and ambiguity of natural - language descriptions, traditional estimation methods are no longer applicable, resulting in difficulty in accurately predicting the development time and resource requirements of projects.

To address this challenge, the paper proposes a new method to enhance the specification of natural - language problems through the following steps, thereby achieving more accurate effort estimation:

1. **Generate related questions**: Use an LLM to generate multiple questions related to the original user request. These questions are raised based on various aspects that the user may be interested in.
2. **Plan sub - tasks**: Utilize the Planner component in the AI Agent to create a list of required sub - tasks based on the generated set of questions. These sub - tasks include data sources to be accessed, algorithms to be called, and user interfaces to be implemented.
3. **Manual verification and optimization**: Manually verify the generated sub - tasks, remove duplicate or unnecessary tasks, and ensure that the final task list is streamlined and accurate.

Through this method, the paper demonstrates how to restore the precision similar to that of traditional user stories and use cases, thereby enabling better estimation of the development effort and scale of LLM - based software projects.

### Formula example
When discussing the estimation method, the paper does not involve specific mathematical formulas, but its core logic can be represented by the following pseudo - code:

```markdown
Require: List of questions \( Q \)

AllTasks ← ∅

for \( q \in Q \) do
Generate \( N \) related questions \( Q_q \) from \( q \).
\( T_q \leftarrow \text{Planner}(\{q\} \cup Q_q, \text{current tools} = AllTasks, \text{minimize} = \text{True}) \)
for \( t \in T_q \) do
\( t['task'] \leftarrow \text{data source} | \text{algorithm} | \text{UI widget} \)
end for
Manually validate the set \( T_q \).
\( AllTasks \leftarrow AllTasks \cup T_q \)
end for

Manually validate final set of \( AllTasks \)
```

This method not only helps to estimate the development effort more accurately, but also can clearly record the functions that the system will and will not execute, thereby improving the transparency and controllability of the project.