Test Code Generation for Telecom Software Systems using Two-Stage Generative Model

Mohamad Nabeel,Doumitrou Daniil Nimara,Tahar Zanouda

2024-04-14

Abstract:In recent years, the evolution of Telecom towards achieving intelligent, autonomous, and open networks has led to an increasingly complex Telecom Software system, supporting various heterogeneous deployment scenarios, with multi-standard and multi-vendor support. As a result, it becomes a challenge for large-scale Telecom software companies to develop and test software for all deployment scenarios. To address these challenges, we propose a framework for Automated Test Generation for large-scale Telecom Software systems. We begin by generating Test Case Input data for test scenarios observed using a time-series Generative model trained on historical Telecom Network data during field trials. Additionally, the time-series Generative model helps in preserving the privacy of Telecom data. The generated time-series software performance data are then utilized with test descriptions written in natural language to generate Test Script using the Generative Large Language Model. Our comprehensive experiments on public datasets and Telecom datasets obtained from operational Telecom Networks demonstrate that the framework can effectively generate comprehensive test case data input and useful test code.

Software Engineering,Computation and Language,Machine Learning

What problem does this paper attempt to address?

This paper attempts to solve several key problems encountered in the testing of telecommunication software systems:

1. **Complexity of test scenarios**: With the development of 5G network technology, the increase in the number of radio nodes, and the introduction of Open Radio Access Network (O - RAN), telecommunication networks have become increasingly complex and support multiple standards and multi - vendor deployments. This makes it difficult for large - scale telecommunication software companies to develop and test software applicable to all deployment scenarios.

2. **Limitations of manual testing**: Traditional software testing relies on manually defining and executing test cases, which is not only time - consuming and labor - intensive but also requires in - depth technical expertise. As the network continues to evolve, new failures may occur in uncovered scenarios, and it is difficult for manual testing to comprehensively cover these newly emerging problems.

3. **Challenges of multi - vendor interoperability**: In O - RAN networks, products from multiple vendors need to be supported, which places higher requirements on testing to ensure that devices from different vendors can work together.

To address the above challenges, the paper proposes an automated test script generation framework based on a two - stage generation model. Specifically:

- **First stage**: Use time - series generation models (such as diffusion models and TimeGAN) to generate synthetic test input data from historical telecommunication network data. These models can capture the distribution of the original data and generate meaningful test inputs, even if these input scenarios have not occurred in the actual data.

- **Second stage**: Combine the generated test input data with test scenarios described in natural language and use large - language models (LLM) to generate test scripts. This process not only reduces the test script generation time but also improves the test coverage and efficiency.

Through this method, the paper aims to improve the testing efficiency and quality of telecommunication software systems, especially in the face of complex and changeable network environments.