Abstract:The increasing interest in Large Language Models (LLMs) within the telecommunications sector underscores their potential to revolutionize operational efficiency. However, the deployment of these sophisticated models is often hampered by their substantial size and computational demands, raising concerns about their viability in resource-constrained environments. Addressing this challenge, recent advancements have seen the emergence of small language models that surprisingly exhibit performance comparable to their larger counterparts in many tasks, such as coding and common-sense reasoning. Phi-2, a compact yet powerful model, exemplifies this new wave of efficient small language models. This paper conducts a comprehensive evaluation of Phi-2's intrinsic understanding of the telecommunications domain. Recognizing the scale-related limitations, we enhance Phi-2's capabilities through a Retrieval-Augmented Generation approach, meticulously integrating an extensive knowledge base specifically curated with telecom standard specifications. The enhanced Phi-2 model demonstrates a profound improvement in accuracy, answering questions about telecom standards with a precision that closely rivals the more resource-intensive GPT-3.5. The paper further explores the refined capabilities of Phi-2 in addressing problem-solving scenarios within the telecom sector, highlighting its potential and limitations.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in the telecommunications field, although large - language models (LLMs) have revolutionary potential, their deployment is restricted by model size and computational requirements, especially in resource - constrained environments. Specifically, the paper explores whether small - language models (SLMs) can reduce these resource requirements while maintaining performance. The paper addresses this issue by evaluating the knowledge - understanding ability of a small - language model named Phi - 2 in the telecommunications field and enhancing its performance by introducing Retrieval - Augmented Generation (RAG) technology. The main contributions of the paper include: 1. **Evaluating Phi - 2's telecommunications knowledge**: By using the TeleQnA dataset, a comparative evaluation of the knowledge of Phi - 2, GPT - 3.5, and GPT - 4 in the telecommunications field was carried out. The results show that although Phi - 2 is small in scale, it still performs well on certain tasks. 2. **Application of RAG technology**: By integrating an external knowledge base of telecommunications standard specifications, the performance of Phi - 2 in handling complex problems and knowledge - intensive tasks was significantly improved. Experimental results show that the RAG technology increased the accuracy of Phi - 2 in the "standard specification" category from 44.27% to 56.63%, approaching the level of GPT - 3.5. 3. **Practical application cases**: Through two specific tasks - network modeling and user - association problems - the practical application ability and limitations of Phi - 2 in the telecommunications field were demonstrated. In particular, the RAG - enhanced Phi - 2 showed higher accuracy in constructing energy - consumption models. Overall, this paper aims to explore the feasibility and advantages of small - language models in the telecommunications field and overcome their inherent limitations through technological innovation (such as RAG) so that they can operate effectively in resource - constrained environments.

Telecom Language Models: Must They Be Large?

Large Language Models for Telecom: Forthcoming Impact on the Industry

Tele-LLMs: A Series of Specialized Large Language Models for Telecommunications

Using Large Language Models to Understand Telecom Standards

TelecomGPT: A Framework to Build Telecom-Specfic Large Language Models

Understanding Telecom Language Through Large Language Models

Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities

Design of a large language model for improving customer service in telecom operators

TeleOracle: Fine-Tuned Retrieval-Augmented Generation with Long-Context Support for Network

TelecomRAG: Taming Telecom Standards with Retrieval Augmented Generation and LLMs

Leveraging Fine-Tuned Retrieval-Augmented Generation with Long-Context Support: For 3GPP Standards

Tele-FLM Technical Report

Broadening Access to Simulations for End-Users via Large Language Models: Challenges and Opportunities

Large Language Models aren't all that you need

Large Language Models for Scholarly Ontology Generation: An Extensive Analysis in the Engineering Field

The Quo Vadis of the Relationship between Language and Large Language Models

Language in Vivo vs. in Silico: Size Matters but Larger Language Models Still Do Not Comprehend Language on a Par with Humans

Observations on LLMs for Telecom Domain: Capabilities and Limitations

A Survey of Large Language Models