Abstract:The fifth-generation (5G) offers advanced services, supporting applications such as intelligent transportation, connected healthcare, and smart cities within the Internet of Things (IoT). However, these advancements introduce significant security challenges, with increasingly sophisticated cyber-attacks. This paper proposes a robust intrusion detection system (IDS) using federated learning and large language models (LLMs). The core of our IDS is based on BERT, a transformer model adapted to identify malicious network flows. We modified this transformer to optimize performance on edge devices with limited resources. Experiments were conducted in both centralized and federated learning contexts. In the centralized setup, the model achieved an inference accuracy of 97.79%. In a federated learning context, the model was trained across multiple devices using both IID (Independent and Identically Distributed) and non-IID data, based on various scenarios, ensuring data privacy and compliance with regulations. We also leveraged linear quantization to compress the model for deployment on edge devices. This reduction resulted in a slight decrease of 0.02% in accuracy for a model size reduction of 28.74%. The results underscore the viability of LLMs for deployment in IoT ecosystems, highlighting their ability to operate on devices with constrained computational and storage resources.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the increasingly complex cybersecurity challenges in the 5G ecosystem, especially the advanced security threats in Internet of Things (IoT) applications such as intelligent transportation, connected healthcare, and smart cities. Specifically, the paper aims to develop a robust intrusion detection system (IDS) to deal with the complex and evolving network attacks in 5G networks and ensure efficient deployment on resource - constrained edge devices. ### Overview of Main Problems 1. **Security Challenges in 5G Networks**: - 5G networks support a wider range of applications and services, such as intelligent transportation, connected healthcare, and smart cities, which introduce new security risks. - Network attacks are becoming more and more complex and personalized, and traditional firewalls and other security measures are difficult to deal with effectively. 2. **Limitations of Existing Intrusion Detection Systems**: - Existing intrusion detection systems (IDS) mainly rely on signature detection (SIDS) or anomaly detection. The former depends on predefined rules, and the latter depends on learning normal traffic. - These methods perform poorly in the face of unknown attacks, especially in a heterogeneous and dense environment like 5G. 3. **Requirement for Privacy Protection**: - In the 5G environment, the use of personal data increases the risk of privacy leakage. Therefore, a method that can perform model training while protecting data privacy is required. ### Solutions To solve the above problems, the paper proposes an intrusion detection system based on federated learning (Federated Learning, FL) and large - language models (LLMs). The core of this system is an optimized BERT model, which is adjusted to adapt to the resource limitations of edge devices. The specific contributions are as follows: 1. **Efficient Federated Intrusion Detection System**: - Use federated learning to collaboratively train the model on multiple devices while keeping the data local, thereby protecting user privacy. - The model performs well in both centralized and federated learning environments. The accuracy rate reaches 97.79% in the centralized environment, and also reaches a relatively high accuracy rate in the federated environment (IID and non - IID data). 2. **Optimized BERT Model**: - By reducing the number of model layers and applying linear quantization, the model size is significantly reduced, making it suitable for deployment on resource - constrained edge devices. - The model size is reduced by 89.85%, and further compressed by 92.76%, but only slightly affects the accuracy (0.02%). 3. **Handling Non - Independent and Identically Distributed (non - IID) Data**: - The paper explores the model convergence under different data distributions (IID vs. non - IID), and finds that more client participation and longer local training time help improve model performance, and can even reach an accuracy rate close to 97% under non - IID data. ### Summary This paper provides an intrusion detection solution that is both efficient and privacy - protecting by combining federated learning and an optimized BERT model, especially suitable for the complex environment of the 5G ecosystem. This research not only shows the potential of LLMs in the field of cybersecurity but also emphasizes its practical application value on resource - constrained devices.

Efficient Federated Intrusion Detection in 5G ecosystem using optimized BERT-based model

Revolutionizing Cyber Threat Detection With Large Language Models: A Privacy-Preserving BERT-Based Lightweight Model for IoT/IIoT Devices

Federated Learning for 5G and Beyond, a Blessing and a Curse- An Experimental Study on Intrusion Detection Systems

A Federated Learning-Based Approach for Improving Intrusion Detection in Industrial Internet of Things Networks

Beyond Detection: Leveraging Large Language Models for Cyber Attack Prediction in IoT Networks

Federated learning for malware detection in IoT devices

Federated Deep Learning for Intrusion Detection in IoT Networks

FedMSE: Federated learning for IoT network intrusion detection

Effective Intrusion Detection in Heterogeneous Internet-of-Things Networks via Ensemble Knowledge Distillation-based Federated Learning

Securing 5G/6G IoT Using Transformer and Personalized Federated Learning: an Access-Side Distributed Malicious Traffic Detection Framework

A cognitive security framework for detecting intrusions in IoT and 5G utilizing deep learning

Enhancing Intrusion Detection In Internet Of Vehicles Through Federated Learning

Federated transfer learning for attack detection for Internet of Medical Things

FL-IDS: Federated Learning-Based Intrusion Detection System Using Edge Devices for Transportation IoT

Enhancing IoT Security Against DDoS Attacks through Federated Learning

Improved Intrusion Detection Based on Hybrid Deep Learning Models and Federated Learning

Privacy-Preserving Intrusion Detection in Software-defined VANET using Federated Learning with BERT

Deep Learning-Inspired IoT-IDS Mechanism for Edge Computing Environments

NGMD: next generation malware detection in federated server with deep neural network model for autonomous networks

A review of Federated Learning in Intrusion Detection Systems for IoT

Privacy-Preserving Federated Learning-Based Intrusion Detection Technique for Cyber-Physical Systems