Online Anomaly Detection Based On Reservoir Sampling and LOF for IoT devices

Tomasz Szydlo

DOI: https://doi.org/10.48550/arXiv.2206.14265

2022-06-29

Abstract:The growing number of IoT devices and their use to monitor the operation of machines and equipment increases interest in anomaly detection algorithms running on devices. However, the difficulty is the limitations of the available computational and memory resources on the devices. In the case of microcontrollers (MCUs), these are single megabytes of program and several hundred kilobytes of working memory. Consequently, algorithms must be appropriately matched to the capabilities of the devices. In the paper, we analyse the processing pipeline for anomaly detection and implementation of the Local Outliner Factor (LOF) algorithm on a MCU. We also show that it is possible to train such an algorithm directly on the device, which gives great potential to use the solution in real devices.

Machine Learning

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are: implementing real - time anomaly detection algorithms on resource - constrained Internet of Things (IoT) devices, especially in the case of limited computing and memory resources of microcontroller units (MCUs). Specifically, the researchers are faced with two main research questions: 1. **What types of anomaly detection algorithms can be implemented on MCUs**: - Since the computing and memory resources of MCUs are very limited (for example, the program memory is only a few megabytes and the working memory is only a few hundred kilobytes), traditional anomaly detection algorithms may not be directly applicable to these devices. - Therefore, it is necessary to find anomaly detection algorithms suitable for MCUs and ensure that they can operate effectively in these resource - constrained environments. 2. **How to directly train these algorithms on the device and collect device profiles for correct operation**: - Training anomaly detection algorithms usually requires a large amount of data and computing resources, but in practical applications, the device may need to be adjusted according to the specific deployment environment. - Therefore, the researchers explored how to directly train anomaly detection algorithms on the device to adapt to specific application scenarios and device characteristics. To solve these problems, the paper proposes an online anomaly detection method based on Reservoir Sampling and Local Outlier Factor (LOF) algorithm. This method achieves effective use of resources in the following ways: - **Reservoir Sampling**: It is used to select a fixed - size sample set from the data stream, thereby reducing the amount of data that needs to be stored, enabling the LOF algorithm to run on MCUs. - **LOF algorithm**: An unsupervised anomaly detection method that can calculate the local density deviation of data points and compare them with neighboring points, thereby identifying abnormal points. In this way, the paper demonstrates the possibility of achieving efficient anomaly detection on resource - constrained MCUs and proves that these algorithms can be directly trained on the device, which has broad practical application potential.

Online Anomaly Detection Based On Reservoir Sampling and LOF for IoT devices

LSTM-Markov based efficient anomaly detection algorithm for IoT environment

Deep Learning for Anomaly Detection in IoT Systems: Techniques, Applications, and Future Directions

Anomaly Detection IDS for Detecting DoS Attacks in IoT Networks Based on Machine Learning Algorithms

Analysis of Machine Learning Algorithms for Anomaly Detection on Edge Devices

Anomaly detection framework for IoT-enabled appliances using machine learning

Efficient Approach for Anomaly Detection in IoT Using System Calls

System Statistics Learning-Based IoT Security: Feasibility and Suitability

ADRIoT: an Edge-Assisted Anomaly Detection Framework Against IoT-Based Network Attacks

A data-driven approach for intrusion and anomaly detection using automated machine learning for the Internet of Things

Software Defined Machine Learning Based Anomaly Detection in Fog Based IoT Network

AnoML-IoT: An End to End Re-configurable Multi-protocol Anomaly Detection Pipeline for Internet of Things

Anomaly Detection for IOT/Cloud-Based Model in Fog Computing Using Machine Learning

Leveraging Machine Learning for Real-time Anomaly Detection and Self-Repair in IoT Devices

Lightweight Collaborative Anomaly Detection for the IoT using Blockchain

DSORT-MCU: Detecting Small Objects in Real-Time on Microcontroller Units

Anomaly Detection in IoT Networks: Machine Learning Approaches for Intrusion Detection

A Survey of AI-Based Anomaly Detection in IoT and Sensor Networks

Enhancing IoT Security: Optimizing Anomaly Detection through Machine Learning

Anomaly Detection Fog (ADF): A federated approach for internet of things

Anomaly detection in IoT-based healthcare: machine learning for enhanced security