On-line Anomaly Detection and Qualification of Random Bit Streams

Cesare Caratozzolo,Valeria Rossi,Kamil Witek,Alberto Trombetta,Massimo Caccia
2024-09-19
Abstract:Generating random bit streams is required in various applications, most notably cyber-security. Ensuring high-quality and robust randomness is crucial to mitigate risks associated with predictability and system compromise. True random numbers provide the highest unpredictability levels. However, potential biases in the processes exploited for the random number generation must be carefully monitored. This paper reports the implementation and characterization of an on-line procedure for the detection of anomalies in a true random bit stream. It is based on the NIST Adaptive Proportion and Repetition Count tests, complemented by statistical analysis relying on the Monobit and RUNS. The procedure is firmware implemented and performed simultaneously with the bit stream generation, and providing as well an estimate of the entropy of the source. The experimental validation of the approach is performed upon the bit streams generated by a quantum, silicon-based entropy source.
Cryptography and Security
What problem does this paper attempt to address?
### What problems does this paper attempt to solve? This paper aims to solve the problem of ensuring high - quality and robustness when generating random bitstreams, especially in applications such as network security. Specifically, the paper focuses on how to detect and evaluate anomalies in truly random bitstreams online. The following are the main problems and solutions in the paper: 1. **Ensuring the generation of high - quality random numbers**: - High - quality random numbers are crucial for reducing risks related to predictability and system vulnerabilities. True Random Numbers (TRNs) provide the highest level of unpredictability. - However, the processes used to generate random numbers may have biases, and these biases need to be carefully monitored. 2. **Online detection and evaluation of the quality of random bitstreams**: - The paper proposes a method based on the NIST Adaptive Proportion Test (APT) and Repetition Count Test (RCT), combined with the Monobit Test and RUNS Test for statistical analysis. - These tests are implemented in FPGA firmware and can be executed in real - time while the bitstream is being generated without affecting the generation rate and provide an estimate of the entropy source. 3. **Experimental verification**: - This method has been experimentally verified using bitstreams generated by silicon - based quantum entropy sources, ensuring the effectiveness and reliability of the method. ### Specific problem description - **Challenges in proving randomness**: Traditional statistical tests such as the NIST test suite and TestU01, although providing detailed entropy evaluations, require a large amount of bit data and have high computational complexity, and are not suitable for online quality evaluation. - **Known security vulnerabilities**: For example, the low - entropy problem in the Debian Linux OpenSSL version in 2008, and the significant entropy loss in Internet of Things devices due to the slow generation speed of random bits. These problems indicate the importance of online randomness quality evaluation methods. ### Contributions of the paper - **Proposing an online anomaly detection procedure**: Using RCT and APT in NIST health tests and combining Monobit and RUNS statistical results, a new method is proposed to estimate the lower limit of the source entropy. - **FPGA implementation**: Ensure online execution without affecting the bit generation rate. - **Experimental verification**: Experimental verification is carried out using bitstreams generated by silicon - based quantum random number generators (QRNGs), demonstrating the effectiveness of the method. ### Conclusion The paper has developed a statistical online implementation method that can detect catastrophic failures of entropy sources by monitoring changes in the mean during the production process, and can effectively identify even in the case of bias at the single - bit level. This method only requires a relatively small number of bits and can identify bias at a 99.7% confidence level. In addition, by introducing ISN (Inter - Sequence Number) and its distribution analysis, the hypothesis is further verified. Symbol - level tests (RCT and APT) are also used to estimate the entropy of the bitstream, and the test results of four different QRNG boards are consistent, providing a lower - limit estimate of the unbiased bitstream. Future work may include evaluating the frequency of occurrence of each symbol in the alphabet in APT, which is very important for estimating the minimum entropy.