ANALYSIS OF NATURAL LANGUAGE TEXTS IN THE TASK OF ASSESSING REQUIREMENTS FOR THE DESIGN OF AIRCRAFT AVIONICS

B. V. Vishnyakov,I. V. Sgibnev,Yu. A. Solodelov,D. V. Krasnoshchekov,K. A. Nikolaev,G. V. Voronkov,S. A. Brianskiy,A. S. Shtraukh,A. D. Vlasenkova
DOI: https://doi.org/10.14489/vkit.2024.04.pp.017-029
2024-04-01
Abstract:The problem of automatic assessment of technical requirements is analyzed. One of the aspects considered is the rationale for the preparation of specific data that should be used in the training of a “transformer” neural network. For this purpose, we compare the software requirements with the design requirements. The criteria for the requirement correctness check are provided, the cases of violation of these criteria are shown. The method that allows to classify requirements according to their compliance and non-compliance with the rules is proposed. Such state-of-the-art transformer architectures as RuBERT, RuBERT-tiny, RuRoBERTa-large, RuDeBERTa, RuDeBERTa-distill are considered. The small dataset of software requirements containing both correct and incorrect examples for the “indivisibility”, “completeness”, “unambiguity” and “correctness” rules to conduct experiments was used. The F1-score to evaluate the algorithms is calculated. Experimental results show that the RuBERT-tiny model demonstrates optimal performance on this small dataset in terms of minimum inference time and the highest value of the F1-score for most rules. High values of the F1-score for the other models were also obtained. It can be concluded that the considered neural network architectures can be used for the tasks of automatic assessment and classification of technical requirements. The use of a “transformer” neural network can potentially improve the quality of preliminary analysis and reduce the time spent on verifying their correctness in the software development process.
What problem does this paper attempt to address?