Abstract:By virtue of being prevalently written in natural language (NL), requirements are prone to various defects, e.g., inconsistency and incompleteness. As such, requirements are frequently subject to quality assurance processes. These processes, when carried out entirely manually, are tedious and may further overlook important quality issues due to time and budget pressures. In this paper, we propose QAssist -- a question-answering (QA) approach that provides automated assistance to stakeholders, including requirements engineers, during the analysis of NL requirements. Posing a question and getting an instant answer is beneficial in various quality-assurance scenarios, e.g., incompleteness detection. Answering requirements-related questions automatically is challenging since the scope of the search for answers can go beyond the given requirements specification. To that end, QAssist provides support for mining external domain-knowledge resources. Our work is one of the first initiatives to bring together QA and external domain knowledge for addressing requirements engineering challenges. We evaluate QAssist on a dataset covering three application domains and containing a total of 387 question-answer pairs. We experiment with state-of-the-art QA methods, based primarily on recent large-scale language models. In our empirical study, QAssist localizes the answer to a question to three passages within the requirements specification and within the external domain-knowledge resource with an average recall of 90.1% and 96.5%, respectively. QAssist extracts the actual answer to the posed question with an average accuracy of 84.2%. Keywords: Natural-language Requirements, Question Answering (QA), Language Models, Natural Language Processing (NLP), Natural Language Generation (NLG), BERT, T5.

Natural Test Generation for Precise Testing of Question Answering Software.

QATest: A Uniform Fuzzing Framework for Question Answering Systems.

Knowledge Graph Driven Inference Testing for Question Answering Software

Improve the Quality of ARC Systems Based on the Metamorphic Testing

Automated Testing for Text-to-Image Software

Quality Assurance of Bioinformatics Software: A Case Study of Testing a Biomedical Text Processing Tool Using Metamorphic Testing

Syn-QA2: Evaluating False Assumptions in Long-tail Questions with Synthetic QA Datasets

Supporting maintenance and testing for AI functions of mobile apps based on user reviews: An empirical study on plant identification apps

(QA)$^2$: Question Answering with Questionable Assumptions

Can ChatGPT advance software testing intelligence? An experience report on metamorphic testing

ComQA: Question Answering over Knowledge Base Via Semantic Matching.

QADYNAMICS: Training Dynamics-Driven Synthetic QA Diagnostic for Zero-Shot Commonsense Question Answering

Automatic Test Generation for Mutation Testing on Database Applications.

Analysis of QA System Behavior against Context and Question Changes

AI-based Question Answering Assistance for Analyzing Natural-language Requirements

Automatic Question Generation for Repeated Testing to Improve Student Learning Outcome

On the Generation of Medical Question-Answer Pairs

Improving the Quality of Computational Science Software by Using Metamorphic Relations to Test Machine Learning Applications

Automatic True/False Question Generation for Educational Purpose

Intergenerational Test Generation for Natural Language Processing Applications