Abstract:Natural Language Processing for Requirements Engineering (NLP4RE) is an area of research and development that seeks to apply natural language processing (NLP) techniques, tools, and resources to the requirements engineering (RE) process, to support human analysts to carry out various linguistic analysis tasks on textual requirements documents, such as detecting language issues, identifying key domain concepts, and establishing requirements traceability links. This article reports on a mapping study that surveys the landscape of NLP4RE research to provide a holistic understanding of the field. Following the guidance of systematic review, the mapping study is directed by five research questions, cutting across five aspects of NLP4RE research, concerning the state of the literature, the state of empirical research, the research focus, the state of tool development, and the usage of NLP technologies. Our main results are as follows: (i) we identify a total of 404 primary studies relevant to NLP4RE, which were published over the past 36 years and from 170 different venues; (ii) most of these studies (67.08%) are solution proposals, assessed by a laboratory experiment or an example application, while only a small percentage (7%) are assessed in industrial settings; (iii) a large proportion of the studies (42.70%) focus on the requirements analysis phase, with quality defect detection as their central task and requirements specification as their commonly processed document type; (iv) 130 NLP4RE tools (i.e., RE specific NLP tools) are extracted from these studies, but only 17 of them (13.08%) are available for download; (v) 231 different NLP technologies are also identified, comprising 140 NLP techniques, 66 NLP tools, and 25 NLP resources, but most of them—particularly those novel NLP techniques and specialized tools—are used infrequently; by contrast, commonly used NLP technologies are traditional analysis techniques (e.g., POS tagging and tokenization), general-purpose tools (e.g., Stanford CoreNLP and GATE) and generic language lexicons (WordNet and British National Corpus). The mapping study not only provides a collection of the literature in NLP4RE but also, more importantly, establishes a structure to frame the existing literature through categorization, synthesis and conceptualization of the main theoretical concepts and relationships that encompass both RE and NLP aspects. Our work thus produces a conceptual framework of NLP4RE. The framework is used to identify research gaps and directions, highlight technology transfer needs, and encourage more synergies between the RE community, the NLP one, and the software and systems practitioners. Our results can be used as a starting point to frame future studies according to a well-defined terminology and can be expanded as new technologies and novel solutions emerge.

A Survey of Natural Language Processing in Information Retrieval

Progress in the Application of Natural Language Processing to Information Retrieval Tasks

NLPIR: A theoretical framework for applying natural language processing to information retrieval

Using NLP or NLP Resources for Information Retrieval Tasks

A Survey on Natural Language Processing for Programming

Large Language Models for Information Retrieval: A Survey

Research on Information Extraction:A Survey

Pruning dependency trees in a syntactic information retrieval model

A Survey of Document-Level Information Extraction

A Review of Advanced Topics in Information Retrieval

Application of Natural Language Processing Technology in Educational Resources Retrieval

Some Ideas on Natural Language Processing

Information Retrieval and Question Answering NLP Approach: An Artificial Intelligence Application

Interactive Natural Language Processing

Natural Language Processing Technologies in Radiology Research and Clinical Applications.

Neural Information Retrieval: A Literature Review

A Survey (NLP) Natural Language Processing and Transactions on (NNL) Neural Networks and learning Systems

Chinese Natural Language Processing: From Text Categorization to Machine Translation

Natural Language Processing for Requirements Engineering

A Deep Look into neural ranking models for information retrieval