Abstract:Natural Language Processing for Requirements Engineering (NLP4RE) is an area of research and development that seeks to apply natural language processing (NLP) techniques, tools, and resources to the requirements engineering (RE) process, to support human analysts to carry out various linguistic analysis tasks on textual requirements documents, such as detecting language issues, identifying key domain concepts, and establishing requirements traceability links. This article reports on a mapping study that surveys the landscape of NLP4RE research to provide a holistic understanding of the field. Following the guidance of systematic review, the mapping study is directed by five research questions, cutting across five aspects of NLP4RE research, concerning the state of the literature, the state of empirical research, the research focus, the state of tool development, and the usage of NLP technologies. Our main results are as follows: (i) we identify a total of 404 primary studies relevant to NLP4RE, which were published over the past 36 years and from 170 different venues; (ii) most of these studies (67.08%) are solution proposals, assessed by a laboratory experiment or an example application, while only a small percentage (7%) are assessed in industrial settings; (iii) a large proportion of the studies (42.70%) focus on the requirements analysis phase, with quality defect detection as their central task and requirements specification as their commonly processed document type; (iv) 130 NLP4RE tools (i.e., RE specific NLP tools) are extracted from these studies, but only 17 of them (13.08%) are available for download; (v) 231 different NLP technologies are also identified, comprising 140 NLP techniques, 66 NLP tools, and 25 NLP resources, but most of them—particularly those novel NLP techniques and specialized tools—are used infrequently; by contrast, commonly used NLP technologies are traditional analysis techniques (e.g., POS tagging and tokenization), general-purpose tools (e.g., Stanford CoreNLP and GATE) and generic language lexicons (WordNet and British National Corpus). The mapping study not only provides a collection of the literature in NLP4RE but also, more importantly, establishes a structure to frame the existing literature through categorization, synthesis and conceptualization of the main theoretical concepts and relationships that encompass both RE and NLP aspects. Our work thus produces a conceptual framework of NLP4RE. The framework is used to identify research gaps and directions, highlight technology transfer needs, and encourage more synergies between the RE community, the NLP one, and the software and systems practitioners. Our results can be used as a starting point to frame future studies according to a well-defined terminology and can be expanded as new technologies and novel solutions emerge.

NLPIR: A theoretical framework for applying natural language processing to information retrieval

A Survey of Natural Language Processing in Information Retrieval

Development and Evaluation of Task-Specific NLP Framework in China.

Progress in the Application of Natural Language Processing to Information Retrieval Tasks

Using NLP or NLP Resources for Information Retrieval Tasks

A natural language interface to a graph-based bibliographic information retrieval system

Some Ideas on Natural Language Processing

Interactive Natural Language Processing

New Approaches for Natural Language Understanding based on the Idea that Natural Language encodes both Information and its Processing Procedures

Synergistic Interplay between Search and Large Language Models for Information Retrieval

Neural Information Retrieval: A Literature Review

An Intelligent Platform For Information Retrieval

Tutorial: Modern Theoretical Tools for Understanding and Designing Next-generation Information Retrieval System

Natural Language Processing for Requirements Engineering

Large Language Models for Information Retrieval: A Survey

IR Design for Application-Specific Natural Language: A Case Study on Traffic Data

Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community

A proposed conceptual framework for a representational approach to information retrieval

Natural language processing applications in library and information science

Deep Reinforcement Learning for Information Retrieval: Fundamentals and Advances