Abstract:Normative non-functional requirements specify constraints that a system must observe in order to avoid violations of social, legal, ethical, empathetic, and cultural norms. As these requirements are typically defined by non-technical system stakeholders with different expertise and priorities (ethicists, lawyers, social scientists, etc.), ensuring their well-formedness and consistency is very challenging. Recent research has tackled this challenge using a domain-specific language to specify normative requirements as rules whose consistency can then be analysed with formal methods. In this paper, we propose a complementary approach that uses Large Language Models to extract semantic relationships between abstract representations of system capabilities. These relations, which are often assumed implicitly by non-technical stakeholders (e.g., based on common sense or domain knowledge), are then used to enrich the automated reasoning techniques for eliciting and analyzing the consistency of normative requirements. We show the effectiveness of our approach to normative requirements elicitation and operationalization through a range of real-world case studies.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenges in the definition and consistency checking of Normative Non - Functional Requirements (NRs). Specifically, these requirements stipulate the constraints that the system must abide by in order to avoid violating social, legal, ethical, empathetic, and cultural norms. Since these requirements are usually defined by non - technical stakeholders (such as ethicists, lawyers, social scientists, etc.), who have different areas of expertise and priorities, it is very difficult to ensure the well - formedness and consistency of these requirements. ### Core problems of the paper 1. **Definition and consistency of normative requirements**: - **Problem background**: Software systems are increasingly becoming an important part of our infrastructure and performing daily tasks. Therefore, ensuring the responsible integration of these systems has become crucial, including preventing them from violating social, legal, ethical, empathetic, and cultural (SLEEC) norms. - **Specific challenges**: Normative requirements (NRs) are usually defined by non - technical stakeholders, and these stakeholders may have different terms, goals, priorities, and responsibilities. Therefore, it is very challenging to ensure that these requirements have no formal problems (such as conflicts and redundancies). 2. **Limitations of existing methods**: - **Formal languages**: Previous research has proposed a specific - domain language (SLEEC) to formalize normative requirements and use formal methods to analyze the consistency of these requirements. However, these methods assume that all referenced capabilities are independent, ignoring the implicit relationships between many capabilities in practice. - **Lack of semantic relationships**: Existing automated reasoning techniques cannot capture the semantic relationships between events and measures in the rules, which may lead to inaccurate analysis. ### Solutions in the paper 1. **Utilizing large - language models (LLMs)**: - **Extracting semantic relationships**: The paper proposes a method of using large - language models (LLMs) to extract semantic relationships from the abstract representation of system capabilities. These relationships are usually implicitly assumed by non - technical stakeholders based on common sense or domain knowledge. - **Enhancing automated reasoning**: The extracted semantic relationships are used to enrich automated reasoning techniques to better identify and analyze the consistency problems of normative requirements. 2. **Light - weight logical filtering algorithm**: - **Preventing mis - identification**: In order to prevent the wrong semantic relationships generated by LLMs from affecting the consistency analysis, the paper also proposes a light - weight logical filtering algorithm, which real - time repairs logical inconsistencies through a set of reasoning rules and heuristics. - **Manual review**: Ultimately, stakeholders still need to further screen these relationships according to their domain knowledge. ### Main contributions 1. **LLM - based technology**: Proposed a technology based on LLMs for extracting semantic relationships between system capabilities. 2. **RAINCOAT method**: Combined the new semantic relationship extraction technology with the existing automated reasoning methods to develop a systematic end - to - end method RAINCOAT for the collection and analysis of normative requirements. 3. **Effectiveness evaluation**: Through multiple real - world case studies, the effectiveness and usability of the LLM - based technology and the RAINCOAT method were extensively evaluated. ### Significance The paper introduces an innovative method RAINCOAT, which captures the implicit semantic relationships that stakeholders inherently understand but often fail to clearly express during the requirement - collection process by using LLMs. This method not only helps bridge the gap between the explicit formalization of normative requirements and the implicit understanding of non - technical stakeholders, but also can coexist harmoniously with existing automated reasoning techniques, thereby analyzing normative requirements more meticulously and identifying potential conflicts and redundancies.

Normative Requirements Operationalization with Large Language Models

Agile Methodology for the Standardization of Engineering Requirements Using Large Language Models

nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Models

Using Large Language Models for Natural Language Processing Tasks in Requirements Engineering: A Systematic Guideline

Norm Violation Detection in Multi-Agent Systems using Large Language Models: A Pilot Study

Engineering Safety Requirements for Autonomous Driving with Large Language Models

Prioritizing Software Requirements Using Large Language Models

Requirements Engineering and Large Language Models: Insights From a Panel

A Software Engineering Perspective on Testing Large Language Models: Research, Practice, Tools and Benchmarks

Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding

Large Language Models as Fiduciaries: A Case Study Toward Robustly Communicating With Artificial Intelligence Through Legal Standards

Natural Language Processing for Requirements Formalization: How to Derive New Approaches?

Harnessing the power of LLMs for normative reasoning in MASs

Shaping the Emerging Norms of Using Large Language Models in Social Computing Research

Eliciting Problem Specifications via Large Language Models

Impact of Large Language Models on Generating Software Specifications

Foundational Challenges in Assuring Alignment and Safety of Large Language Models

Large Human Language Models: A Need and the Challenges

Are We Testing or Being Tested? Exploring the Practical Applications of Large Language Models in Software Testing

A collection of principles for guiding and evaluating large language models

Large Language Models for Software Engineering: Survey and Open Problems