Abstract:Compliance checking is an essential part of a construction project. The recent rapid uptake of building information models (BIM) in the construction industry has created more opportunities for automated compliance checking (ACC). BIM enables sharing of digital building design data that can be used for compliance checking with legal requirements, which are conventionally conveyed in natural language and not intended for machine processing. Creating a computable representation of legal requirements suitable for ACC is complex, costly, and time-consuming. Large language models (LLMs) such as the generative pre-trained transformers (GPT), GPT-3.5 and GPT-4, powering OpenAI's ChatGPT, can generate logically coherent text and source code responding to user prompts. This capability could be used to automate the conversion of building regulations into a semantic and computable representation. This paper evaluates the performance of LLMs in translating building regulations into LegalRuleML in a few-shot learning setup. By providing GPT-3.5 with only a few example translations, it can learn the basic structure of the format. Using a system prompt, we further specify the LegalRuleML representation and explore the existence of expert domain knowledge in the model. Such domain knowledge might be ingrained in GPT-3.5 through the broad pre-training but needs to be brought forth by careful contextualisation. Finally, we investigate whether strategies such as chain-of-thought reasoning and self-consistency could apply to this use case. As LLMs become more sophisticated, the increased common sense, logical coherence, and means to domain adaptation can significantly support ACC, leading to more efficient and effective checking processes.

Rule Extrapolation in Language Models: A Study of Compositional Generalization on OOD Prompts

Rule Extrapolation in Language Models: A Study of Compositional Generalization on OOD Prompts

Out-of-distribution generalization via composition: a lens through induction heads in Transformers

Distilling Task-specific Logical Rules from Large Pre-trained Models

It Ain't That Bad: Understanding the Mysterious Performance Drop in OOD Generalization for Generative Transformer Models

Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs

Enabling Large Language Models to Learn from Rules

Distilling Rule-based Knowledge into Large Language Models

Rulebreakers Challenge: Revealing a Blind Spot in Large Language Models' Reasoning with Formal Logic

Look Before You Leap: A Universal Emergent Decomposition of Retrieval Tasks in Language Models

Abstract Rule Learning for Paraphrase Generation

A Theory of Emergent In-Context Learning as Implicit Structure Induction

Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability

RuleR: Improving LLM Controllability by Rule-based Data Recycling

Distributed Rule Vectors is A Key Mechanism in Large Language Models' In-Context Learning

Chain of Logic: Rule-Based Reasoning with Large Language Models

Position Paper: Generalized grammar rules and structure-based generalization beyond classical equivariance for lexical tasks and transduction

Faith and Fate: Limits of Transformers on Compositionality

Learning Rules from KGs Guided by Language Models

Using Large Language Models for the Interpretation of Building Regulations

Large Language Models are Good Multi-lingual Learners : When LLMs Meet Cross-lingual Prompts