Abstract:Formal program specifications play a crucial role in various stages of software development. However, manually crafting formal program specifications is rather difficult, making the job time-consuming and labor-intensive. It is even more challenging to write specifications that correctly and comprehensively describe the semantics of complex programs. To reduce the burden on software developers, automated specification generation methods have emerged. However, existing methods usually rely on predefined templates or grammar, making them struggle to accurately describe the behavior and functionality of complex real-world programs. To tackle this challenge, we introduce SpecGen, a novel technique for formal program specification generation based on Large Language Models. Our key insight is to overcome the limitations of existing methods by leveraging the code comprehension capability of LLMs. The process of SpecGen consists of two phases. The first phase employs a conversational approach that guides the LLM to generate appropriate specifications for a given program. The second phase, designed for where the LLM fails to generate correct specifications, applies four mutation operators to the model-generated specifications and selects verifiable specifications from the mutated ones through a novel heuristic selection strategy. We evaluate SpecGen on two datasets, including the SV-COMP Java category benchmark and a manually constructed dataset. Experimental results demonstrate that SpecGen succeeds in generating verifiable specifications for 279 out of 385 programs, outperforming the existing purely LLM-based approaches and conventional specification generation tools like Houdini and Daikon. Further investigations on the quality of generated specifications indicate that SpecGen can comprehensively articulate the behaviors of the input program.

DSM: a Specification Mining Tool Using Recurrent Neural Network Based Language Model

Automatic online specification mining

Mining Message Flows using Recurrent Neural Networks for System-on-Chip Designs

Impact of Large Language Models on Generating Software Specifications

Large Language Models Based Automatic Synthesis of Software Specifications

Using a Large Language Model to generate a Design Structure Matrix

ST-ASDNET: A BLSTM-FCN-Transformer basedASD Classification Model for Time-series fMRI

Finding related events for specification mining

Specify What? Enhancing Neural Specification Synthesis by Symbolic Methods

DSMN: A Personalized Information Retrieval Algorithm Based on Improved DSSM.

Can You Explain That, Better? Comprehensible Text Analytics for SE Applications

Enabling Open-World Specification Mining via Unsupervised Learning

SpecGen: Automated Generation of Formal Program Specifications via Large Language Models

A Domain-Specific Modeling Approach Supporting Tool-Chain Development with Bayesian Network Models.

PPTL Specification Mining Based on LNFG

Drug Specification Named Entity Recognition Base on BiLSTM-CRF Model

EEG-SSM: Leveraging State-Space Model for Dementia Detection

SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding

Semantic Feature Learning via Dual Sequences for Defect Prediction

Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders

Research on Mongolian Speech Recognition Based on FSMN.