Abstract:Large language models can produce creative and diverse responses. However, to integrate them into current developer workflows, it is essential to constrain their outputs to follow specific formats or standards. In this work, we surveyed 51 experienced industry professionals to understand the range of scenarios and motivations driving the need for output constraints from a user-centered perspective. We identified 134 concrete use cases for constraints at two levels: low-level, which ensures the output adhere to a structured format and an appropriate length, and high-level, which requires the output to follow semantic and stylistic guidelines without hallucination. Critically, applying output constraints could not only streamline the currently repetitive process of developing, testing, and integrating LLM prompts for developers, but also enhance the user experience of LLM-powered features and applications. We conclude with a discussion on user preferences and needs towards articulating intended constraints for LLMs, alongside an initial design for a constraint prototyping tool.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to explore how to impose constraints on the outputs of large language models (LLMs) from a user - centered perspective. Specifically, by surveying 51 experienced industry professionals, the paper has understood what types of output constraints users need and the reasons behind them in the actual development and use of LLMs. The paper mainly solves the following problems: 1. **RQ1: Which practical application scenarios need or can benefit from imposing constraints on LLM outputs?** - Through the analysis of 134 real - use cases, the paper has identified six main categories of output constraints: low - level constraints (such as structured output, valid JSON objects, multiple - choice, length limits) and high - level constraints (such as semantic constraints, style constraints, preventing hallucinations, following instructions). These constraints help ensure that LLM outputs meet specific format, length, semantic and style requirements. 2. **RQ2: What are the benefits and impacts of imposing constraints on LLM outputs?** - Benefits for developers include improving prompt - based development efficiency, reducing the number of trial - and - error attempts, simplifying post - processing logic, and reducing overall inference cost and latency. - Benefits for users include meeting product and UI requirements, enhancing user experience and trust, improving data quality and integrity, ensuring code executability, and maintaining output consistency and formatting. 3. **RQ3: How do users want to express their constraints on LLM outputs?** - Survey results show that users prefer to use graphical user interfaces (GUIs) to specify low - level constraints because this method is faster, more reliable and more flexible. High - level constraints are more often expressed through natural language because these constraints are often more subjective and complex. ### Main contributions of the paper - **Comprehensive classification system**: Summarized the low - level and high - level output constraints required by LLM users, which are derived from 134 real - use cases. - **Benefits for developers and users**: Elaborated on the multi - faceted benefits of imposing constraints for developers and users. - **Exploration of user preferences**: Explored user preferences when expressing constraints, whether through GUIs or natural language. - **ConstraintMaker prototype tool**: Designed an early tool prototype that enables users to visually prototype and test LLM output constraints, with preliminary user feedback. ### Conclusion Through systematic investigation and analysis, the paper has revealed the importance and necessity of imposing constraints on LLM outputs in practical applications. This not only helps improve development efficiency and user experience, but also enhances user trust and satisfaction in the LLM system. Meanwhile, the design tools and methods proposed in the paper provide valuable references for future research and applications.

"We Need Structured Output": Towards User-centered Constraints on Large Language Model Output

Evaluating, Understanding, and Improving Constrained Text Generation for Large Language Models

A Comprehensive Evaluation of Constrained Text Generation for Large Language Models.

Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models

Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data?

Let Me Speak Freely? A Study on the Impact of Format Restrictions on Large Language Model Performance.

Automata-based constraints for language model decoding

Optimizing Large Language Models for Dynamic Constraints through Human-in-the-Loop Discriminators

ConCodeEval: Evaluating Large Language Models for Code Constraints in Domain-Specific Languages

Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation

Open-domain Implicit Format Control for Large Language Model Generation

A Simple but Effective Approach to Improve Structured Language Model Output for Information Extraction

Supporting Sensemaking of Large Language Model Outputs at Scale

Exploring the Capabilities of Large Language Models for Generating Diverse Design Solutions

StrucText-Eval: Evaluating Large Language Model's Reasoning Ability in Structure-Rich Text

The Ability of Large Language Models to Evaluate Constraint-satisfaction in Agent Responses to Open-ended Requests

Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study

Strategic Optimization and Challenges of Large Language Models in Object-Oriented Programming

CFBench: A Comprehensive Constraints-Following Benchmark for LLMs

Why is constrained neural language generation particularly challenging?