Meaning-Typed Programming: Language-level Abstractions and Runtime for GenAI Applications

Jason Mars,Yiping Kang,Jayanaka L. Dantanarayana,Kugesan Sivasothynathan,Christopher Clarke,Baichuan Li,Krisztian Flautner,Lingjia Tang

2025-01-17

Abstract:Software is rapidly evolving from being programmed with traditional logical code, to neuro-integrated applications that leverage generative AI and large language models (LLMs) for application functionality. This shift increases the complexity of building applications, as developers now must reasoning about, program, and prompt LLMs. Despite efforts to create tools to assist with prompt engineering, these solutions often introduce additional layers of complexity to the development of neuro-integrated applications. This paper proposes meaning-typed programming (MTP), a novel approach to simplify the creation of neuro-integrated applications by introducing new language-level abstractions that hide the complexities of LLM integration. Our key insight is that typical conventional code already possesses a high level of semantic richness that can be automatically reasoned about, as it is designed to be readable and maintainable by humans. Leveraging this insight, we conceptualize LLMs as meaning-typed code constructs and introduce a by abstraction at the language level, MT-IR, a new meaning-based intermediate representation at the compiler level, and MT Runtime, an automated run-time engine for LLM integration and operations. We implement MTP in a production-grade Python super-set language called Jac and perform an extensive evaluation. Our results demonstrate that MTP not only simplifies the development process but also meets or exceeds the efficacy of state-of-the-art manual and tool-assisted prompt engineering techniques in terms of accuracy and usability.

Programming Languages,Artificial Intelligence

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to solve the complexity problems of integrating generative AI and large - language models (LLMs) into traditional programming. Specifically, it focuses on the following points:

1. **Complexity of Prompt Engineering**:
- Developers need to manually construct precise instructions, manage text context, and handle the inherent variability of LLM outputs.
- Prompt engineering lacks the structure, consistency, and debugging capabilities common in traditional programming environments.

2. **Complexity of Input/Output Conversion**:
- Parsing LLM outputs and converting them back into objects in the program is a difficult and error - prone process, especially when dealing with complex object types and the unpredictable output formats of different LLMs.

3. **Maintenance and Adaptability**:
- When using different LLMs (such as GPT, Gemini, Llama), the outputs can vary greatly, which makes it difficult to use general - purpose code for output format conversion.
- The evolution of models also increases the complexity of code maintenance.

To solve these problems, the paper proposes **Meaning - Typed Programming (MTP)**, a new method that simplifies the creation of neural - integrated applications by introducing new language - level abstractions, thus hiding the complexity of LLM integration. The core idea of MTP is to utilize the rich semantic information already existing in traditional code, automatically infer the developer's intentions, and convert these intentions into prompts that LLMs can understand.

### Main Contributions

1. **Proposing the Concept of Semantic - Typed Programming**: Regarding LLMs as semantically - typed code constructs, enabling LLMs to be seamlessly integrated into traditional programs without explicit prompt engineering.
2. **Designing and Implementing the MTP Framework**:
- Introducing the `by` operator for intuitively integrating LLMs.
- Designing MT - IR (Semantic - Typed Intermediate Representation), which collects and organizes semantically - rich information at the compilation level.
- Implementing MT - Runtime, an automated runtime engine for managing LLM operations.
3. **Implementing MTP in the Jac Language**: Jac is a production - level Python superset language. Extensive evaluations have been carried out, and the results show that MTP significantly reduces development complexity, the number of lines of code, and costs, while improving runtime performance.
4. **Providing Empirical Analysis**: Demonstrating how MTP simplifies neural - integrated programming by taking advantage of the progress in LLM capabilities.
5. **Multiple Case Studies**: Demonstrating the application of MTP in complex problem areas, analyzing its impact on code structure, development workflows, and complexity reduction.

Through these contributions, MTP not only simplifies the development process but also achieves or exceeds the effectiveness of existing manual and tool - assisted prompt - engineering techniques in terms of accuracy and usability.