Granite Code Models: A Family of Open Foundation Models for Code Intelligence

Mayank Mishra,Matt Stallone,Gaoyuan Zhang,Yikang Shen,Aditya Prasad,Adriana Meza Soria,Michele Merler,Parameswaran Selvam,Saptha Surendran,Shivdeep Singh,Manish Sethi,Xuan-Hong Dang,Pengyuan Li,Kun-Lung Wu,Syed Zawad,Andrew Coleman,Matthew White,Mark Lewis,Raju Pavuluri,Yan Koyfman,Boris Lublinsky,Maximilien de Bayser,Ibrahim Abdelaziz,Kinjal Basu,Mayank Agarwal,Yi Zhou,Chris Johnson,Aanchal Goyal,Hima Patel,Yousaf Shah,Petros Zerfos,Heiko Ludwig,Asim Munawar,Maxwell Crouse,Pavan Kapanipathi,Shweta Salaria,Bob Calio,Sophia Wen,Seetharami Seelam,Brian Belgodere,Carlos Fonseca,Amith Singhee,Nirmit Desai,David D. Cox,Ruchir Puri,Rameswar Panda
2024-05-07
Abstract:Large Language Models (LLMs) trained on code are revolutionizing the software development process. Increasingly, code LLMs are being integrated into software development environments to improve the productivity of human programmers, and LLM-based agents are beginning to show promise for handling complex tasks autonomously. Realizing the full potential of code LLMs requires a wide range of capabilities, including code generation, fixing bugs, explaining and documenting code, maintaining repositories, and more. In this work, we introduce the Granite series of decoder-only code models for code generative tasks, trained with code written in 116 programming languages. The Granite Code models family consists of models ranging in size from 3 to 34 billion parameters, suitable for applications ranging from complex application modernization tasks to on-device memory-constrained use cases. Evaluation on a comprehensive set of tasks demonstrates that Granite Code models consistently reaches state-of-the-art performance among available open-source code LLMs. The Granite Code model family was optimized for enterprise software development workflows and performs well across a range of coding tasks (e.g. code generation, fixing and explanation), making it a versatile all around code model. We release all our Granite Code models under an Apache 2.0 license for both research and commercial use.
Artificial Intelligence,Computation and Language,Software Engineering
What problem does this paper attempt to address?
The paper attempts to address the limitations of current large language models (LLMs) in code generation tasks, especially in enterprise software development environments. Specifically, the paper points out the following issues: 1. **Model Size and Deployment Cost**: While very large general-purpose LLMs can achieve excellent coding performance, their size makes deployment costly. 2. **Limitations of Small-Scale Models**: Although smaller code-focused models can achieve excellent code generation performance in smaller packages, their performance in other coding tasks (such as fixing and interpreting code) may not be as good as code generation. 3. **Transparency and Trust Issues**: Even some open models may affect enterprise trust in critical tasks and regulated environments due to the opacity of data sources and processing methods. 4. **Licensing Restrictions**: The licensing terms of existing open LLMs may restrict and complicate enterprise use. To address these issues, the paper introduces the **Granite Code model series**, a set of highly capable code generation models designed to support various coding tasks in enterprise software development. These models have the following features: - **Comprehensive Code-Related Task Capabilities**: Including code generation, interpretation, fixing, editing, translation, etc., demonstrating their ability to handle diverse coding tasks. - **Enterprise-Grade Trustworthiness**: All models are trained on licensed data and adhere to IBM's AI ethics principles, ensuring enterprise-grade trustworthy use. All Granite Code models are released under the Apache 2.0 license, suitable for research and commercial use. Through these improvements, the Granite Code models aim to provide an efficient, reliable, and economical solution for enterprise software development.