ChipExpert: The Open-Source Integrated-Circuit-Design-Specific Large Language Model

Ning Xu,Zhaoyang Zhang,Lei Qi,Wensuo Wang,Chao Zhang,Zihao Ren,Huaiyuan Zhang,Xin Cheng,Yanqi Zhang,Zhichao Liu,Qingwen Wei,Shiyang Wu,Lanlan Yang,Qianfeng Lu,Yiqun Ma,Mengyao Zhao,Junbo Liu,Yufan Song,Xin Geng,Jun Yang
2024-07-26
Abstract:The field of integrated circuit (IC) design is highly specialized, presenting significant barriers to entry and research and development challenges. Although large language models (LLMs) have achieved remarkable success in various domains, existing LLMs often fail to meet the specific needs of students, engineers, and researchers. Consequently, the potential of LLMs in the IC design domain remains largely unexplored. To address these issues, we introduce ChipExpert, the first open-source, instructional LLM specifically tailored for the IC design field. ChipExpert is trained on one of the current best open-source base model (Llama-3 8B). The entire training process encompasses several key stages, including data preparation, continue pre-training, instruction-guided supervised fine-tuning, preference alignment, and evaluation. In the data preparation stage, we construct multiple high-quality custom datasets through manual selection and data synthesis techniques. In the subsequent two stages, ChipExpert acquires a vast amount of IC design knowledge and learns how to respond to user queries professionally. ChipExpert also undergoes an alignment phase, using Direct Preference Optimization, to achieve a high standard of ethical performance. Finally, to mitigate the hallucinations of ChipExpert, we have developed a Retrieval-Augmented Generation (RAG) system, based on the IC design knowledge base. We also released the first IC design benchmark ChipICD-Bench, to evaluate the capabilities of LLMs across multiple IC design sub-domains. Through comprehensive experiments conducted on this benchmark, ChipExpert demonstrated a high level of expertise in IC design knowledge Question-and-Answer tasks.
Hardware Architecture,Artificial Intelligence,Machine Learning
What problem does this paper attempt to address?
The paper aims to address the high professional barriers and research and development challenges in the field of integrated circuit (IC) design. Although large language models (LLMs) have achieved significant success in multiple domains, existing LLMs often fail to meet the specific needs of students, engineers, and researchers in the IC design field. Therefore, the potential of LLMs in the IC design domain has not been fully developed. To address these issues, the research team introduced ChipExpert—the first open-source teaching assistant LLM specifically designed for the IC design field. ChipExpert is trained based on the Llama-3 8B model and goes through several key stages, including data preparation, continued pre-training, instruction-guided supervised fine-tuning, preference alignment, and evaluation. Through these steps, ChipExpert acquires a vast amount of IC design knowledge and learns how to professionally answer user queries. Additionally, to reduce model hallucinations, the research team developed a retrieval-augmented generation (RAG) system based on an IC design knowledge base. The paper also introduces the first IC design benchmark dataset, ChipICD-Bench, to evaluate the performance of LLMs across multiple IC design subfields. Comprehensive experiments show that ChipExpert demonstrates a high level of professional competence in question-answering tasks within the IC design field, particularly in basic concepts and advanced research questions, making it competitive with current state-of-the-art models. This indicates that ChipExpert has the potential to lower learning barriers for students, reduce training costs for engineers, and promote innovation and efficiency improvements across the integrated circuit industry.