Enhancing Evaluation and Feedback in Computer Organization Labs with an Automated RISC-V Processor Verification Framework
Yichuan Gao,Ziang Liu,Shanshan Li,Weidong Liu
DOI: https://doi.org/10.1109/fie58773.2023.10343047
2023-01-01
Abstract:This Practice Work-in-Progress paper presents an innovative approach for providing better evaluation and feedback for student-built Central Processing Units (CPUs) in Computer Organization course settings. A common lab project in such courses is the implementation of a simple CPU in Hardware Description Language (HDL). For example, students major in Computer Science at Tsinghua university are required to build a pipelined RISC-V processor during the autumn semester of their third year. However, due to the complexity of CPU internal states, debugging can be time-consuming for students, while assessing their designs and providing efficient feedback can be challenging for instructors. To overcome this challenge, we propose an automated verification framework using directed assembly code generation and trace comparison. Our framework uses two types of assembly test cases: one set specifically designed to trigger pipeline data and control hazards, and another set randomly generated with controlled instruction types and counts. In order to apply this industry-proven method to a wide variety of student designs, we also developed a technique for identifying common processor structures, such as register files and memory buses, that is essential for locating critical signals required for generating trace data via simulation. To evaluate the correctness of the student-designed CPUs, the recorded register and memory writing events are compared with the log generated by the Spike RISC-V ISA simulator, and the matching percentage between the two records is calculated. To provide feedback to students, we also present detailed information on the mismatches, which can help them identify bugs or areas for improvement in their designs. Tests on real student code demonstrate that our framework is more effective in detecting hidden logic errors, compared to traditional evaluation methods used in our teaching practice. The proposed method is beneficial for instructors as it saves time and improves the feedback loop in computer organization labs. Moreover, it provides additional data for assessing students' independent thinking and practical skills. With the growing popularity of the RISC-VISA in education, this approach is potentially applicable to the teaching of computer systems on a broader scale.