CS1-LLM: Integrating LLMs into CS1 Instruction

Annapurna Vadaparty,Daniel Zingaro,David H. Smith IV,Mounika Padala,Christine Alvarado,Jamie Gorson Benario,Leo Porter

2024-04-17

Abstract:The recent, widespread availability of Large Language Models (LLMs) like ChatGPT and GitHub Copilot may impact introductory programming courses (CS1) both in terms of what should be taught and how to teach it. Indeed, recent research has shown that LLMs are capable of solving the majority of the assignments and exams we previously used in CS1. In addition, professional software engineers are often using these tools, raising the question of whether we should be training our students in their use as well. This experience report describes a CS1 course at a large research-intensive university that fully embraces the use of LLMs from the beginning of the course. To incorporate the LLMs, the course was intentionally altered to reduce emphasis on syntax and writing code from scratch. Instead, the course now emphasizes skills needed to successfully produce software with an LLM. This includes explaining code, testing code, and decomposing large problems into small functions that are solvable by an LLM. In addition to frequent, formative assessments of these skills, students were given three large, open-ended projects in three separate domains (data science, image processing, and game design) that allowed them to showcase their creativity in topics of their choosing. In an end-of-term survey, students reported that they appreciated learning with the assistance of the LLM and that they interacted with the LLM in a variety of ways when writing code. We provide lessons learned for instructors who may wish to incorporate LLMs into their course.

Computers and Society,Software Engineering

What problem does this paper attempt to address?

This paper describes a reform in the teaching of an introductory computer science course (CS1) by integrating large language models (LLMs) such as ChatGPT and GitHub Copilot. The researchers found that LLMs can solve most of the assignments and exam problems in a beginner programming course. As a result, they redesigned the course by reducing the emphasis on grammar instruction and instead focusing on the skills required to write software using LLMs, including code interpretation, code testing, and problem decomposition into smaller functions. The course includes frequent formative assessments and three open-ended projects covering areas such as data science, image processing, and game design to showcase student innovation.

The design principles of the course include:
1. Encouraging students to use LLMs throughout the course.
2. Preparing students to adapt to the industry environment of using LLMs.
3. Applying existing research to improve the outcomes of underrepresented student groups.
4. Providing opportunities for creativity.
5. Offering equal services to students who continue studying computer science and those who do not.

After the course adjustment, students expressed comfort in using LLMs for programming, with the majority believing it helped their learning. However, some students had concerns about relying too heavily on these tools. Most students read and test the code when interacting with LLMs, but sometimes they found the responses confusing.

In conclusion, this paper explores how to adapt CS1 courses to new programming practices in the context of the increasing prevalence of LLMs and provides lessons learned for other teachers who want to integrate LLMs.