Exploring the Role of AI Assistants in Computer Science Education: Methods, Implications, and Instructor Perspectives

Tianjia Wang,Daniel Vargas-Díaz,Chris Brown,Yan Chen
DOI: https://doi.org/10.1109/VL-HCC57772.2023.00018
2023-11-13
Abstract:The use of AI assistants, along with the challenges they present, has sparked significant debate within the community of computer science education. While these tools demonstrate the potential to support students' learning and instructors' teaching, they also raise concerns about enabling unethical uses by students. Previous research has suggested various strategies aimed at addressing these issues. However, they concentrate on the introductory programming courses and focus on one specific type of problem. The present research evaluated the performance of ChatGPT, a state-of-the-art AI assistant, at solving 187 problems spanning three distinct types that were collected from six undergraduate computer science. The selected courses covered different topics and targeted different program levels. We then explored methods to modify these problems to adapt them to ChatGPT's capabilities to reduce potential misuse by students. Finally, we conducted semi-structured interviews with 11 computer science instructors. The aim was to gather their opinions on our problem modification methods, understand their perspectives on the impact of AI assistants on computer science education, and learn their strategies for adapting their courses to leverage these AI capabilities for educational improvement. The results revealed issues ranging from academic fairness to long-term impact on students' mental models. From our results, we derived design implications and recommended tools to help instructors design and create future course material that could more effectively adapt to AI assistants' capabilities.
Human-Computer Interaction
What problem does this paper attempt to address?
### What problems does this paper attempt to solve? This paper aims to explore the role, impact, and coping strategies of artificial intelligence assistants (such as ChatGPT) in computer science education. Specifically, the research attempts to solve the following key problems: 1. **Evaluating the capabilities of AI assistants**: - The researchers evaluated ChatGPT's performance in solving 187 problems from six computer science courses with different difficulty levels. These problems cover multiple types, including multiple - choice questions, short - answer questions, and programming questions. - The purpose is to understand the performance of AI assistants on different types and levels of computer science problems and identify their strengths and limitations. 2. **Exploring question modification methods to prevent abuse**: - The research proposed two question modification methods to help teachers and teaching assistants adjust course content and prevent students from misusing AI assistants. - **Method 1 (M1)**: Make the question more difficult for AI assistants to solve by adding misleading or interfering information. - **Method 2 (M2)**: Require students to verify the answers generated by AI assistants to reduce shallow learning and improve students' critical thinking abilities. 3. **Understanding teachers' views and coping strategies**: - The researchers conducted semi - structured interviews with 11 computer science teachers to discuss their views on AI assistants and how to adapt to the changes brought by these tools. - The interview content includes teachers' impressions of AI assistants' performance, opinions on question modification methods, concerns about academic integrity issues, and how they deal with these challenges in teaching. 4. **Design suggestions and tool recommendations**: - Based on the research results, the author proposed design suggestions and tool recommendations to help teachers better adapt to the functions of AI assistants, improve course materials, and ensure that students' learning experiences are more personalized, fair, and adaptable. ### Summary By evaluating the performance of AI assistants, exploring question modification methods, and understanding teachers' views, this paper finally provides a series of design suggestions and tool recommendations for computer science education to cope with the opportunities and challenges brought by AI assistants. The core of the research is how to support teachers to more effectively adapt to the functions of AI assistants, prevent students from abusing them, and improve students' learning experiences.