Abstract:Large language models (LLMs) and prompt engineering hold significant potential for advancing computer programming education through personalized instruction. This paper explores this potential by investigating three critical research questions: the systematic categorization of prompt engineering strategies tailored to diverse educational needs, the empowerment of LLMs to solve complex problems beyond their inherent capabilities, and the establishment of a robust framework for evaluating and implementing these strategies. Our methodology involves categorizing programming questions based on educational requirements, applying various prompt engineering strategies, and assessing the effectiveness of LLM-generated responses. Experiments with GPT-4, GPT-4o, Llama3-8b, and Mixtral-8x7b models on datasets such as LeetCode and USACO reveal that GPT-4o consistently outperforms others, particularly with the "multi-step" prompt strategy. The results show that tailored prompt strategies significantly enhance LLM performance, with specific strategies recommended for foundational learning, competition preparation, and advanced problem-solving. This study underscores the crucial role of prompt engineering in maximizing the educational benefits of LLMs. By systematically categorizing and testing these strategies, we provide a comprehensive framework for both educators and students to optimize LLM-based learning experiences. Future research should focus on refining these strategies and addressing current LLM limitations to further enhance educational outcomes in computer programming instruction.

What problem does this paper attempt to address?

The problems that this paper attempts to solve mainly focus on how to optimize computer programming education through large - language models (LLMs) and prompt engineering. Specifically, the research aims to explore the following key issues: 1. **Systematically classify prompt - engineering strategies**: Can prompt - engineering strategies be systematically classified according to different educational needs and problem types? Such a classification helps teachers and students customize prompts according to specific problem types and educational goals, thereby maximizing the effectiveness of computer programming teaching. Understanding these categories is crucial for developing highly - targeted and efficient learning paths. 2. **Expand the capabilities of LLMs**: Can different strategies endow LLMs with the ability to solve problems beyond their directly solvable range? This exploration aims to unlock the potential of LLMs to guide students through complex or unfamiliar problem - solving processes. By identifying strategies that can expand the capabilities of LLMs, their supporting role in the learning journey can be enhanced, providing support even in challenging scenarios. 3. **Establish an evaluation framework**: Can a robust framework be established to test the effectiveness of various prompt - engineering strategies and provide comprehensive implementation guidelines? This framework will enable teachers to systematically evaluate and optimize the LLM - based learning experience, ensuring that students reap the maximum benefits. By developing and validating these guidelines, a structured approach can be provided for enhancing educational practices and outcomes through LLMs. By solving these problems, this research aims to pave the way for personalized and adaptive coding education, enabling students to solve problems with greater confidence and a deeper understanding of core programming concepts.

Enhancing Computer Programming Education with LLMs: A Study on Effective Prompt Engineering for Python Code Generation

A Systematic Review on Prompt Engineering in Large Language Models for K-12 STEM Education

Promptly: Using Prompt Problems to Teach Learners How to Effectively Utilize AI Code Generators

LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Language

Interactions with Prompt Problems: A New Way to Teach Programming with Large Language Models

Prompt Engineering or Fine Tuning: An Empirical Assessment of Large Language Models in Automated Software Engineering Tasks

Prompt Problems: A New Programming Exercise for the Generative AI Era

Towards Goal-oriented Prompt Engineering for Large Language Models: A Survey

Towards a Catalog of Prompt Patterns to Enhance the Discipline of Prompt Engineering

CSEPrompts: A Benchmark of Introductory Computer Science Prompts

What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use

Do Advanced Language Models Eliminate the Need for Prompt Engineering in Software Engineering?

A Communication Theory Perspective on Prompting Engineering Methods for Large Language Models

Educational Personalized Learning Path Planning with Large Language Models

Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review

Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study

How Beginning Programmers and Code LLMs (Mis)read Each Other

Navigating the Pitfalls: Analyzing the Behavior of LLMs as a Coding Assistant for Computer Science Students—A Systematic Review of the Literature

AceCoder : An Effective Prompting Technique Specialized in Code Generation

StudentEval: A Benchmark of Student-Written Prompts for Large Language Models of Code

Strategic Optimization and Challenges of Large Language Models in Object-Oriented Programming