Abstract:Competitive programming has become a popular way for programmers to test their skills. Large-scale online programming contests attract millions of experienced programmers to compete against each other. Competition-level programming problems are challenging in nature, and participants often fail to solve the problem on their first attempt. Some online platforms for competitive programming allow programmers to practice on competition-level problems as well, and the standard feedback for an incorrect practice submission is the first test case that the submission fails. Often, the failed test case does not provide programmers with enough information to resolve the errors in their code, and they abandon the problem after several more unsuccessful attempts. We present Clef, the first data-driven tool that can generate feedback on competition-level code automatically by repairing programmers' incorrect submissions. The key development is that Clef can learn how to generate repairs for incorrect submissions by examining the repairs that other programmers made to their own submissions over time. Since the differences between an incorrect program and a correct program for the same task may be significant, we introduce a new data structure, merge trees, to capture the changes between submissions. Merge trees are versatile: they can encode both large algorithm-level redesigns and small statement-level alterations. Clef applies the patterns it learns from a database of submissions to generate repairs for new submissions outside the database. We evaluated Clef on six real-world problems from Codeforces, the world's largest platform for competitive programming. Clef achieves 42.1% accuracy in repairing programmers' incorrect submissions. Even when given incorrect submissions from programmers who never found the solution to a problem on their own, Clef repairs the users' programs 34.1% of the time.

Towards a Fictional Collective Programming Scenario: an Approach Based on the EIF Loop

Three Contrasting Pairs’ Collaborative Programming Processes in China’s Secondary Education

Improve Feedback Mechanism in Programming Assessment Systems with Progress Indicators and Reward to Foster Students' Growth Mindset: (abstract Only)

Towards understanding the effective design of automated formative feedback for programming assignments

Analysing Computational Thinking in Collaborative Programming: A Quantitative Ethnography Approach.

An Intelligent Tutoring System for Programming Education Based on Informative Tutoring Feedback: System Development, Algorithm Design, and Empirical Study

Assessment of Programming Language Learning Based on Peer Code Review Model: Implementation and Experience Report

No Man is an Island: Towards Fully Automatic Programming by Code Search, Code Generation and Program Repair

Personalized Programming Guidance based on Deep Programming Learning Style Capturing

Understanding Learners' Problem-Solving Strategies in Concurrent and Parallel Programming: A Game-Based Approach

A Pair Programming Framework for Code Generation Via Multi-Plan Exploration and Feedback-Driven Refinement

A Machine Learning-Based Adaptive Feedback System to Enhance Programming Skill Using Computational Thinking

Toward Motivating Participants to Assess Peers' Work More Fairly: Taking Programing Language Learning As an Example

EgoCoder: Intelligent Program Synthesis with Hierarchical Sequential Neural Network Model

Research and Design of ACM-oriented C Language Experimental Teaching

Co-evolutionary automatic programming for software development

Effects of Different Feedback Strategies on Academic Achievements, Learning Motivations, and Self-Efficacy for Novice Programmers

From Critique to Computational Thinking: A Peer-Assessment-Supported Problem Identification, Flow Definition, Coding, and Testing Approach for Computer Programming Instruction

Can Distributed Practice Improve Students' Efficacy in Learning Their First Programming Language?

Automated Feedback Generation for Competition-Level Code

Evaluating Contextually Personalized Programming Exercises Created with Generative AI