Abstract:Concept inventories are standardized assessments that evaluate student understanding of key concepts within academic disciplines. While prevalent across STEM fields, their development lags for advanced computer science topics like dynamic programming (DP) -- an algorithmic technique that poses significant conceptual challenges for undergraduates. To fill this gap, we developed and validated a Dynamic Programming Concept Inventory (DPCI). We detail the iterative process used to formulate multiple-choice questions targeting known student misconceptions about DP concepts identified through prior research studies. We discuss key decisions, tradeoffs, and challenges faced in crafting probing questions to subtly reveal these conceptual misunderstandings. We conducted a preliminary psychometric validation by administering the DPCI to 172 undergraduate CS students finding our questions to be of appropriate difficulty and effectively discriminating between differing levels of student understanding. Taken together, our validated DPCI will enable instructors to accurately assess student mastery of DP. Moreover, our approach for devising a concept inventory for an advanced theoretical computer science concept can guide future efforts to create assessments for other under-evaluated areas currently lacking coverage.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to address the deficiencies of dynamic programming (DP), a complex algorithm design technique, in teaching evaluation. Specifically, the authors developed and preliminarily validated a Dynamic Programming Concept Inventory (DPCI) to fill the gap in standardized assessment tools for advanced computer science topics such as dynamic programming. #### Concepts and challenges of dynamic programming Dynamic programming is an algorithm design technique for constructing problem solutions by using optimal sub - structures and overlapping sub - problems. As an advanced algorithmic method, dynamic programming is considered one of the most difficult concepts for computer science students to master. Students often encounter the following difficulties when understanding dynamic programming: - Unable to recognize that dynamic programming is the best method to solve the problem. - When attempting to use dynamic programming, unable to correctly select the base case or determine the appropriate overlapping sub - problems. #### Deficiencies of existing assessment tools Although Concept Inventories (CI) have been widely used in many STEM fields, there is a lack of rigorously validated assessment tools for advanced computer science topics such as dynamic programming. Currently, although there are some draft concept inventories for algorithm analysis, there is no concept inventory covering a wide range of algorithm design techniques (including but not limited to greedy algorithms, divide - and - conquer algorithms, dynamic programming, etc.). #### Main contributions of the paper 1. **Develop and validate DPCI**: The authors described in detail the iterative process of constructing the DPCI, including how to design multiple - choice questions based on the student misunderstandings identified in previous studies. 2. **Psychometric validation**: Through preliminary psychometric validation of 172 undergraduate students, ensure that the problems in the DPCI are of moderate difficulty and can effectively distinguish the understanding abilities of students at different levels. 3. **Guide future research**: The methods in this paper can provide references for the development of concept inventories for other theoretical and advanced computer science topics. #### Method overview The authors followed a set of established research steps to develop the DPCI, including: 1. **Determine the topic of evaluation**: Focus on the core concepts of dynamic programming. 2. **Establish common misunderstandings**: Based on previous studies, identify common misunderstandings of students in dynamic programming. 3. **Create questions**: Design multiple - choice questions that can reveal these misunderstandings. 4. **Validation process**: Have teachers evaluate the effectiveness of the questions. 5. **Revision and repeated validation**: Continuously improve the questions according to the feedback until high - quality standards are reached. Through this method, the authors successfully developed an effective Dynamic Programming Concept Inventory, helping teachers accurately assess students' understanding of dynamic programming. In addition, their research also provides valuable experience and guidance for the development of assessment tools for other advanced computer science topics. --- Hope the above summary is helpful for you to understand the goals of this paper! If you have more questions or need further information, please feel free to let me know.

Construction and Preliminary Validation of a Dynamic Programming Concept Inventory

A General Concept Inventory for Introductory Computer Science - A Work in Progress

Chained computerized adaptive testing for the Force Concept Inventory

dpvis: A Visual and Interactive Learning Tool for Dynamic Programming

Applying Cognitive Diagnostic Models to Mechanics Concept Inventories

Analyzing-Evaluating-Creating: Assessing Computational Thinking and Problem Solving in Visual Programming Domains

Development and testing of the Draw-a-Programmer test (DAPT) to explore elementary preservice teachers’ conceptions of computational thinking

Domain-Independent Dynamic Programming

Toward A Framework for Formative Assessment of Conceptual Learning in K-12 Computer Science Classrooms

A multi-national, multi-institutional study of assessment of programming skills of first-year CS students

Biological Concepts Instrument (BCI): A diagnostic tool for revealing student thinking

Interview validation of the Physics Lab Inventory of Critical thinking (PLIC)

Domain-Independent Dynamic Programming: Generic State Space Search for Combinatorial Optimization

Exploring Pedagogical Content Knowledge of Physics Instructors and Teaching Assistants Using the Force Concept Inventory

PDL: Scaffolding Problem Solving in Programming Courses

What are we assessing? An analysis of the most common concept inventories in physics

ChatGPT's quality: Reliability and validity of concept inventory items

Exploring the Predictive Potential of Complex Problem-Solving in Computing Education: A Case Study in the Introductory Programming Course

A Game-Centered, Interactive Approach for Using Programming Exercises in Introductory Physics

Lessons from a Large-Scale Assessment: Results from Conceptual Inventories

Evaluating the quality of multiple‐choice question pilot database: A global educator‐created tool for concept‐based pharmacology learning