Abstract:National calls in science, technology, engineering, and technology education reform efforts have advanced the wide-scale engagement of students in undergraduate research for the preparation of a workforce and citizenry able to attend to the challenges of the 21st century. Awareness of the potential benefits and costs of these experiences has led to an emerging literature base outlining gains in participants’ cognitive, affective, and conative domains to support the impact of undergraduate research for students of all backgrounds; however, the majority of this work has relied on self-report data limiting inferences to the causal effects on student learning. As part of a larger project on apprentice-like undergraduate research experiences (UREs) in the physical sciences, the present exploratory study complemented indirect self-report data with direct performance data to assess the development of chemistry students’ scientific thinking skills over a research experience. Performance data were collected using the Performance assessment of Undergraduate Research Experiences (PURE) instrument, a validated tool designed to assess changes in chemistry students’ analytical and data driven decision-making skills through open-response tasks situated in real-world problems from primary literature. Twenty-four summer research students in chemistry (46% women; 50% 1st/2nd year students; 42% first time URE participant) from seven colleges and universities provided baseline and post-intervention performance data. Differences in pre/post-response task correctness provided a direct measure of individual changes in student competencies. Early study findings indicate the positive contributions of UREs to student's competencies in the areas of problem-solving, experimental design and the use of research techniques, data analysis and the interpretation of results, and the evaluation of primary literature. Survey data were also collected on students’ self-skill ratings to allow comparisons between perceived and demonstrated competencies, which were found to be weakly correlated. This work begins to offer direct evidence to the effect of UREs on student learning progressions as well as the potential use of performance test data in evaluating the success of research training interventions designed to improve scientific thinking skills.

Measuring Learning Gains in Chemical Education: A Comparison of Two Methods

A comparison of Hake's g and Cohen's d for analyzing gains on concept inventories

Identifying Keys to Success in Innovative Teaching: Student Engagement and Instructional Practices as Predictors of Student Learning in a Course Using a Team-Based Learning Approach

Comparison and Evaluation of Learning Outcomes from an International Perspective: Development of a Best-Practice Process

Short-Term and Long-Term Effects of POGIL in a Large-Enrollment General Chemistry Course

Dynamic Models of Learning and Education Measurement

Evaluation of the Open-Ended Green Chemistry Generic Comparison (GC)2 Prompt for Probing Student Conceptions about the Greenness of a Chemical Reaction

Reducing the prior-knowledge achievement gap by using technology-assisted guided learning in an undergraduate chemistry course

Understanding the State of the Art for Measurement in Chemistry Education Research: Examining the Psychometric Evidence

Chemical Education Research: Improving Chemistry Learning

Students’ knowledge acquisition and ability to apply knowledge into different science contexts in two different independent learning settings

Learning of Core Disciplinary Ideas: Efficacy Comparison of Two Contrasting Modes of Science Instruction

Measuring the Effectiveness of Learning Resources Via Student Interaction with Online Learning Modules

Evaluating the development of chemistry undergraduate researchers’ scientific thinking skills using performance-data: first findings from the performance assessment of undergraduate research (PURE) instrument

Comparison of student confidence and perceptions of biochemistry concepts using a team-based learning versus traditional lecture-based format

Using Multiple-Choice Questions to Assess Student Acquisition of Scientific Inquiry Skills and Enhance Laboratory Design and Outcomes

Improving Science Achievement—Is It Possible? Evaluating the Efficacy of a High School Chemistry and Physics Project-Based Learning Intervention

Universal Design for Learning Chemistry Instruction for Students With and Without Learning Disabilities

Development and Assessment of a Molecular Structure and Properties Learning Progression

Lessons from a Large-Scale Assessment: Results from Conceptual Inventories

Scaffolded Instruction Improves Student Understanding of the Scientific Method & Experimental Design