Abstract:Previous research provides evidence that ratings from students and external observers can deliver reliable and valid measures of teaching quality. However, there is still a lack of research on how stable these measures are across time and classes, which is particularly important when they are used for evaluations. We addressed this issue in 3 different studies from Germany. Study 1 examined stability over time using a longitudinal sample of 174 teachers and 106 classes. The results show that student ratings of teaching quality are relatively stable when the same class rates the same teacher in 2 subsequent school years. However, we also found teacher-independent stabilities, which occurred when the same students rated different teachers. Study 2 examined stabilities in student ratings across classes. In this study, each teacher (N = 95) was evaluated by 2 different classes (N = 190). The results of a 3-level model reveal rather low stabilities across classes within teachers. Study 3 drew on a unique dataset in which 1 single teacher taught the same content to 30 different classes of the same grade level. Teaching quality as rated by students and external observers varied considerably between these classes. Differences in teaching quality were related to student learning, and the motivation of students in a class was a significant predictor of differences in teaching quality. In summary, the results of these studies show that both researchers and policymakers should be cautious when interpreting measures of teaching quality as indicators of teacher quality.

The student evaluation of teaching and likability: what the evaluations actually measure

Students' evaluations of university teaching: Dimensionality, reliability, validity, potential baises, and utility.

Student Ratings of College Teaching: Reliability, Validity, and Usefulness

Students' evaluations of university teaching: Research findings, methodological issues, and directions for future research

Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility.

Class size, students’ evaluations, and instructional effectiveness

The student evaluation of teaching and the competence of students as evaluators

Validity of student evaluations of instructional effectiveness: A comparison of faculty self-evaluations and evaluations by their students.

Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related

Validity of students' evaluations of college teaching: A multitrait–multimethod analysis.

Validity of students' evaluations of teaching effectiveness: Cognitive and affective criteria.

The relative importance of teaching evaluation criteria from the points of view of students and faculty

The multidimensionality of students' evaluations of teaching effectiveness: The generality of factor structures across academic discipline, instructor level, and course level

Multidimensional students' evaluations of teaching effectiveness: A profile analysis

"He just teaches whatever he thinks is important": Analysis of comments in student evaluations of teaching

The influence of student, course, and instructor characteristics in evaluations of university teaching

Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness.

Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness

Don’t blame the teacher? The need to account for classroom characteristics in evaluations of teaching quality.

Are Student Evaluations of Teaching Effectiveness Valid for Measuring Student Learning Outcomes in Business Related Classes? A Neural Network and Bayesian Analyses

Management by results: Student evaluation of faculty teaching and the mis-measurement of performance