Many states and districts are adopting commercially available teacher observation instruments for the professional practice component of their evaluation systems. A report by Regional Educational Laboratory (REL) Mid-Atlantic provides new evidence to help educators choose among five widely used “off-the shelf” teacher observation instruments and decide how much weight to attach to scores on different dimensions of the instruments.
REL Mid-Atlantic analyzed the dimensions of instruction that the instruments measure, the relationships between teacher observation scores and their contributions to student achievement growth, and whether teachers’ observation ratings are affected by the characteristics of the students in their classrooms.
This study was designed to help school district and state decisionmakers weigh the benefits and drawbacks of these observation instruments by considering their similarities and differences in content, which observation scores most strongly and consistently predict teachers’ contributions to student growth, and potential bias in observation scores.
Researchers analyzed the content of the Classroom Assessment Scoring System (CLASS), Framework for Teaching (FFT), Protocol for Language Arts Teaching Observations (PLATO), Mathematical Quality of Instruction (MQI), and UTeach Observational Protocol (UTOP).
Results show that eight of ten dimensions of instruction are captured in all five instruments, but instruments differ in the number and types of elements they assess within each dimension. Observation ratings in all dimensions with quantitative data were significantly but modestly correlated with teachers’ value-added scores—with classroom management showing the strongest and most consistent correlations. Finally, among teachers who were randomly assigned to groups of students, observation ratings for some instruments were associated with the proportion of nonwhite and lower achieving students in the classroom, more often in ELA classes than in math classes. Findings reflect conceptual consistency across the five instruments, but also differences in the coverage and the specific practices they assess within a given dimension. They also suggest that observation scores for classroom management more strongly and consistently predict teacher contributions to student achievement growth than scores in other dimensions. Finally, the results indicate that the types of students assigned to a teacher can affect observation ratings, particularly in ELA classrooms.
When selecting among instruments, states and districts should consider which provide the best coverage of priority dimensions, how much weight to attach to various observation scores in their evaluation of teacher effectiveness, and how they might target resources toward particular classrooms to reduce the likelihood of bias in ratings.
Read the report at: http://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=4474
