A new working paper by C. Kirabo Jackson, Assistant Professor at Northwestern, takes a closer look at value-added measures and their application to high school teachers. Jackson notes that there is ample research at the elementary level detailing the importance of individual teachers on student outcomes in reading and math, but there isn’t much supporting this conclusion at the secondary level. He also observes that “value-added methodologies designed for elementary school teachers may be inappropriate for measuring teacher quality in high school,” mainly because students are placed on different tracks in high school with different teachers, as opposed to spending time with one main teacher all day.
At the high school level, “even with random assignment of students to teachers, if different teachers teach in different tracks, and students in different tracks are exposed to different treatments, there will be bias due to ‘track treatment effects.’” Track treatment effects may be due to other teachers, the content of other courses, or explicit track level requirements/treatments. This results in two possibilities for selection bias: non-random placement of students due to their track, or omitted variables bias due to the different treatment of students based on their track. Given all of this, using value-added measures at the high school level may be an inefficient (and unfair) basis for teacher evaluation policies.
In a detailed study of 9th grade Algebra and English teachers’ student test scores, Jackson found substantial bias due to track-specific treatments. Once tracks are accounted for in the model, more than half of the teacher effects disappeared. In other words, it doesn’t seem to matter which English teacher a student has, though there are some teacher effects present in Algebra.
Jackson also found evidence that because students are sorted into tracks based on previous achievement, some tracks have higher pre-test achieving students than would exist if students were assigned randomly. There is no evidence of substantial sorting on other student characteristics within tracks.
Third, for Algebra there is a significant relationship between teachers’ previous value-added scores and current value-added score. However, the persistence of value-added effects is weaker than has appeared in elementary school studies (only about 1/3 as powerful). For English, no significant teacher effects were found.
Finally, Jackson found that Algebra teachers have no meaningful effect on English scores, and vice versa. This is contrary to what other studies have found. Jackson accounts for this difference by concluding other research has used methods that are biased and do not separate out the effects of random, non-persistent classroom effects.
To read the full paper, please visit http://works.bepress.com/c_kirabo_jackson/22/
