Value Added Methods

A friend who is a math teacher in LA with some cautions on relying too heavily on value added methods:

 

It is truly encouraging to see so much ferment in the educational reform arena, especially as we see portions of the status quo starting to buckle.

 

At the same time, I think you may be going down a dead-end path with your (and many other reformers') embrace of Value Added Methods as a silver bullet for an objective measure of teacher performance.

 

I think that most policy makers and reformers are completely unaware of just how controversial the use of these models for the uses that most school districts and reformers want to use them for is.  Statisticians and mathematicians have written numerous criticisms over the last several years highlighting the theoretical shortcomings of most VAM approaches.  Among these are 1)  lack of transparency (you practically have to be a Ph.D. to understand what the model is doing, much less explain it in a non-technical way to stakeholders, 2)  no stability in ranking (a recent study showed that only about 20% of the teachers ranked in the top quintile in one year were in the top quintile the next year),  3)  poor data quality - often missing or corrupt,  4) the assumption that students take standardized tests seriously (their results on standardized tests often do not affect the grades they receive in the class),   5)  disaggregating the effect of a teacher in one subject from the performance of a student in another subject (e.g. a math teacher explaining mixture problems very well and consequently the student doing well in chemistry), etc.  The papers by Wainer in the Winter, 2011 issue of CHANCE magazine and by Ewing in the May, 2011 Notices of the American Mathematical Society give two (for the most part) non-technical overviews of the problems.

 

Please don't misunderstand.  I believe teachers should be ranked at least in part by objective measures which can be traced back to standardized tests.  I just think we need to continue to look for ways to do it which can pass a rigorous vetting by professionals.  Otherwise you are just setting people up for a "mathematical mugging".