Stanley Pogrow has produced a paper concluding that most education research is of little value to people in schools. Research validating programs to develop the reading skills of students born into poverty, for example, validates programs that are not effective in practice, he says.
Pogrow’s paper analyzes in easy-to-understand language the validity of the gold standard scientific methodology used by the top research journals and the What Works Clearinghouse (WWC) to determine whether practices are effective, and examines the history of effective practices policies and their actual effectiveness. The author concludes that the increasingly sophisticated methods used to assess the effectiveness of practices (a) are flawed and exaggerate actual effectiveness, and (b) do not provide the type of information practitioners need. As a result, research on effective practices tends to mislead rather than inform practice and is a major reason why efforts to reform high-poverty schools have had limited success.
Pogrow’s paper, published in the peer-reviewed Education Policy Archives argues for educational research that identifies effective practices and brings them to scale by focusing on actual achievement gains. The author states:
It appears that we need to start over and rethink the methodological approach used to identify effective practices in research journals and government panels. We would probably do better to look at the simpler methods employed by obstetrics and the improvement science of health services. Both of these have established track records of identifying and disseminating effective practices that seem to have produced greater improvements in clinical practice than the experimental RCT model that education has viewed as the only model of rigorous science. Any future methodology for certifying programs to be effective should put a premium on the actual, unadjusted performance of students receiving the treatment, particularly at-risk subgroups, and whether the improvements (a) are ones that would likely be clearly visible to practitioners and parents, and (b) whether such improvements have occurred “reasonably” consistently in case studies. Such an approach would require a major culture change in the education research community. And it would require practitioners, policymakers, and researchers to consider the likelihood that practices previously considered to have the most proven track records of scientific evidence may, in fact, not be effective.
To read the paper, see http://files.constantcontact.com/d6ed868c001/a3bdea34-ca46-4947-8471-6d36795a4e83.pdf
