Last week, a joint letter from the chancellors of The California State University, the University System of Maryland, and the State University of New York was delivered to Brian Kelly, Editor of U.S. News and World Report. The letter was among many that have been submitted by colleges and universities in protest over a proposed rating of teacher education programs by the National Council on Teacher Quality (NCTQ) and U.S. News and World Report.
The full text of the letter appears below:
While we have a long and deep commitment to public accountability, we are writing to inform you that we have advised our Colleges and Schools of Education to delay participation in the U.S. News and World Report and National Council on Teacher Quality (NCTQ) review of the nation’s teacher preparation programs until numerous serious concerns about the methodology are addressed. Our public institutions are highly supportive of accountability and transparency, but review procedures must be valid and fair in every dimension. We believe that if done according to legitimate standards, the process can actually be used as a means of deep program analysis and improvement.
Our universities are fully supportive of using findings from program reviews to reform and improve programs, a strategic approach we strongly endorse and use regularly. Such reviews, however, must be based upon rigorous study and characterized by openness to institutional input at all critical steps.
The work of NCTQ has to-date been characterized by questionable research methodology, inappropriate tactics in data collection, and a lack of opportunity for institutions to correct errors in data. Our many specific concerns with the proposed study include the following:
• Insufficient detail has been made available regarding the data to be collected, the methods for scoring, and the rater attributes. We cannot be assured of the rigor of the study’s procedures. The previous NCTQ methods for evaluation raise serious concerns about the accuracy of the data collected by NCTQ and the validity of conclusions derived from the data. It is essential that our institutions be able to review all key aspects of the methodology and data collection in advance.
• We are especially concerned with the primary focus on review of inputs. An emphasis on course syllabi and other program inputs is altogether inappropriate. Analyses are needed that reflect the comprehensive types of evidence on program outcomes and impacts collected by programs. Our institutions have a thorough commitment to evaluation of both candidates and programs. They utilize state-of-the-art teacher performance assessment instruments and a range of reliable outcome measures. For example, the California State University has the most comprehensive program evaluation procedure in the nation, which has for a decade collected outcome data from both graduates themselves and from the principals at their school sites. It is now also capable of value-added assessment for program evaluation. The several types of rigorous candidate and program assessment that leading institutions like our own have developed should be reflected in the review. The State University of New York and the University System of Maryland employ similar evidence-based strategies for program assessments.
• Attention also needs to be given to the most critical factors in program quality. Two reports issued over the past year describe clinical preparation of teachers as central—the report of the National Council for Accreditation of Teacher Education, Transforming Teacher Education through Clinical Practice: A National Strategy to Prepare Effective Teachers, and of the National Academy of Sciences, Preparing Teachers: Building Evidence for Sound Policy. Both contain highly respected analyses of relevant evidence regarding teacher preparation program effectiveness and emphasize the significance of clinical approaches to teacher preparation. Research validated approaches such as these need to be recognized fully in the study methodology.
• The most current work on standards for teacher education need to be reflected in the survey approach. These include the comprehensive work of the Interstate New Teacher Assessment and Support Consortium (InTASC), spearheaded by the Council of Chief State School Officers and the National Governors Association. The work of dozens of groups around the nation addressing the national common core standards, state implementation, and the implications for teacher education need to be reflected in the study.
• Additionally, for appropriate and significant reasons, the NCTQ process used in the reviews of Illinois and Texas education schools has been highly criticized. It is essential that NCTQ makes a commitment to accepting corrections to evaluations of programs provided by institutions of higher education. In previous studies, when institutions have provided valid corrections, there has not been a process ensuring that they be considered and accepted when accurate.
• Further, we object to the process used when institutions wish to withdraw from the study. In the past, when institutions have sought to withdraw, frequently due to sound concerns, NCTQ has refused and indicated that if the institution does not comply, then results would be based on what they are able to find online and elsewhere. This is not an appropriate response.
In sum, as public institutions, we are especially committed to rigorous accountability and ongoing transparency and believe that there can be significant value in external and independent reviews. It is essential, however, that the methods of such studies be rigorous and that their procedures ensure our programs have the opportunity to verify that the data and results are valid and reflect their distinctive quality. Unless we have assurances by March 15, 2011, indicating that each of our concerns, which are similar to those expressed by all other major universities in the nation are addressed, we will urge all of our campuses not to participate in the survey.
NCTQ has now posted its standards and the indicators for those standards on its website. They have also promised a publicly available forum for institutions to challenge ratings and analyses of their programs that are inaccurate. However, NCTQ has noted that this feature will only be available after the analysis and ratings are concluded and released to the public.
The National Council on Teacher Quality has also revised how the study will handle institutions that do not participate. Instead of simply stating that the programs in question failed the standards, U.S. News and the council will use whatever information they have to produce an “estimated” ranking. (The magazine plans to identify institutions where it has used such estimates.) Kate Walsh, president of the council, said that the rankings project had every right to make judgments about institutions that do not participate. “[P]rograms are certainly free to refuse to cooperate,” the letter says. “But doing so frees us to render our judgment…”
Robert Morse, who directs the higher education rankings of U.S. News, said in an interview that the magazine had no intention of backing away from the project or the methodology.
See the American Association of Colleges for Teacher Education’s collection of website resources on this topic.
See articles on this topic in The Chronicle of Higher Education, Inside Higher Ed, Education Week and The New York Times.
