We’re reposting two recent stories below because of their timeliness and relevance as the end-of-the-school year standardized tests descend on classrooms everywhere.
In the first piece, the Houston Federation of Teachers and seven individual teachers are filing a federal suit claiming that the standardized tests used to evaluate teachers are full of defects and don’t reflect actual ability. Rather than let such tests be used to arbitrarily punish teachers, educators are standing up and demanding a voice in their workplace.
The second piece reports on how The American Statistical Association, “the largest organization in the United States representing statisticians”, comes down against using standardized tests to evaluate teachers. So Houston teachers, worry not, the numbers are on your side.
HFT and seven teachers file lawsuit against Houston ISD
Originally posted here.
HOUSTON—Highly recognized, good teachers are receiving poor evaluations because of a grossly flawed value-added algorithm that should be changed, seven Houston teachers and the Houston Federation of Teachers said today in an unprecedented lawsuit filed in U.S. District Court for the Southern District of Texas.
The lawsuit details numerous problems with the Houston Independent School District’s Education Value-Added Assessment System, or EVAAS. Its statistical methodology uses a student’s performance on prior standardized tests to predict academic growth in the current year, though what is considered a sufficient level of growth is not defined. A teacher’s EVAAS score is supposed to measure the effect, or added value, of a teacher on a student’s academic growth over the school year. The school district uses this deeply flawed methodology for decisions about teacher evaluation, bonuses and termination, yet it is a “black box” system in which the methodology is considered proprietary and confidential.
“Due to a faulty, incomprehensible and secret formula, good teachers like the ones filing this suit are being labeled failures and our entire education system is being reduced to a numbers game,” said American Federation of Teachers President Randi Weingarten. “Testing isn’t aligned with the purposes of public education. It doesn’t measure big-picture learning, critical thinking, resilience, creativity or curiosity, yet those are the qualities that great teaching brings out in a student. The fixation on testing has literally drained the joy out of learning. We’ve always been leery of value-added models, and we have enough evidence to make clear that not only has VAM not worked, it’s been really destructive and it in no way helps improve teaching and learning.”
Daniel Santos, an award-winning sixth grade social studies teacher at Jackson Middle School, said the evaluation system is failing him, his students and his profession.
“It’s dispiriting and insulting to be told I’m ineffective, a judgment that doesn’t mesh with my classroom performance or the time and effort I devote to my students. Texas is using a broken evaluation system that isn’t properly identifying who really needs help to improve,” Santos said, adding, “My students are being tested on material that is not aligned with our curriculum.”
While there have been other suits challenging VAM, the Houston suit is unique because it was brought by highly recognized teachers who contend their poor EVAAS ratings do not correlate with their actual performance nor take into account socioeconomic or demographic variables in predicting student performance. The complaint also charges that the district directed and/or pressured school administrators to “manufacture deficiencies or otherwise find fault with the instructional practices” by teachers who received low EVAAS ratings. Those teachers would be placed on a growth plan to correct the supposed faulty instructional practice.
Myla Van Duyn, who teaches ninth grade biology at Davis High School, received an unwarranted low EVAAS score and contends her classroom observation scores were artificially lowered to be aligned with the EVAAS scores.
“This is demoralizing,” said Van Duyn, who is leaving the school district at the end of the school year, a decision motivated in part by the flawed evaluation system.
“EVAAS is driving out great Houston teachers because they’d rather work in a place that respects teachers. Everybody can do better, but the EVASS method does not accurately account for all of the variables that go into a child’s ability to answer questions on a multiple choice test,” Van Duyn said.
Andy Dewey, another plaintiff, is a history teacher at Carnegie Vanguard High School who developed the curriculum for two history courses and has been consistently rated well by his administrators. “Houston’s evaluation system is sold as a system to support and strengthen teaching, but it’s actually a bait and switch sham that’s weakening instruction and not helping teaching or learning at all,” Dewey said. “Teachers are told their scores are low but are not given information about what they did or did not do to cause their students to perform less than predicted.”
Gayle Fallon, president of the Houston Federation of Teachers, said: “The district is using a woefully inaccurate, educationally destructive evaluation system. If a car had as many design defects as the EVAAS system, it would be recalled as a lemon. EVAAS should be replaced with a fair and meaningful system that will actually help improve teaching and learning.”
AFT’s Weingarten said the real culprit is school districts’ fixation on testing, noting none of the top-performing countries subjects their students to as much testing as the United States.
“The testing obsession has turned kids into test scores and teachers into algorithms. Rote memorization and testing don’t prepare our students for the 21st century. We need to help kids problem solve, think critically, learn to be persistent and work in teams,” Weingarten said. “This country has spent billions on accountability, not on the improvement of teaching and learning at the classroom level, and value-added models are the leading edge of this misguided effort,” she said.
Weingarten said districts should develop, with teachers, a comprehensive evaluation system based on multiple measures that accurately identifies teachers needing improvement and that provides targeted help. To reclaim the promise of public education, she said there should be safe and welcoming neighborhood schools, access to early childhood education, wraparounds services at schools to address health, social and emotional issues, and project-based instruction.
Weingarten first questioned the fairness and accuracy of value-added metrics in 2007. It also has been criticized as inaccurate and an unstable measure of teacher performance by, most recently, the American Statistical Association, as well as RAND Corp. researchers, the National Academy of Sciences and the Economic Policy Institute. The ASA said the majority of the variation in test scores is attributable to factors outside the teachers’ control.
The federal lawsuit states teachers’ due process constitutional rights are being violated because the EVAAS system is not an accurate or reliable indicator of teachers’ performance and because “a cloak of secrecy” prevents teachers from verifying or challenging their EVAAS score. The suit also contends teachers’ equal protection constitutional rights are being violated because the school district directs and/or pressures school administrators to align teachers’ instructional practice scores with their EVAAS scores—two separate evaluation components—so those with below-average EVAAS scores receive arbitrary, harsher instructional practice scores. The suit also contends that the standards for acceptable growth are arbitrary, vague and constantly being recalibrated.
In addition to the Houston Federation of Teachers, Daniel Santos, Myla Van Duyn, and Andy Dewey, the other plaintiffs are Ivan Castillo (a fourth-grade bilingual teacher who has taught at Briscoe Elementary School since 1994, was the school’s bilingual teacher of the year in 2013-14 and is a finalist for bilingual elementary teacher for the Houston Independent School District) ; Paloma Garner (a ninth-grade biology teacher at David High School, has received several national awards recognizing her mentoring skills and has a consistent record of strong performance); Araceli Ramos (a ninth-grade English teacher at Austin High School who was selected in 2014 by a school official to tutor teachers on the Texas English Language Proficiency Assessment System) and Joyce Helfman (an eighth-grade English teacher at Johnston Middle School who has had a consistent record of strong performance and is well respected by her colleagues).
Statisticians slam popular teacher evaluation method
By Valerie Strauss
Originally posted here.
You can be certain that members of the American Statistical Association, the largest organization in the United States representing statisticians and related professionals, know a thing or two about data and measurement. That makes the statement that the association just issued very important for school reform.
The ASA just slammed the high-stakes “value-added method” (VAM) of evaluating teachers that has been increasingly embraced in states as part of school-reform efforts. VAM purports to be able to take student standardized test scores and measure the “value” a teacher adds to student learning through complicated formulas that can supposedly factor out all of the other influences and emerge with a valid assessment of how effective a particular teacher has been.
These formulas can’t actually do this with sufficient reliability and validity, but school reformers have pushed this approach and now most states use VAM as part of teacher evaluations. Because math and English test scores are available, reformers have devised bizarre implementation methods in which teachers are assessed on the test scores of students they don’t have or subjects they don’t teach. When Michelle Rhee was chancellor of D.C. public schools (2007-10), she was so enamored with using student test scores to evaluate adults that she implemented a system in which all adults in a school building, including the custodians, were in part evaluated by test scores.
Assessment experts have been saying for years that this is an unfair way to evaluate anybody, especially for high-stakes purposes such as pay, employment status, tenure or even the very survival of a school. But reformers went ahead anyway on the advice of some economists who have embraced the method (though many other economists have panned it). Now the statisticians have come out with recommendations for the use of VAM for teachers, principals and schools that school reformers should — but most likely won’t — take to heart.
Here’s part of what they said:
*VAMs are generally based on standardized test scores and do not directly measure potential teacher contributions toward other student outcomes.
*VAMs typically measure correlation, not causation: Effects – positive or negative – attributed to a teacher may actually be caused by other factors that are not captured in the model.
The entire statement is below.
Some economists have gone so far as to say that higher VAM scores for teachers lead to more economic success for their students later in life. Work published by the National Bureau of Economic Research, done by authors Raj Chetty, John N. Friedman and Jonah E. Rockoff, has made that claim, though there are some big problems with their research, according to an analysis of their latest study published by the National Education Policy Center at the University of Colorado Boulder. The analysis finds a number of key problems with the report making the link between VAM of teachers and financial success of students, including the fact that their own results show that VAM calculation for teachers is unreliable.
You can read the analysis below, after the American Statistical Association’s statement.
The evidence against VAM is at this point overwhelming. The refusal of school reformers to acknowledge it is outrageous.