OPINION: Value-added assessments: has the data been cooked?



imageBy David Lyell (left), LAUSD teacher

A report published this month by UC Berkeley economist Jesse Rothstein raises serious ethical questions about the objectivity of an analysis of “value-added” models by The Bill and Melinda Gates Foundation.

In his report, “Review of Learning About Teaching,” Rothstein concludes that The Gates Foundation’s “Measures of Effective Teaching” (MET) Project contains, “troubling indications that the Project’s conclusions were predetermined.” Rothstein asserts that “the Gates Foundation has widely circulated a stand-alone policy brief (with the same title as the research report) that omits the full analysis, so even careful readers will be unaware of the weak evidentiary basis for its conclusions.”

To promote “value-added” as a measure of teacher effectiveness, The LA Times hired Richard Buddin, a professor at UCLA, and paid him an undisclosed sum to conduct a statistical analysis of student test data. Buddin has himself been rated as ineffective by his own students on a website where UCLA students rate teachers.

In August 2010, LA Times reporters Jason Felch, Jason Song, and Doug Smith wrote a host of articles touting the benefits of “value-added” as a way of measuring teacher effectiveness, along with a database rating teachers.

On August 31, 2010, LAUSD Deputy Superintendent John Deasy proposed for value-added measures on tests to account for 30 percent of teacher evaluations. Six Board Members all voted for the proposal, with one dissent: School Board Member Marguerite LaMotte. The proposal now faces a major obstacle. It must be agreed to as part of collective bargaining negotiations in order for it to take effect.

Superintendent Cortines has said he wants LAUSD teachers to take seven furlough days for the 2011-12 school year, despite the fact that the LAUSD spends $100 million on non-mandated standardized tests — that is, testing not required by law — and $43 million on the eight mini-district offices.

I recently spoke to Deputy Superintendent John Deasy, and got his reaction to this story.

To critics like myself who think value-added should not at all be a component of teacher evaluations, Deasy said, “I take that at face value, and say I appreciate your position. I respectfully disagree.” He continued, “There will be margins of error. No question about that,” adding, “when people say that it’s ineffective, I think the issue is, for me, how do you use a balanced set of multiple measures to take a look at teacher effectiveness?”

Conceding that value-added models are imperfect, and reportedly have a plus or minus error of 45 points, and a teacher’s livelihood could be at stake, Deasy said it shouldn’t be the only measure of teacher effectiveness.

“A good observation by a well-trained principal,” said Deasy. “I believe that should be the majority indicator.”

When asked about the assessments of Buddin by his students on the Bruin website, where some students rated Buddin as “ineffective” and “boring,” Deasy responded, “I do think it matters how students perceive the learning experience with a teacher. Once again, I don’t think it is the only metric that should be used.”

I stated one of my objections to value-added, namely, that it perpetuates the demonization of teachers. I said, “I wish we did a better job supporting teachers. 50 percent of them quit within the first five years, even more so at charter schools. And part of it is because there is just a lack of support for teachers in the classroom.” Deasy responded: “I happen to agree with you completely on that. We have an enormous obligation to support teachers.”

When I proposed that cuts should be kept away from the classroom, especially given that LAUSD spends $100 million on non-mandated testing, and $43 million on the eight mini-district offices, Deasy responded, “My response to that is going to have to be, given the budget, as I read it, and given what has happened since the governor has taken office, all of those areas are going to have to be examined for further reduction.”

When pressed on this, and asked, “So you want to keep cuts away from the classroom?” Deasy responded, “Absolutely. We’ve already cut too much.” He added that the last cuts should be in the classroom.

Whether Deasy will stick to his word, and keep cuts away from the classroom, as stated, remains to be seen. How much he will stick to a deeply flawed methodology of evaluating teachers, also known as “value-added,” also remains to be seen.

Phone calls and emails seeking comment from UCLA instructor Buddin, LA Times reporters Felch, Song, Smith, and representatives from The Bill and Melinda Gates Foundation have not been returned. Additional phone calls and emails seeking comment from the following proponents of using value-added as a component in teacher evaluations have also not been returned: LAUSD Superintendent Ramon Cortines, LAUSD School Board Members Monica Garcia, Yolie Flores, Tamar Galatzan, Nury Martinez, Richard Vladovic, and Steve Zimmer, Los Angeles Mayor Antonio Villaraigosa, and US Education Secretary Arne Duncan.

To read Jesse Rothstein’s Rothstein’s critique of The Gates’ Foundation’s Measures of Effective Teaching Project, click here.

What do you think about Deasy’s position on value-added assessments? Let us know in the comments below.

Read more from David Lyell at davidlyell.blogspot.com.

OPINION: Walk in the shoes of a teacher



imageBy Sujata Bhatt, a teacher at Grand View Elementary

All names have been changed to protect students’ privacy.

The first month of school has ended, and many teachers working in urban districts have already gone through the seven stages of grief. Shock and denial over the skills of incoming students. Pain at the thought of how much work it will take to get them up to speed. Anger at the many factors that have brought these students to the classroom in this state of unpreparedness. Depression in the face of the magnitude of the task ahead. Reflection on what it will take to get the students going. The upward turn as students begin to cohere as a class. Reconstruction as the teacher finds methods and strategies to teach this group of students. And, finally, a glimmer of hope that maybe, just maybe, the job will get done.

In short, as the barbs and battle cries in the Great Education Debate fly all around us, we teachers have gotten down to work. We have assessed our students to see where they are in reading and math skills, in writing and comprehension. We have pored through databases to analyze test scores and fill out numerous worksheets to satisfy the experts and district number crunchers. But equally importantly, we have gotten to know our students as human beings.

I know that four out of my 24 fourth-graders read at grade level. One knows her times tables. Almost no one knows his or her phone numbers. Most don’t know their addresses. Many don’t know that they live in the city of Los Angeles, which is in the state of California in the nation of the United States on the continent of North America. Four did not read a single book over the summer. One has missed a week of school already. The other students say she has gone to Mexico, but we don’t know for sure, and we don’t know when she’ll be back. One has already moved to Bakersfield.

I know that fifteen out of twenty four parents showed up at an evening class meeting I held. Two sent notes that they couldn’t make it because they were working; I don’t know about the remaining seven.

I know that Osvaldo’s mom will meet me at the gate once a week to check on how he’s doing.

I know that Avery’s mom wants me to let her know about his behavior every single day, but she won’t come to class meetings or Back to School Night.

I know that David has a handicapped sister at home which makes it hard for his single mom to give him attention, but if I call her and let her know how he’s doing, she’ll make the time for him.

I know that Timothy’s mom will verbally support everything I do but has no time or energy to follow up on it.

I know that Harry has five siblings from five different dads and his mom has “f*** you” tattooed on her eyebrow.

I know that Nicandro’s dad drinks, curses, and (according to him) beats him, and his older brother is in and out of juvie.

I know that Monica’s mother has promised to take her to the California locales we’re exploring in Social Studies if she works hard and does well.

I know that Emilio’s family is taking him to Yosemite because we’re studying it in science. I know that Rigo’s mom is saving to buy him a laptop after I showed her how much extracurricular support the kids can find on the web.

I know that Liam will do anything in order to get computer time.

I know Aline has an enormous talent in motivating her classmates, but that she can only keep focused for 20-minute stretches.

I know that Isabel needs to speak up more, and Reginald needs to learn to listen to others more.

I know that Steven wants to be President of the United States so he can help poor people and those without documents. To motivate him, I just need to ask him if he thinks what he’s doing is presidential enough.

I could go on and on about their lives, their dreams, their weaknesses, their stories, and the year is just beginning.

Does any of this information matter in the Great Education Debate? No, say the experts, because it is not data. It is not quantifiable. It is not part of standardized testing. It can’t be used in Value-Added Measurements. It is merely anecdotal.

But is it valuable? It’s what teachers learn in order to help each individual student move forward. It’s what we have to work both with and against — even as all those so-called experts are telling us how and how not to do our jobs. I wish just for a month they’d walk in our shoes.