
Eleven New York City education reporters were huddling on e-mail last October 20, musing over ways to collectively pry a schedule of school closings out of a stubborn press office, when the chatter stopped cold. Word had filtered into their message bins that the city was about to release a set of spreadsheets showing performance scores for 12,000 of the city’s 80,000 teachers—names included. Few understood better than the beat reporters that this wonky-sounding database was a game changer.
The Los Angeles Times already had jolted newsrooms across the country back in August, when it published 6,000 public school teachers’ names next to its own performance calculations. New York education reporters, though, were considerably more reluctant to leap on this bandwagon. They found themselves with twenty-four hours to explain a complex and controversial statistical analysis, first to their editors and then to the public, while attempting to fend off the inevitable political and competitive pressure to print the names next to the numbers, something nearly every one of them opposed. “I stayed up all night kind of panicked,” said Lindsey Christ, the education reporter for the local NY1 television station, “writing a memo to everyone in the newsroom explaining what was coming and what was at stake.”
It may seem odd that a geeky algorithm has become such a hot topic in education, but it is another indication of how a group of well-connected newcomers to the contentious world of education policy has influenced the national conversation on the subject. As a group—mostly Wall Street financiers, political lobbyists, and venture philanthropists—they are drawn to the tools and terms of business economics. In this case, that means something called “value-added metrics,” which estimate the worth of a teacher by analyzing her students’ test scores over time.
Supporters of this technique argue that teacher evaluations require objective rigor, calculated with statistics. Weak teachers, they argue, should not hide behind a subjective, protective system that undermines children’s futures. Critics counter that the calculations are incomplete, misleading, and often wrong. Teachers wonder how a number built on test questions can capture what it takes to help a student wrestle with ideas, say, or learn to write with voice. Wouldn’t it make more sense, they ask, to use student work, peer mentoring, and rigorous classroom observations for a more meaningful evaluation? Economists on all sides of the debate agree that these stats cannot paint a whole picture of effective teaching. So, the critics say, why print them indelibly next to teachers’ names?
But numbers have an allure. Governors and mayors facing huge budget cuts are demanding easier ways (read: rankings) to fire the worst teachers and reward the best. Washington likes numbers too. In the past year, eleven states including New York, Florida, and North Carolina have agreed to use student scores to evaluate teachers in exchange for federal Race to the Top grants.
So perhaps it was inevitable that elements of the free-market reform movement would land in the laptops of New York’s education reporters, with enough force to diminish the quality of the conversation about the city’s public schools.
The battle over the numbers is in part a battle over control. For decades, neither of the two national teacher unions has done enough to shed their more arcane rules, which has made innovation difficult. Still, the assault surprised the unions at first, mostly because it came from unexpected places, including the press. Steven Brill’s lopsided 2009 piece, “The Rubber Room,” in The New Yorker, was among the first to frame the current reform climate. He portrayed a war between good-guy marketplace reformers and villainous unions, blistering the UFT, the United Federation of Teachers, for its part in negotiating rules that led to the city’s practice of warehousing tenured teachers faced with discipline cases into “rubber rooms,” with little to do but punch the clock. The powerful union deserved the ridicule. Still, Brill allowed Schools Chancellor Joel Klein, the story’s white-hat protagonist, to dance around the irony that he had been in charge of the system, and thus the so-called “rubber rooms,” for seven years.

I don’t happen to know any education reporters who were drawn to this complex beat in order to pore over spreadsheets, or score an interview with Bill Gates as an education expert. Most pine for more time to spend in classrooms, in science projects with preschoolers, in rapt discussions with teachers or principals or parents. Most are inspired by education’s expansive connections to culture, science, politics, and the world of ideas. The best education reporters are skilled at the invaluable art of connecting the dots for readers between policy from on high and reality in the classroom. Yet education reporters have increasingly found themselves herded toward a narrow agenda that reflects the corporate-style views of the new reformers, pulling them farther and farther away from the rich and messy heart and soul of education.
So human interest stories right? Find a few successful teachers in a mediocre system, write a couple of puff pieces, make sure they include poor disadvantaged kids, and ignore the boring numbers.
#1 Posted by Rory, CJR on Tue 8 Mar 2011 at 02:57 PM
Why no mention of the fifth-grade teacher who committed suicide after the LA Times' rankings came out? We don't know for sure it was because of the database, but it does raise red flags about the validity and worth of publishing such untested figures. Journalists/editors might ask if they could actually cause harm if they publish.
#2 Posted by Kristi, CJR on Tue 8 Mar 2011 at 07:56 PM
It was a crackpot analysis with no validity or established methodology. The guy just pulled the methods out of thin air. Subsequent analysis proved that the methodology was not valid, especially at the individual level, as noted in this piece. It should never have seen the light of day. But the LA Times has published a number of rabid pieces against the LA school districts like this. They've always had a pro-business and anti-working class bias, but they've gotten far worse since the Trib and then Zell took over. The credibility of the LA Times on this issue is zero -- it's a shame, too, because they do such good journalism in other areas.
One should always be careful about allowing journalists anywhere near numbers. It hurts their brains.
#3 Posted by James, CJR on Tue 8 Mar 2011 at 08:31 PM
This article gives a thoughtful and thorough overview of the current climate in education policy and in education journalism.
But I think it needs to be clearer about where the Los Angeles Times' project went beyond the question of printing teachers' names and value-added stats in a situation where that information is released by officials (as in New York City).
The Times decided to (as this article says) "develop its own job performance system" for teachers. It acquired the previously unreleased, unpublished value-added stats on its own. Then it didn't just publish the statistics but rather showed each teacher's rating in a graphic format on a continuum from "most effective" to "least effective."
I think that a complete exploration would address the ethics and efficacy of the Times' project, given that situation. Is it within the scope of professional journalists to develop a job performance system for a profession outside their own? Is it valid? Is it ethical?
I attended a panel discussion at UC Berkeley about the Times series some months ago, which included Times reporter Jason Felch. At one point, Felch -- coming under sharp criticism by most of the other panelists -- accused fellow panelist Richard Rothstein, a nationally respected education researcher, of "defending the status quo." As anyone who follows the education-reform debate knows, the charge that critics of corporate education reform are "defenders of the status quo" is a standard soundbite from the "reform" backers. It was revealing to hear Felch display his partisanship with that blurt.
With the value-added project, the Los Angeles TImes went beyond being the messenger to being the judge and jury. That's what journalism critics should be examining.
#4 Posted by Caroline Grannan, San Francisco, CJR on Tue 8 Mar 2011 at 08:46 PM
Ms. Grannan, you make some very good points.
As a life-long resident of Los Angeles and basically a staunch supporter of the Los Angeles Times, I am not surprised that an LAT reporter revealed his bias as you described. The LAT has been, since the 1960's, rabidly anti-union, and that bias often spills over into their coverage of schools and education issues, blaming the teachers union for all the ills of the school district. And let's be clear, LA Unified has very, very serious issues -- in fact, the entire state has experienced a severe decline in its once magnificent educational system ever since Prop 13. Add to that the fact of the large (~33%) immigrant population, the intractable problem of poverty in Los Angeles, and the unwieldy nature of the huge bureaucracy that is LA Unified. But to the LAT, and the rightwingers, the school problem is all United Teachers Los Angeles (UTLA's) fault.
Here is a fact for Mr Hancock. Your quote:
"For decades, neither of the two national teacher unions has done enough to shed their more arcane rules, which has made innovation difficult."
displays a lack of understanding about unions.
In fact, every word, every phrase, every paragraph of every collective bargaining contract is negotiated between union representatives and management representatives These what you call "arcane rules" are negotiated and agreed upon between both parties. Unions don't just make up rules, arcane or not, out of thin air and impose them on everyone else. If a rule is so "arcane" that it is detrimental to the education of children, then let us not let management off the hook. But it may not really be "arcane", if you actually read the contract.
In fact, it is not the union's purview at all to determine competence or quality of teachers. That is a management function, and management has a number of tools available to improve performance: performance evaluations, escalating goal-setting and discipline all the way through firing. Where the union comes in is protecting employees from arbitrary job actions. In a collective bargaining agreement, the terms for discipline of the employee is agreed upon by both sides. So if you can actually find evidence of the myth of inability to fire incompetent teachers -- beyond the anecdotal parade of horribles -- you should look at why management is failing to do its job. That's just a fact. I suggest that you gain understanding of the collective bargaining process by sitting in on a series of collective bargaining negotiations.
#5 Posted by James, CJR on Tue 8 Mar 2011 at 09:29 PM
09/30/10
STANDARDIZED TESTS: THE TALE OF TWO LATINO TEACHERS
By James J. Lyons, Esq.
Hispanic Link News Service
I remember reading in the early 1980s a terse news story, only a few paragraphs in length, about the Educational Testing Service’s corporate decision to re-test students in Los Angeles who had done extraordinarily well on the Advanced Placement exam in calculus.
The students, who attended Garfield High School in East Los Angeles, were Hispanic and poor. Not said, but implied, was that their high scores were fraudulent, the product of cheating and chicanery.
They were retested and the results were the same; the impoverished Latino students passed the extremely rigorous exam, winning college credit for college-level calculus. There was no fraud involved, just a passionate Bolivian immigrant teacher by name of Jaime Escalante.
Jaime Escalante’s story inspired the movie “Stand and Deliver” and actor Edward James Olmos received an Academy Award nomination for his portrayal of this immigrant teacher.
Released in 1988, “Stand and Deliver” taught two potent lessons about schooling: don’t underestimate the academic ability of students who are poor and Hispanic, and don’t dismiss the power of a passionate Latino teacher.
Now I read a longer news story about a 39-year-old Hispanic teacher from the eastern part of Los Angeles. Rigoberto Ruelas loved teaching. He started as a teacher aide at Miramonte Elementary School when he was 22. Four years later, after receiving his education degree and credential, he returned to Miramonte as a fifth-grade teacher. Last week, Rigoberto taught his last class. This past weekend, Rigoberto committed suicide.
Rigoberto was deeply depressed. Not because he had been laid off or terminated as so many teachers have been in California and elsewhere because of our “under-performing” economy. He was depressed because his name had been listed in a controversial database created and published by the Los Angeles Times. It identified him as slightly “less effective” than other L.A. teachers. The database and its publication are part of a nationwide campaign to “reform” education.
The education “reform” campaign has been a major plank in President Obama’s mid-term election platform. Both the President and his basketball playing buddy from Chicago, Education Secretary Arne Duncan, refer to education as the civil rights challenge of our time, citing the dismal high-school completion and college attendance rates of racial and ethnic minority students.
This “education reform” campaign is radically different from the grassroots civil rights movements which shaped U.S. history. It is the brainchild of the Business Roundtable and has been bankrolled by billionaires. Microsoft founder Bill Gates, Eli Broad (founder of SunAmerica, now part of the financial/insurance behemoth AIG which taxpayers bailed out), and the Walton family members who control the global retail empire known as Walmart are just a few of the hugely wealthy folks funding the campaign.
Little wonder then that the campaign has been the object of unprecedented media coverage. Little wonder that the back-to-school programming of the major television networks, both broadcast and cable, has given wall-to-wall coverage to the campaign’s Holy Grail: higher standardized test scores in math and reading.
And little wonder that the Los Angeles Times created and published its database of teacher effectiveness based on student standardized test scores. The media, whether broadcast or print, live off of advertising revenue, and big corporations are the biggest advertisers.
Despite repeated warnings by experts in education testing and statistics that the Times database of teacher-ratings were unreliable and misleading, the paper published its grades for about 6,000 third- through fifth-grade teachers in the Los Angeles Unified School District. Edu
#6 Posted by James J. Lyons, CJR on Tue 8 Mar 2011 at 09:31 PM
“It’s in the public interest,” said Trontz of The New York Times. “If we find the data is so completely botched, or riddled with errors that it would be unfair to release it, then we would have to think very long and hard about releasing it.” How would they know? the statistical margin of error is very high, will they report this as well? Also I think its interesting that this article did not mention that it was the Hechinger Institute, run out of Teachers College at Columbia that financed this project; Teachers College, of course, is a sister institution to Columbia Journalism school, the sponsor of this publication.
Indeed, there were debates afterward among TC professors about whether this funding was ethical and/or aligned with university principles. Here is a quote from Prof. Tom Corcoran:
http://www.niemanlab.org/2010/09/l-a-times-controversial-teacher-database-attracted-traffic-and-got-funding-from-a-nontraditional-source/
Tom Corcoran 09/21/2010 06:06 PM
Nieman's one-sided coverage neglects to point out the high error rate in VAM analysis, the failure of the LA Times to check school rosters to make sure the scores were assigned to the right teachers (standard procedure elsewhere), and the problems associated with high absenteeism and student mobility (endemic in LA). Furthermore, the piece also fails to note that Hechinger which is associated with Teachers College, and uses its university affiliation, funded research without going through TC's institutional Review Board which would have insisted on protection for the human subjects included in the analysis. All in all, this was an injustice to LA's teachers, a bad day for social science, and a poor ethical decison by Hechinger. "
.
#7 Posted by Leonie Haimson, CJR on Tue 8 Mar 2011 at 09:31 PM
The term "venture philanthropist" is laughable if it weren't so misleading.
#8 Posted by dpjbro, CJR on Wed 9 Mar 2011 at 10:46 AM
This article doesn't mention the studies cited by Diane Ravitch showing that teacher value-added scores are widely variable over time for individual teachers and hence useless: a teacher's class can make big test-score gains one year and not the next, for variable reasons that may or may not reflect professional expertise. If "value-added" scores one year aren't predictive of performance in following years, what's the point? She also cites data that shows "value-added" gains can disappear in following years, suggesting that this form of statistical analysis does not measure real learning.
#9 Posted by Ann Kjellberg, CJR on Fri 11 Mar 2011 at 01:25 AM
The most interesting aspect of the Los Angeles Times’ handling of this whole issue is how the newspaper responded to criticism – by diving into a bunker and attempting to spin the findings from the University of Colorado, Boulder research as supporting their reporting.
I found this of particular interest, and wrote about it on my organization’s Blog: http://bit.ly/hxjFUw because the Times behaved identically when its coverage of the field I know best, child welfare, came under fire.
As for the general issue of how to reform education; as I read this excellent story, I kept wondering: Why would anyone expect the best ideas to come from the same people who gave us Windows Vista and “Clippy”?
Richard Wexler
Executive Director
National Coalition for Child Protection Refrom
www.nccpr.org
#10 Posted by Richard Wexler, CJR on Sun 13 Mar 2011 at 07:25 AM
As an education reporter, I find it interesting and disturbing that these philanthropies, politicians and advocates have co-opted the word "reform," to the extent that the word has become shorthand for merit pay, charters, value-added, etc. It is as if there were no other opinions on how to improve education, merely these ideas and "the status quo." Perhaps we need to focus more on other alternative viewpoints, rather than just pitting two sides against one another in a narrative that has become a cliche.
I hope no reporters are taking the Media Bullpen to heart-- it has apparently co-opted the word "accuracy" to mean "consistent with our viewpoints." I blogged (http://bit.ly/eMvpyP) about this issue and would love to heard more from fellow ed reporters. Who uses Media Bullpen, and why?
#11 Posted by Amy Crawford, CJR on Mon 21 Mar 2011 at 02:51 PM
There is no great mystery to quality education below the college level. Take any educational group, by state, by city, by community the one persistent factor is money. Whether it is the wealth of the family or the wealth of the community and, therefore, its school system, the more you spend the better you get. That doesn't ring true? Take a look at a comparison between states' per student expenditures and any measure of educational success that you care to use. The wealth factor works on two levels. On the one hand the children come in better prepared and from generally more stable and healthier home lives. On the other hand wealthy communities spend enough to have smaller class size, better maintained properties and more expansive curriculum
#12 Posted by Jac, CJR on Thu 24 Mar 2011 at 05:33 PM
Public Education is in shambles. Let's fix it.
Step 0: decide if this is a country, a loose aggregate of States, or a confederation of other corporate interests. Assume that the affected populace has a stake in the outcome of decisions made, and that they may cause economic harm to lesser bodies if ignored.
Step 1: establish a national education plan, linked to a national economic plan. Use that plan to promote those persons successfully completing a course of study in a line of work suitable to their abilities. Within that plan, provide alternative paths for those persons who are unmotivated, unwilling, and/or unable to progress within the expected time frame of course work.
Step 2: require Congress to fully-fund implementation of the economic and education plans. Annually, Congress shall assure that the Executive branch has in place, and has fully implemented, an on-going 20-year plan in support of economic priorities leading toward the economic and social growth determined by Congress, with five year review increments. Define "growth" as you will. Expected outcomes will be measured bi-annually. Variance from expectations will be examined and noted. Corrections to plans will be implemented by the Executive branch each five years.
Step 3: Measure the plans' progress in each part’s ability to promote student performance and welfare. Congress must direct the Executive branch to address significant variance from expectations. To accommodate this, the Executive branch should be charged to establish and maintain a detailed database of causally associated information.
Step 4: Evaluate the courses of action, the plans, and the implementation(s) thereof.
Repeat.
#13 Posted by Marshal H. Mercer, CJR on Mon 28 Mar 2011 at 11:02 AM
People who are trying to use the merit pay, performance measure, test score metrics on teachers should watch something.
http://www.youtube.com/watch?v=rrkrvAUbU9Y
Manufacturing cars isn't the same as manufacturing educated kids. The same incentives don't apply. You want kids to do well? Manufacture an environment where kids and teachers are respected and cared about and where basic standards of necessities are met without regard to the property taxes of the district. You can't make universal standards for teachers when there aren't universal standards for teachers' environments.
#14 Posted by Thimbles, CJR on Sat 23 Apr 2011 at 07:21 AM
The biggest, most vicious lie told by the reformers and the media is the notion it is "difficult" if not impossible to fire "bad" teachers. It is laughably easy to fire teachers, and they don't have to be "bad," either. Mostly it is older, more "expensive" or veteran teachers who are targeted for dismissal by idiot or vindictive principals or other administrators who are in fact those education personnel who are almost impossible to fire. Firings happen all the time thanks to holes in administrative law that makes it easy for districts to flout the few civil service protections (inaccurately called "tenure") teachers have and with no consequences for their actions.
Thousands of newer teachers each year are "non-renewed" (not teachers who are laid off--RIF'd--or teachers with temporary contracts) which is another term for being fired. They can be fired for any reason at all or for no reason. Thousands are forced into retirement or force to resign in lieu of dismissal. Those are effectively firings. Few teachers fight dismissals because the odds of "winning" those rigged hearings are more remote than winning the lottery. However, if teachers don't fight them, they stand to lose unemployment benefits and if they resign, would likely be thought of as having done something wrong by other districts. My point is comparatively few teachers are fired for true misconduct or criminal behavior. Many of them take severance agreements--resignations. The few teachers who are dangerous to students or engage in other criminal behavior are already dealt with through the court system and by licensing boards.
The biggest problem with education reporting is few reporters have any background at all in education as teachers. If they did, they would understand what hell teachers go through every single day of their lives.
#15 Posted by Susan Nunes, CJR on Sun 26 Feb 2012 at 11:46 AM