3. Testing the testers
This story last week in the Wall Street Journal recounted how a slew of obvious, even hilarious, errors in the questions posed on English and math tests given to students in the third through eighth grades in New York State has “eroded trust in the statewide exams.” In one case, a now-infamous passage about a pineapple and a hare intended to test reading comprehension was so nonsensical that Gawker posted it and dared readers to answer the questions about it posed to eighth graders. If you want to see a great story begging to be written, click here and read the passage and the questions about it.
The company that wrote this test is Pearson, which is also a major force in text books, as well as the publisher of the Financial Times and co-owner of the Economist. Who at Pearson wrote this gibberish? (I’m assuming they didn’t borrow people from the FT or the Economist.) What’s the overall test creation process? One would have thought it would be quite elaborate given that this was part of a $32 million contract Pearson won from New York State to write the tests. What’s the profit margin on work like this? How many layers of approval did that reading passage go through? Who in the state school system vetted the tests and missed this and so many other screwups? Can the state get its money back? If not, why not? What did Pearson promise to get the testing contract in the first place?
For an ambitious news organization (I’d love to see it on 60 Minutes), all of this, in all its delicious detail, would be a lead-in to a broad-gauge overview of testing quality and the testing industry. At a time when the linchpin of the education reform movement sweeping the country is the push to use student progress on tests as an ingredient in evaluating teacher effectiveness, the quality of the tests - and their cost - has never been more important.