BA: We started with a definition of what is a promise, and we said that a promise is not a position statement. It is a pledge or guarantee of specific action that is verifiable. Now there are some that are more verifiable than others. Ultimately those ones with the fuzzy verbs are still legitimate promises that our readers are going to want us to make a call on, and we will explain how we do it. We’ll do our best to be fair and objective and precise about it. But I think that’s our value here. It’s not enough to just list the promises. But I think we want to go the extra mile, and say “Let us interpret this.”

KB: What kind of reporting do you do in order to adjudicate these?

BA: We talk to people in the field: experts, analysts, academics, whoever can help us make those valued judgments. Then the writer will make a recommendation.

Then usually three editors or reporters get together and review it. Most of the time we agree, and some of the time we’re down or up a notch, and other times we’re significantly different. There is a very thoughtful, deliberate process in doing these things. In the case of the Obameter so far, because we’re creating something new, we’re still sort of figuring it out.

KB: Are some of the promises things we should expect to hear your reporters ask about during White House press conferences?

BA: No. We talk to the White House through phone calls and e-mail, because it’s a very specific thing. It’s not something that Robert Gibbs can answer from the podium. You want to get to the issue person on the press staff who deals with that one. It will be more of an issue-specific thing, but it’s not the sort of thing you do with the White House press office.

The White House press office has had a difficult transition, in part because they’ve had difficultly getting e-mail in the last couple of weeks. They have not yet appointed one person to deal with us, so we’ve been going to different people depending on the topic. In the grand scheme of things, we haven’t graded that many promises yet, and most of our work in the past two weeks has been Truth-O-Meter items on the stimulus rather than Obameter items.

Now we are about to launch a probably week long, or maybe two-week long, series of things of new Obameter items that would come out of the stimulus. There’s been relatively little action on the Obameter in the last week. Starting tomorrow, there will be four or five new Obameter items every day. For the last few weeks, the action has been less about Obama’s campaign promises because it’s early in the administration; they haven’t had time to make much progress. It’s not like we’re calling them every day and ask: “Hey, what’s up with No. 237?” Instead, we’ve been taking these many, many claims that people have been making about the stimulus bill and fact-checking them for the Truth-O-Meter.

KB: So, the folks in the press office know you’re doing this?

BA: Absolutely. I saw Robert Gibbs at an awards dinner last week and he’s very familiar with us. Some of the other people in the press office have occasionally tried to urge us to give rulings that are more positive to Obama. And we would expect that, just as anyone who we’re fact-checking would try to make the case that their facts are correct.

KB: Do you think this promise-by-promise approach is the best way to evaluate the success of a president?

BA: I would hope that readers are sophisticated enough to know that this is just one measurement of the presidency. As I said in my editor’s note when we launched, the fact that a promise is broken is not necessarily a bad thing. It could be that the president stops pursuing something because he decides that he doesn’t have support for it, or he has other things that have a higher priority. Case in point, one that we now have rated as “broken,”—his promise to create a $3,000 tax credit for each new job that a business creates. That was a promise that Obama made in the campaign. When he first came to the White House, that promise was discussed on Capitol Hill and it received lots of opposition.

Katia Bachko is on staff at The New Yorker.