Floyd Norris raises a very good point in his column today on merchant credit-card fees: Why do we have a system that has cash buyers, who tend to be poorer, subsidizing credit and debit-card purchasers, who tend to be richer?

It’s a very good way to think about the interchange fees Visa, Mastercard, and the like charge retailers every time you swipe your card. That comes to about 2 percent of every plastic transaction.

You think retailers just eat that 2 percent? I don’t think so. These fees end up boosting the price of goods for everybody. That’s fine if you’re a card user. But what about the people paying with cash or even food stamps, as one Norris source points out:

“Food stamp recipients pay the higher credit prices for things they buy with food stamps,” said Mallory Duncan, the general counsel of the National Retail Federation. “They are subsidizing the people who get miles.”

Norris has a terrific paragraph here with a bunch of stuff that’s new to me:

How much less depends on what kind of business the store has. (Food stores pay smaller fees than clothing stores.) It depends on what kind of market power the store has. (Home Depot gets a bigger share than Fred’s Hardware.) It depends on whether the card is a credit or debit card. (Debit cards have lower fees.) It even depends on whether a credit card offers rewards. So when I use the Visa card that gives me airline miles, the merchant gets less than if I use a basic Visa card.

And he asks the right question:

All of this invites the basic question: If the store gets less when I use my credit card than it does when you pay cash, why should we pay the same price?

I’m wondering if it would be better to just charge the plastic-using consumer the fees.

Ryan Chittum is a former Wall Street Journal reporter, and deputy editor of The Audit, CJR's business section. If you see notable business journalism, give him a heads-up at rc2538@columbia.edu.