I completely disagree with what you're saying, but it may be because of the city that I'm living in - Columbia, SC. Many, many shops and stores down here don't take credit cards and it is not rude to the customer at all. In most cases the store has been around since before credit cards were popular and it is you who should adapt to the store, not the other way around. And there will always be stores that couldn't function with the cost of processing credit cards (not only the transaction cost, but the time and equipment cost to setup and maintain a processing system).

Basically you learn to have an extra $20 on you when you're walking around certain areas of the city and it's not a big deal at all.