You're arguing scale, and I agree that it's probably not possible to make everything accessible (not until we get those brain-controlled robots, anyway ;) ).
But if the question is "Where do you set the bar," we already have laws for that. Are they too many?
I think in some cases yes, in others not so much... I mean in Cali, there are places that literally don't have a restroom for customers because of risk of suit over buildings nearly half a century old.
For others, maybe not as much. In the end, it's a balancing game, and I keep seeing discussion (especially political) lose all sense of nuance or negotiation.
But if the question is "Where do you set the bar," we already have laws for that. Are they too many?