The counter argument is simple. The right to property is inalienable and absolute.
If your neighbours can dictate what you can and cannot build on your own land can you even say that you really own it?
The idea that a small cartel of homeowners can just artificially lower supply is anti freedom and anti free market.
Then such "neighbours" are welcome to vacate their houses and move to a communist country of their liking, which has no respect for private property. I am sure they will find a lot of commonality in values. Let everyone else enjoy freedom
