There is nothing wrong per se with SPIR-V to provide anything much better. What you said has indeed nothing to do with the intermediate language itself.
It's relevant to what you choose to use as the standard. A language can be fine on its own terms, but also you should only install it with a layer on top and not by itself.