For programmatic usage oftentimes SOTA isn't useful.
For example, I have software that summarizes articles and classifies links on webpages to build a synthetic RSS feed, both of which use LLMs, neither of which need a SOTA model.
I'll probably use LLMs to bootstrap a dataset of native ads in articles, and there again, I don't really need a SOTA model.
If it's for more open ended tasks like writing code though, I agree that at this point SOTA models make more sense to use.
For example, I have software that summarizes articles and classifies links on webpages to build a synthetic RSS feed, both of which use LLMs, neither of which need a SOTA model.
I'll probably use LLMs to bootstrap a dataset of native ads in articles, and there again, I don't really need a SOTA model.
If it's for more open ended tasks like writing code though, I agree that at this point SOTA models make more sense to use.