Google returns the best result based on both it's calculations, and click history of what clicks were most successful for a search.

LLM's don't really have that same response partially because it's strength is writing one sentence many different ways. The many different ways to write a sentence doesn't mean it's the best way. If it can write deep sentences, keeping a coherent, connected arc through sentences and stories

LLMs' also generally return the "best" answer as the most "common" one, without weight towards outliers as easily that might be the most true, or the best.

The definition of what is "good" and "correct" can also vary quite a bit, especially with writing.

AI can be configured to look for patterns humans might not see, but we also know humans can see things and scenarios that LLM's aren't trained on and can miss getting to.

As we can tell with AI copy, it all starts to sound the same even if it's new. Real writing ages differently. It can be much more of a finger print. This is an area I'm hoping to learn more about from the talented writers in my life - it seems the better the writer, the more they can see the holes of LLM and also be the best power users of LLMs by their superior ability to use words whether they realize it or not.