> The knowledge is not in danger, in fact, it has never been safer. The AI models have read every manual that no human reads. They will recite, flawlessly and forever, exactly how all machines work.
That's wrong, and that's exactly why the loss of knowledge is such a problem. LLMs do not, and cannot, actually know a single thing. They are a statistical model, not knowledge. When they give out wrong information (and they always will, by their very nature), you need someone with actual knowledge to be able to recognize the BS and correct it. But we are losing the knowledge, and unless things change we will be no better off than the people in dystopian sci-fi stories who pray to the machine god because nobody knows how it actually works.
I can already see future where there is group of people goading AI to right direction by repeatedly changing what they ask slightly. And then memorise the times when you got right incantation for seemingly right outcome. Without actually understanding or being able to reason about process in the middle. Possibly a seemingly inconsequential misspoken word or typo can lead to better or worse outcome. Or maybe just not saying "Please" will sometimes produce wrong output. and other times you must not use it...
Yes. And since LLMs can not improve knowledge - I mean, they can generate new arrangements of information, but they have no idea whether any of it real or making sense, unless humans - explicitly or through training - tell them, the more we rely on LLM knowledge the less the quality of it would be. Right now the LLMs are mainly in auxiliary role, so most of the knowledge erosion they generate is laughed at and relatively quickly corrected. But would this hold once the role of generative AIs increases? We already essentially entered the chaos period with news content - there's so much noise that it's basically impossible to know if any news message you read is true or manipulated somehow. This is going to start happening to more fundamental knowledge too, either on purpose or just by the force of the probabilistic nature of generative AI.
I was just thinking. What is the future when LLM is used for both code running itself and then also designing the hardware it runs on? Will it be sustainable? Or will there be some sort of error cascade that destroys the whole thing?
Maybe AGI is impossible with current model as it simply can not reliably improve itself... Enough errors in any part of loop will stop the progression.
This comment might have gone hard in 2023. Now it seems out of place. LLM's do hold knowledge empirically. They sometimes give out wrong information, like humans. And you need someone with better knowledge to correct it.
> But we are losing the knowledge
No we aren't and this is spreading FUD. Things have always been like this. Its called specialisation and this is how society progresses. I don't know how the supply chain worked to get the food to my table. That's why its so cheap!
That's wrong, and that's exactly why the loss of knowledge is such a problem. LLMs do not, and cannot, actually know a single thing. They are a statistical model, not knowledge. When they give out wrong information (and they always will, by their very nature), you need someone with actual knowledge to be able to recognize the BS and correct it. But we are losing the knowledge, and unless things change we will be no better off than the people in dystopian sci-fi stories who pray to the machine god because nobody knows how it actually works.