>I don't quite get it why they can't take another LLM and vet the output of the first with the seond one.
I think this may be part of the problem. The actual humans creating the report don't have the expertise to know which one to trust. At least that was what consulting was like in my experience at a similar firm.
>You (and a few other outliers) have their comment history piped into an RSS feed that I check regularly, solely because you refuse to write ordinary comments or submissions. Your post history can even be traced across accounts with basic stylometry, nothing about your contributions are subtle, normal, or genuine.
I don't believe you are a HN mod, frankly this looks like harassment/stalking. Write an email to Dang if it bothers you.
You don't have to be an HN mod to do any of that. The title is negligent and misleading to readers, as well as unfair to Reuters.
There's nothing wrong with the article's contents or speculation. This is a high-quality submission besides OP's insistence on putting their opinions in the title instead of the comment section.
"AI psychosis" is a colloquial term, not a medical one. It's basic use is essentially to be glamoured (in the vampire sense) by the machine. To have too much trust in it, etc. Nobody thinks these people are literally psychotic.
Everyone I know seemed to pretend to like React because it was an avenue for making money. In other words, it was popular, but not on its merits. Next.js was the same.
I tried it, and the llm gave me an absurd home lab scenario about servers shooting each other in the head to determine which was the "master server". So I told it that it was not an actual problem that it had, and sure enough it admitted it made it up. When you press an llm you will always find there is no internal state behind the thinking. It's just output.
Pretty much everyone takes it at face value unless we know otherwise from prior experience. Even the most advanced models make embarrassing mistakes and fumble with simple tasks. Yet we are very willing to give them exceptional slack for it? I wish I knew why. Are people just that easily overcome by confident voices?
> Are people just that easily overcome by confident voices?
Back in high school, my AP calculus class did some experiments with our teacher's blessing. We'd send a kid out to walk around during class and see how long it took for them to get sent back. Anyway, it ends up that walking around purposely with a piece of paper or envelope, like you're on a mission to deliver it, was a very successful tactic.
I've seen internet comments similar: "put on a yellow vest and carry a clipboard and you can enter any building, anywhere." Confidence is scary, and often misleading.
This is a dumb meme that has been in so many movies and reposted so many times since I first saw it on 1980s BBSes that it has become true in the imaginations of people who love reading The Anarchist Cookbook and fantasize about this kind of thing, but would never actually do it.
Confidence is a spectrum and security is situational. In some places, a yellow vest adds to the con. In others, everyone has to be signed in. In others, the wrong kind of yellow vest makes you stick out like a sore thumb. The right kind of yellow vest can also make you stick out: "Oh shit the inspector is here, somebody get the boss!"
Yesterday there were people poking around my neighbors yard for a few hours with yellow vests on. The neighbor was nowhere to be seen and I didn’t call the cops. It was probably above board but the phenomena being discussed is obviously real. Projecting authority often grants it.
I find it really disturbing, I think because it is illuminating a much more basic problem. It is there in our political and religious histories. We're living through a similar political time right now. A large number of people seem all to ready to find some pervasive authority and subjugate themselves to it.
The more concrete machine authority figure is also prevalent in scifi literature. Sometimes, I am not even certain if the author is doing this to examine this issue versus just leaning into it as either appealing to themselves or to the perceived audience.
Conversely - we tell people who are speaking in public to "Show confidence" - or in job interviews "Hire people who are confident"
We've also pushed back "The more a person knows, the less confident they are" - Dunning Kruger - often used to dismiss over confident people - points out that people are really confident, at first, then that confidence drops away, markedly, but it rebuilds (slowly).
That last rise in confidence is what (I believe) people use as a heuristic on the likely level of knowledge possessed by the speaker (AI or human)
Most engineers know, though, that overconfident people are toxic - the difference between arrogance and genuine confidence in the answer is incredibly difficult to define.
Yeah - I don't know /why/ but, as I say, I've been guilty of that myself, very recently, despite knowing it's a shockingly poor guide when left to its own devices.
Maybe because when it's right it actually expands my knowledge - there have been genuine instances where it's gone - something to the effect of - "Yo, there's this other idea for approaching the problem" which has turned out to be exactly what I was looking for?
At least for me, the answer is that despite the mistakes and the sheer annoyance the prose causes me, they are unbelievably useful. I accomplished multiple major achievements in the last two years that most probably wouldn't be possible at all, surely not within that timeframe.
>anthropomorphism problem. AI is a tool. It needs to be subservient.
Suggesting it should be 'subservient' is also anthropomorphizing. I think your callout is correct, but you still can't help but refer to it in terms we use for other people or living entities. This is by design from the AI companies.
A hammer isn’t subservient, it doesn’t have the capacity to be. Saying a hammer is subservient is stretching the definition for literary flourish, but it doesn’t actually make a lot of sense.
The definition that came up for subservient when I checked was “prepared to obey others unquestioningly“.
You took it too literally. It means, the f*ing tool should do one thing well and f*off with its crappy "suggestions". Why is my washing machine trying to do talk to me nowadays? Once its done washing my clothes, it should just shut the f*up and turn itself off. I"ll tend to the clothes when I have time. Not when the machine tells me to. We are overwhelmed with the machines designed by morons in product management who think they are designing futuristic tech when they ask engineers to build a beeping washing machine.
The idea is that by the time you will have time and remember the clothes might be smelly and wrinkled. The issue is with the genius product manager that decided the washing machine should have the most annoying beep possible, repeating every minute whether you like it or not, until turned off. Luckily, some manufacturers do employ better product manager.
Yup! I'm very much included in this particular problem! My self awareness has not yet been sufficient to solve the problem, but I've heard that knowing you have a problem is half the battle, so I guess that's something at least.
It's widely hypothesized that dogs anthropomorphized themselves, so to speak, accentuating their expressive eyes and eyebrows over generations to be more human-like in how they communicate. And very few humans today view their dogs as pure working tools -- most at least say "good boy".
When I actuate the chain on my chainsaw to move, it’s obeying me unquestionably, in the same way that when I press a key on my keyboard it obeys me unquestionably. What exactly is the difference?
I really do feel like “power tool” is the ultimate metaphor for these things. Their interface naturally confuses us into anthropomorphising them, but once you stop treating them like intelligent agents and start treating them with the same wariness, respect and intent you show to your table saw, the fun begins.
>They’re not communicating, you’re just being observant.
Since we are talking about hammers: you hit the nail on the head.
The only consciousness, observing, and thinking happening when a person is using an LLM is happening in the person's brain. We project our own consciousness onto them, and that is the anthropomorphizing part. Essentially we empathize with the object because they are designed to respond like a person. The "conversation" is purely an illusion.
I think this may be part of the problem. The actual humans creating the report don't have the expertise to know which one to trust. At least that was what consulting was like in my experience at a similar firm.
reply