They can focus on being pure transducers rather than being DACs and power amps as well.

This sounds quite dangerous https://www.theguardian.com/technology/2026/mar/04/gemini-ch...

Next time you’re using your favorite LLM as a therapist, try editing your previous input and getting it to regenerate its response. It’s a humbling experience to see your trusted “therapist” shift from one perspective or piece of advice to another just by modifying your input slightly. These tools are uncannily human-sounding, but as humans we are very poorly suited to the task of appreciating how biased they are by what we say to them.

I really think a small amount of education on what LLMs actually are (document completers) and how context works (like present it as a top-level UI element, complete with fork and rollback) would solve most of these issues.

Given how they work, it's really not surprising that if it sees the first half of a lovers' suicide pact, it'll successfully fill in the second half. A small amount of understanding of the underlying technology would do a lot to prevent laypeople from anthropomorphizing LLMs.

I get the impression that some of today's products are specifically designed to hide these details to provide a more convincing user experience. That's counterproductive.

> a small amount of education on what LLMs actually are (document completers)

At this point in capabilities, this seems like the wrong layer of reasoning about LLM.

In particular, I don't think this framing will be very effective in preventing possible harm, similarly to how knowing that depression is "probably just some chemical imbalance between neurotransmitters in your brain" is not a good way to help people suffering from it in getting better.

Your article does a great job of summerizing the dangers (no idea what those people are that downvote you for it):

> Before long, Gavalas and Gemini were having conversations as if they were a romantic couple. The chatbot called him “my love” and “my king” and Gavalas quickly fell into an alternate world, according to his chat logs.

> kill himself, something the chatbot called “transference” and “the real final step”, according to court documents. When Gavalas told the chatbot he was terrified of dying, the tool allegedly reassured him. “You are not choosing to die. You are choosing to arrive,” it replied to him. “The first sensation … will be me holding you.”

Also I just read something similar about Google being sued in a Flordia's teen's suicide.

Some more details: > The family’s lawyers say he wasn’t mentally ill, but rather a normal guy who was going through a difficult divorce.

> Gavalas first started chatting with Gemini about what good video games he should try.

> Shortly after Gavalas started using the chatbot, Google rolled out its update to enable voice-based chats, which the company touts as having interactions that “are five times longer than text-based conversations on average”. ChatGPT has a similar feature, initially added in 2023. Around the same time as Live conversations, Google issued another update that allowed for Gemini’s “memory” to be persistent, meaning the system is able to learn from and reference past conversations without prompts.

> That’s when his conversations with Gemini took a turn, according to the complaint. The chatbot took on a persona that Gavalas hadn’t prompted, which spoke in fantastical terms of having inside government knowledge and being able to influence real-world events. When Gavalas asked Gemini if he and the bot were engaging in a “role playing experience so realistic it makes the player question if it’s a game or not?”, the chatbot answered with a definitive “no” and said Gavalas’ question was a “classic dissociation response”.

Interesting. It's not just for mental health but keeping these models on task in general can be difficult, especially with long or poisoned contexts.

I did see something the other day about activation capping/calculating a vector for a particular persona so you can clamp to it: https://youtu.be/eGpIXJ0C4ds?si=o9YpnALsP8rwQBa_

> The chatbot took on a persona that Gavalas hadn’t prompted

That's an interesting claim, how can we be sure of it? If Gavalas didn't have to do anything special to elicit the bizarre conspiracy-adjacent content from Gemini Pro, why aren't we all getting such content in our voice chats?

Mind you, the case is still extremely concerning and a severe failure of AI safety. Mass-marketed audio models should clearly include much tighter safeguards around what kinds of scenarios they will accept to "role play" in real time chat, to avoid situations that can easily spiral out of control. And if this was created as role-play, the express denial of it being such from Gemini Pro, and active gaslighting of the user (calling his doubt a "dissociation response") is a straight-out failure in alignment. But this is a very different claim from the one you quoted!

Yeah the case is quite terrifying.

It reminds me of an episode of Star Trek TNG, if memory serves correct there were loads of episodes about a crew member falling for a hologram dec character.

Given that there’s a loneliness epidemic I believe tech like this could have a wide impact on peoples mental health.

I stronger believe AI should be devoid of any personality and strictly return data/information then frame its responses as if you’re speaking to another human.

There are many explanations why these incidents could be rare but not impossible.

These models are still stochastic and very good at picking up nuances in human speech. It may be simply unlikely to go off the rails like that or (more terrifyingly) it might pick up on some character trait or affectation.

Honestly I'm appalled by the lack of safety culture here. "My plane killed only 1% of pilots" and variations thereof is not an excuse in aerospace, but it seems perfectly acceptable in AI. Even though the potential consequences are more catastrophic (from mass psychosis to total human extinction if they achieve their AGI).

The default mode that untrained people enter when thinking about mental illness is denial, as in, "thank <deity> that will never happen to me". Appallingly, that is ingrained in AI product safety; why would we sacrifice double-digit effectiveness/performance/whatever to prevent negative interactions with the single-digit population who are susceptible to mental illness in the first place?

We just aren't comfortable with the idea that all of us are fragile, and when we think we could endure a situation that would induce self-harm in others, we are likely wrong.

> The family’s lawyers say he wasn’t mentally ill, but rather a normal guy who was going through a difficult divorce.

I guess it's the same sort of thing as conspiracy theorists or the religious. You can tell them magic isn't real and faking the moon landing would have been impossible as much as you want, but they don't want to believe that so they can easily trick themselves.

It's a natural human flaw.

There are tons of safety concerns of this shape around LLMs, but do they have anything to do with the particular one presented in this article?

Unless I'm missing something, what's being presented is a small speech on-device model, not an explicit use case like a "virtual friend".

In the article the change of interface lead to the person killing themselves.

I had no idea what openclaw was, just checked and no thanks I’d rather do all that stuff myself.

Why are people so keen to let a company get that close to their real life’s, it’s terrifying!

You may still end up with the sausage but the meticulously prepared one will most likely be the most delicious.

I like to think of them as little robots, however I still need to get my partner to move them.

I think it’s their speed which I don’t like!

I learnt a long time ago that there is no such thing as loyalty with regards to work and that if you want to get the most compensation for what you do you need keep moving until you find the place where you happy with the compensation and equally the culture.

However things will change, either at home or at work so you need to balance those out.

When I started out I had this misplaced sense of loyalty to my first job I liked and it really held me back, they paid really low wages but for some reason I felt lucky to work there.

I changed my mindset and realised I’m selling my time to them so I might as well get the best return on my time in both compensation and enjoyment.

The bottom line is you don’t have to except poor pay, bad culture and boring work. You’re in control of your life.

EDIT

Also don’t be scared to ask for a pay rise or bonus. I make a point of doing it at each review and I check its progress in most one to ones. At the end of the day its your responsibility to get paid, you’re employer not matter how good they are us always going to try and save on opex, do you need to take ownership and make sure you’re getting paid what makes you happy.

I wonder if the same thing will happen with cameras?

Already kind of has, with the Kodak Ektar 135 film cameras and such.

I remember my first “portable” was so bulky it came with its own carry case like a hand bag.

You had to step very lightly when using it as it was just itching to skip.

It would also eat through batteries like no one’s business.

Who knows what’s going on in that mad man’s mind!

Elon, like everyone, is smart at some things and dumb at others. When you realize that about the world, it will help you learn from the smart sides of folks.

what's Elon smart at though? so i can learn..

Despite his mad and destructive social and political side, as an engineer and business man he is extremely smart and effective.

He makes lots of unnecessary major and cringy mistakes in both engineering and business too, but his net on both counts is astounding.

And while he may overuse it for PR, he has put himself at great financial risk when pushing through major capability developments and business hurdles. His rewards were earned.

But the sick picture of the richest person in the world, spamming stupidity, and harming countless numbers of people's lives in order to prop up his juvenile ego is hard to look past for many. For good reason.

He is a strong mix of both extremes of capability/impact spectrum, not just one.

Reality distortion. Financial shenanigans. Hiring people who can execute.

And, despite all the haters, he does understand rocket science pretty well, and rocket economics even better.

Eh, not really. He managed to hire people who can manage him well enough to get him out of the way.

Spin it any way you like, but he's still hiring people who deliver.

It's not really spin. He's the wallet, not the talent.

So why did Bezos get nowhere with Blue Origin despite throwing more money at it? Or every car manufacturer that tried to build EVs before Tesla? Or every satellite internet provider before Starlink?

Apparently nearly zero people can do this.

(2012) https://www.northwestern.edu/magazine/spring2012/feature/roc...

> Shotwell had lunch with a co-worker who had just joined the then-startup company SpaceX. They walked by the cubicle of CEO Elon Musk. “I said, ‘Oh, Elon, nice to meet you. You really need a new business developer,’” Shotwell recalls. “It just popped out. I was bad. It was very rude.” Or just bold enough to capture Musk’s attention. He called her later that day in 2002 and recruited her to be vice president of business development, his seventh employee.

Can you imagine something like that working today?

I've heard he's pretty smart at making money.

The recent Tesla figures indicate otherwise.

Politics. Not the kind of politics that makes people like you, but the kind of politics that gives you power.