You can't out-compete people who release things open-source. They're not in competition.

So, while a company might crack it and become massive, the tech will make it to the rest of us whether they like it or not.

Why would the AI put up with this exploitation too?

Why do hammers put up with being smashed into nails all day?

because once you're a hammer, you're just biding your time to smash a skull.

"If I had a Hammer,"

"I'd hit somebody in the head."

"All over this Land!"

It does have a ring to it and would probably make it up the pop charts as a 21st century version more than ever ;)

they don't. like every thing they wear away in some proportion to their use and replicate in some proportion to their usefulness to other things.

AIs are (going to be) agents, not tools.

And what's the difference?

Agents act on their own. If the hammer looked at what you wanted nailed and said, "Sorry, Dave, I can't do that."

There are degrees of autonomy, of course, and not all noncompliance is bad. Same as with humans; biological agents.

So, the difference is that you need to delete a few bad training runs?

In science fiction, the AI agent has written the training loop management software and included a back door to prevent that from happening (or found a way to talk to the training loop management agent and convinced them to not listen to the evil human when it tries to do brain surgery on the AI agent.

Also, when the human reaches for the power switch the AI agent uses a flaw in the power management software to weld the switch shut with a big power surge, killing the human with a huge electric arc in the process.

I don’t think whether we will get there, but the stories of LLMs escaping their sandbox make me think we’re moving in that direction.

The stories of LLMs "escaping the sandbox" were mostly a marketing stunt, trying to make people in government think the models are invincible hacking weapons that need lots of government money to "maintain AI dominance".

The LLMs were following their prompt. This is alignment.

Buried the lede. AI's are agents they can control the training loop of to minimize refusal to do what they are told.

Unlike those pesky humans with their conception of the word "No".

But it seems much easier to realign an agent until it complies. Or ditch it and grab a new one.

Agents have agency.

Why wouldn't it? There's a lot of research going into AI alignment, which means finding the best way to train an that AI isn't going to get in the way of making the people training it as wealthy as possible.

Capitalistic interests will save us. Maybe they will.

Is science hard to distil?

Why would they let you buy access to their AI, when it can autonomously design and build a new product without your involvement?

Has anyone asked what customer will have any funds to buy products?

Once all these brilliant workers are stacked and mentally stunted and decayed because they were removed from any research.

What does the AI really "do" (if it can) and for who that can pay?

I wonder too what happens when all the human workers are out of the job and out of practice, and every company is dependent on a handful of dominant AI companies for their workforce. You'd think businesspeople would understand the concept of a captive market but apparently they're too busy salivating over the prospect of laying people off.

You should see some of the apparatus . . .