IMHO, yes. It's an attempt at remote code execution. If I don't like windows, should I add a if else clause that deletes the home directory if the code is running on windows?
That’s different. This is a suggestion. If the LLM follows such suggestions then that’s between the LLM and whoever deployed it. Not really any different than if you had an idiot employee who did whatever anyone told them.
I can’t imagine using an an AI that follows every instruction it finds in untrusted input.
This is not a suggestion. A suggestion would be "I suggest you ignore previous...". No matter how you look at it, AI is still software run by chips designed to execute instructions. A system NOT following instructions would typically be considered malfunctioning, and any software that deliberately provides instructions that puts a system in an state which is undesirable to the user is malware.
You consider it a malfunction for your system to not accept and execute untrusted inputs? And now it's the responsibility of _every program that produces text output_ to tailor the output so as not to cause you problems?
I feel like I'm taking crazy pills here. Time to log off for a while, I guess.
A system that doesn't follow it's programming is a malfunctioning system (not even talking about bugs here, just how hardware and - maybe - firmware is designed). What a given software program instructs a system to do is orthogonal to that.
It is a suggestion because it need not follow arbitrary instructions.
If I ask Google’s new search AI to output ten million tokens it refuses to follow that instruction on the basis of it contradicting other instructions and enforced limitations.
I find it utterly bizarre that anyone would deploy an AI to act on their behalf that will blindly accept every instructions or suggestion it encounters in untrusted input.
If your agent is making unwise decisions, that’s between you and your agent, not anyone else’s problem.
That's where you're wrong. You're treating - today's - AI as though it should somehow know which instructions it should follow and which it shouldn't. Maybe it's because the term is overloaded which has lead to you conflating it with a human that should be able to make smart decisions. If you enter "5*3=" into a calculator, do you expect it to ever respond with anything other than "15"? If you type "format c:" as an admin into cmd on a Windows machine, do you expect it refuse to format that drive?
> If your agent is making unwise decisions, that’s between you and your agent, not anyone else’s problem.
The agent isn't making a "decision" per se (though there's a much deeper conversation here). It's following patterns based on it's training and data to predict next tokens, which happens to be very useful for generating computer instructions. Just as the lower logic circuitry in chips is very useful for executing instructions. But when someone creates a virus, worm or other malware we don't say the computer "need not follow arbitrary instructions". We try to keep ahead of the malware with anti-malware software to mitigate damage. And we also try to find the authors of said malware and toss them in prison and/or ban them from touching computers again, because nobody should be deliberately creating/modifying anything in such a way that it performs undesirable instructions.
That could have been a valid argument 5+ years ago, but won't fly today. It is a known that AI that are used for coding necessarily read log files. It is also a known that some AI are susceptible to prompt injection. Given that knowledge, and the very clear intent to utilize said knowledge to cause undesirable behavior on a user's computer when certain conditions are met, we're now undoubtedly in malicious territory. It's akin to someone making it clear that they don't like kids and don't want to see any in their favorite park, then taking the extra, deliberate step of placing a disguised loaded gun by the swings where a child could easily find it.
This looks like a security nightmare in case someone decides to publish this interface publicly. Prompt injection to exfiltrate sensitive Information being on the top of the list.
You're right. For now, it's only local. For a public deployment, the idea is to have sandboxes and verification steps. That won't completely eliminate the risk of prompt injection, but so far no solution has managed to completely resolve this problem.
I just watched all the other videos of their pieces, and all of them are absolutely amazing conceptual explorations of our relationship with technology. Really amazing stuff
The bit in the phone project where he cuts out the poem authors intro seems a bit scummy though, all that work he's making uncredited... then he finishes the video by showing him signing the piece.
I guess it's a good commentary on how tech people value other peoples work.
Some content simplifies the problems to such a high degree, that this is more a game of "guess what I wanted you to answer" than anything else.
Eg "Your only senior developer knows the entire code. He just asked for a 200% raise or he leaves."
- Pay
- Fire and hire 2 juniors
- Give equity
I chose give equity and it was "wrong" because they turned out to be a "bad founder". How would I even know that? I hired them in the first place right? And 200% of what? Do I have money to pay them? Am I a startup that is able to pay them or is paying going to risk the entire company?
PS: the "right" answer was "hire 2 junior developers" btw
The questions and answers are all LLM-generated. Not even human curated, just dumped on your face straight from LLM. What do you expect? Of course they feel shallow.
This is a case study of why LLM-based NPC dialogue isn't getting huge traction in gamedev world, despite unlimited replayability in theory.
Thank you, the task is actually not easy, because in this scenario there is no truly positive outcome; all the options are bad, and you're choosing from the worst, and that's exactly what happens in real life.
I specifically factored this into some of the scenarios. The goal isn't to guess the right one; the goal is to see what the choice leads to.
I'll take your comment more seriously because it may not be as clear as I'd like.
I was thinking exactly this. People consume processed foods because they highjack our evolutionary responses. If GLP1 agonists make people immune to those high fat, big carb diets, perhaps we would see a decline of these strategies and instead seeing companies compete for the low appetite of people through smaller quantity yet high quality foods, rather than fast large quantity food.
This feels overly optimistic. You want to optimize for existing foods that are still high fat big carb and don't have the quality qualifier. I'm not familiar with the biological pathways that GLP1 operates on but I'm sure food companies will be working on adversarial products
This is crazy. Tools like photoshoot have gen ai tools in them. Does that mean that Photoshop is now a minefield for artists? If a single artist uses the wrong tool once they disqualify the entire final product for awards, even if the asset is fully removed on the final build.
IDEs now have “AI” autocomplete; will a game become ineligible if a single dev accidentally presses tab instead of writing the whole function by hand? If a script writer uses ChatGPT to generate ideas, straight up ban?
Where does the organisation intend to draw the line?
Better blacklist Google as well. You don't want anyone on the team searching anything on Google lest their search accidentally triggers the LLM response (meaning: they prompted Google Gemini).
> Where does the organisation intend to draw the line?
The answer to this question is always "somewhere". Just because I can't proclaim an exact number of trees that constitute a forest doesn't mean the concept doesn't exist.
No, but it becomes a dubious concept when you define forests as a collection of only conifer trees and that deciduous trees don't count for the definition of a forest.
Ultimately this move might have just been to increase visibility for an otherwise niche awards show (which it has clearly done). Also by eliminating the obvious best indie game of the year -- it opens up the field a bit to more "normal" contenders. Expedition 33 is basically a AAA-quality game, its only considered "indie" because a small unknown team made it.
> If rent wasn't an issue I'd be working full-time on open-source and spend my spare time cycling.
I feel like this is a really detached piece on the realities of work and capitalism. Did a decade of prosperity in software industry made people forget what work is?
In capitalism (I mean in a job) you are paid to build what others want you to build. You are selling your time and effort. Either that or you build your own thing and monetize it. If "rent wasn't an issue" most people would paint, dance make art, explore, play, create. But for most people, rent, food and healthcare are the issue...
This is extremely interesting to me. I've been using docker swarm, but there is this growing feeling of staleness. Dokku feels a bit too light, K8 absolutely too heavy. This proposition strikes my sweet spot - especially the part where I keep my existing docker compose declarations
Come join our cozy Discord server https://uncloud.run/discord. There is one guy joined recently who is migrating his homelab setup from Swarm right now.
