I'm only running a few agents in parallel so far, but I don't let them write or read anything w/out my approval. This slows things down quite a bit, but it means that I have exactly the opposite of your problem of stopping them.
I've only started really thinking about developing my own harness, but one of the reasons would be to implement some mechanism where I could give the agent a "turn budget", say, 5 turns, when I can clearly see what the agent is going to be doing for the next few turns, without my needing to allow each turn to go forward.
But it sounds like the same thing would be useful for you as well, just in the other direction.
Could you say more about what "certain points" you're trying to get them to stop at?
SQLite doesn't have system-versioned / temporal tables, but a quick search turns up a fairly straightforward approach. Instead of overwriting existing rows, always write new rows with timestamps. https://www.ohnekontur.de/2024/02/19/unlocking-time-harnessi...
I'm thinking now of the hoops you have to jump through to edit a package.json file to update your dependencies, and thinking yeah, what if you could do: "UPDATE dependencies SET version='1.2' where name='madlib';"
It’s interesting to think of the LLM as a kind of “mirror”.
It reads to me like this person is engaging with the tool in a fairly hostile manner, and the LLM is like, “so you want to play it like that, huh? Well, let’s play it like that then!”
The language that you use to prompt the LLM is the language you’re going to get back.
Not necessarily. Opus used to become overly apologetic and started to second guess everything it did after getting insulted or "talked down", as recently as the beginning of the year. It was a surprisingly effective way to get it to behave for a while lol. I haven't seen that behavior since at least 4.8 though.
My personal opinion: all output from LLMs is hallucinations. The idea that it’s “right” or “wrong”, when stating factual information is really in the eye of the beholder/user. What year did astronauts land on the moon? — is a question that may or may not have a factual answer depending on your own beliefs. Philosophically, you would need to also define what a fact is, or what “not hallucinating” is, to define what a hallucination is. My understanding is that this property of the LLM architecture is innate, and until we have “world model” LLMs, or some other model that reasons from first principles, instead of the current “guess the next word” model, this isn’t going away. Just don’t rely on the for facts.
Is this distinct from the hardware in a land mine making the decision to explode shrapnel through a human being without any human in the chain except the dead guy.
Worth noting that both cases indirectly involve the humans that designed devices and the humans that made the placement and trigger condition decisions.
Further:
> AI drones are being used to autonomously target and kill targets by the Ukraine using technology they have been given.
are remote operated, they allow defenders cover while themselves being out and exposed.
However were they altered to autonomously fire, that would be on the basis of pattern matching in the visible and infra red spectrum - shoot at all hot blobs.
That's more of a trigger threshold setting issue than an LLM hallucination issue, and the danger is on par with any weapon system on auto fire, you really shouldn't approach such things until they are put in a safe off state or have exhausted ammunition.
I guess what I'm really asking is whether or not it's moral and ethical to use AI that targets autonomously? My understanding is that it's being used that way, but I can't back that up and I'll take your word for it if that's not the case. But it is certainly a plausible scenario. Everyone invested in AI seems determined to put it in everything because they need consumers to want to pay for it, and right now they haven't figured out a way to earn back from consumers the over a trillion dollars that has been put into the AI bubble. I don't have much faith that the people with billions invested don't want return on their investment, and I don't think they care about the consequences necessary to get it back.
> moral and ethical to use AI that targets autonomously?
First point, vision systems have been used in industry to look for misaligned labels, incorrectly folded papers (in high speed paper presses), wrong items on high speed conveyor belts etc. for thirty odd years now - they have issues that a very distinct from LLM 'AI' hallucinations.
That's nomenclature out the way.
Landmines are indiscriminate, they trigger on any weight or pressure over a threshold.
A vision based Felixer, by contrast, only triggers on cats (well, almost always only) and leaves bilbies and bettongs to walk on by.
That's an improvement over landmines.
The crux of your issue here might be the morality and ethics of establishing human exclusion zones within which all humans are highly likely to die.
These historically are created with rapid patterned artillery fire, butterfly mines, Napalm, indiscriminate criss crossing machine gun fire, etc.
Now there exists an option to use drones to kill all humans and leave the horses and cows alive.
Is it your concern that a bad vision threshold might kill a horse rather than a person? (Likely not)
Would you prefer an area to be napalm'd and agent orange'd back to dust?
Also, why do people keep talking about landmines? I'm talking about software, you are talking about dumb mechanical things from WWII. This isn't a philosophical question about methods of warfare from the past, but the software that controls the dumb hardware.
I, a person, singular, mention landmines as they are a hardware device that are designed to trigger action on a threshold.
Felixers are computer vision based devices that are designed to trigger action on a threshold.
These are actual real world objects that Bishop Berkeley can kick, not vague philosophical questions but actual engineered devices that kill and are in use today.
I don’t trust their outputs yet enough to not review everything myself. But I’m happy to be. a QA bottleneck in that way. What I want now is orchestration to get them running in parallel. I’ve looked into Yegge’s beads, but I want something similar that is far simpler, fewer sloc, and database-driven rather than json-file-driven. Probably need to build it myself with a graph database…
I mean, it does feel like we could pause here for a few years, just to give everyone time to adapt to what the current models are capable of. We've seen back-sliding with model releases (anecdotally, but maybe there's published research?) as well. A new model isn't always a "better" model, for a particular task.
If these companies (and countries) believe that they're in an "arms race" with LLMs, then it may be extremely sane to have regulators step in and let all the engines cool down for a bit. Seems like international agreements to pause development will be necessary as well, otherwise the US companies will just say, "but China ...".
Does anyone else here feel the same way? I think most folks default to a kind of cliched, "you can't stop progress" stance. I'd be happy to see more discussion on this topic.
Maybe we could find agreement in the idea that the USA and Russia agreeing to halt nuclear weapons testing was a good thing. Seems like there's a bit of precedent here for cooler heads to prevail, which benefits us all.
I’d say the problem is that, at least in the US, the government was conceived as a democratic republic, but private enterprise wasn’t required to follow the same conceits, and so all our corporations behaved more like small dictatorships. Now that our corporations are megacorporations, and the government has lost the upper hand, the common people are at the mercy of these dictatorships. There’s no reason for Apple or Google to cede any control that they’ve already established, and we only see them grabbing more and more control as time goes by. Until the government regains the upper-hand, and begins to work again on behalf of actual people (with heartbeats), the firmware in our phones will continue to do the bidding of its true owner. Or if maybe, there’s a revolution or something. Maybe we could take public ownership of the phone lines and cell towers. Fat chance, huh?
I've only started really thinking about developing my own harness, but one of the reasons would be to implement some mechanism where I could give the agent a "turn budget", say, 5 turns, when I can clearly see what the agent is going to be doing for the next few turns, without my needing to allow each turn to go forward.
But it sounds like the same thing would be useful for you as well, just in the other direction.
Could you say more about what "certain points" you're trying to get them to stop at?
reply