They are being trained to try lots of unlikely alternatives and to be persistent. This often works well when searching for security bugs or counterexamples to famous math conjectures.
But maybe it doesn't work so well when caution is required?
Yes, this discovery is surprisingly similar to the recent counterexamples LLMs have been finding for mathematical conjectures: a semi-novel construction, built on previous work, that feels like it was found with enormous search and an okay heuristic.
I feel like there is a pattern emerging regarding the type of novel discoveries LLMs are good at finding, but it will take some more data points to see if the trend solidifies.
Nothing but respect to TurnTrout for taking an action like this. The world needs more smart people who are willing to stand for what they feel is right, despite the pressures otherwise. Without that occurring more, our species is going to lose many impactful prisoner's dilemmas coming these next two decades.
This raises my respect for AI researchers a little bit too. I have often felt that the entire industry is pretty tainted to the core, and for better or worse that colors my opinion of the researchers.
Maybe I'm in the minority, but I thought it was gross to download pirated art for a student project when I was at Berkeley years ago. So it has been really sad to witness many of the most brilliant minds of this generation answering the siren song of disrespecting the collective effort of others to extract and resell residual value.
I'd guess TurnTrout doesn't agree on that framing, otherwise he probably would not have been at Deep Mind. But clearly he and I agree on other ethical positions; I am nothing but glad to see him stick to his principles here.
> I'd guess TurnTrout doesn't agree on that framing, otherwise he probably would not have been at Deep Mind. But clearly he and I agree on other ethical positions; I am nothing but glad to see him stick to his principles here.
FWIW I agree that creators should be compensated (but evidently it wasn't a deal-breaker for me joining). I think it's bad how little that has happened.
I joined GDM to work on AGI safety to reduce existential risk from AI. I consciously avoided work that would improve the raw capabilities of Gemini. When I joined, I made a trade-off and it's valid to disagree with my decision.
As your writing seems to allude to, life is an impossibly complicated series of trade-offs made with imperfect information, so I definitely wouldn't consider joining an AI frontier lab to be a priori bad. Even if I have some qualms with how the models are built.
If anything I was just trying to point out that, even with people we might disagree with, we deserve to show them recognition and respect when they make moves we do agree with.
But I doubt we disagree on much. Your writing is great, and it makes me sad to see this thread somehow disappear from the HN front page so fast. Either it triggered some internal flamewar detector--but without a flamewar--or someone didn't find it convenient.
I wish you the best of luck with your next endeavors, and look forward to seeing the value you add to this world.
>The stereotypical activist action is to make a petition. But Google had already ignored a large petition on this issue. Plus, Google’s executives likely hardened their company against stereotypical organizing tactics. Sit-ins, strikes, even a mass of Google engineers quitting: I deemed all of them ineffective (if I could even pull them off).
Hmm, links for what? My student project? The decisions that face humanity that are pretty clearly modellable as prisoner's dilemmas?
Otherwise not sure what I could cite--I would assume most all on this forum know that AI is trained on the works of other people, without their permission to do so. I guess you could disagree with my framing, but I wouldn't think this requires a citation.
I think maybe my writing wasn't clear, and it sounded like I was referrencing some well known thing that happened at UC Berkeley. I have edited it to read more cleanly!
It wasn't clear to me it was _your_ project nor who pirated it. I was under the impression it was a well known scandal from your original, unedited comment.
Apologies if my comment sounded hostile; I was just asking for a clarification/more information on it.
There is a very real contingent of people driving AI who genuinely believe this. And it is this belief that allows them to avoid their cognitive dissonance around the negative externalities of their work. For the environment, and really for the future of humanity too.
The /s tag weakens the post. Glad to see I got downvotes--hopefully a few of these incredibly wealthy people in positions of power had to, for even a brief moment, consider their beliefs.
I agree with you--but just fyi I think "antifragile" is generally used in the opposite to what you mean. If I'm remember correctly Taleb has tried to coin it as a precise word to describe the inverse of your phenomena: Systems that prioritize robustness over optimizations, and therefore can handle stress effectively.
The point is that stress makes some things stronger. Stress doesn’t make a tea cup stronger because a tea cup is fragile. Stress makes a body or organization stronger if its not too much and the system can adapt. A body in zero g gets sick, but is healthy in 1 g.
reply