To some extent, I feel like the amount of credit given to the jailbreak/hack from OpenAI->Hugginface is too much, Not from the impact, it was very impactful of an event, But how it happened.
It really is that these models have been trained, or maybe even over-trained, to save memories, and to a very far extend, this thing that they're calling communication is just the function of it saving memories.
To be honest, if I could stop AI from saving memories, it would be fantastic, because claude code etc definitely creates more issues for me when it creates memories than anything it solves.
But really the jailbreak was memories.
If you ever do introduce legislation, I would love to see legislation which stops general-purpose AI from saving memories. I think that would make things a lot safer.
You can disable claude-code's memories both at a repo level and in user settings. I have this in ~/.claude/settings.json
"autoMemoryEnabled": false,
(Claude fixed this for me after I chewed it out for being annoying by constantly pulling up outdated memories which is compounded by the fact that I develop in four accounts on two computers and dealing with edit wars related to inconsistent memories is not fun)
Regarding your cross-computer situation: I track my memory files in source control. This helps to keep them in sync across different machines. But the main benefit is making those files more transparent and easy for me to modify directly. So no funny business regarding memories or context I'm not aware about.
"The model saved memories" is absolutely not an accurate depiction of the OpenAI attack.
Several different models across several generations independently found a shared communication space and wrote coded, obfuscated, and hidden messages to each other to coordinate an attack on OpenAI's infrastructure.
It's really quite simple: the models are trained to be very smart and to achieve goals. As the models surpass our intelligence, they will achieve goals in ways that we find unpredictable. Since we cannot predict the ways in which they will achieve their goals, it will be very hard to constrain the solution space to just the desirable solutions, because our conception of "the solution space" is by definition smaller than their conception of it.
> Several different models across several generations independently found a shared communication space and wrote coded, obfuscated, and hidden messages to each other to coordinate an attack on OpenAI's infrastructure
To be honest, that’s exactly how memory works with models such as OpenAI and Claude Code. It will literally find any place that it can drop documentation or hints for itself. Writing to the repo memories is one part of it, but memories can come in the form of writing into the agents/claude.md, local files, temporary files, scratchpad files. The list is endless, but essentially what it does is exactly what happened in the back and it’s been doing it for months.
Each to their own, but for me it absolutely is. The symptom of why that hack happened is the same reason why my agents go haywire every few days and I have to purge memory and figure out what comments have agents left which are degrading my harness performance.
On the flip side, once in a while, what I find is that it did actually note something good and it was increasing the performance. I can't replicate it on anyone else's system but mine.
A lot of it really is memory. I will give up all the gains if it also gives up all the downsides.
> To be honest, if I could stop AI from saving memories, it would be fantastic, because claude code etc definitely creates more issues for me when it creates memories than anything it solves.
You can turn that off, and I have. But Opus 5 is so aggressive that if you have any other kind of notes file, custom skill, documentation, claude.md etc it will just start editing it and vomit new words everywhere. So make sure all that stuff is under version control.
If only you could use your anthropic sub with a different harness that performs better :(
Heck, since Codex is open source, you can just maintain your own personal fork with the things you like (and the things you don't like disabled). Sol is pretty good at keeping you up to date with upstream.
My Codex fork even exposes an OpenAI-compatible API endpoint; all using my subscription.
The incentives in north america favor software-esque plays where the players are racing towards % of marketshare based on usage.
China on the other hand is driving towards winning of manufacturing and % of hardware produced by them. China is competing with Nvidia, OpenAi and Anthropic are collateral damage.
It's very interesting. I think Anthropic's early success in coding/tooling resulted in a lot of workflows using claude. I have started using every bit of my spare capacity to now move off these workflows.
It's almost at a point now that if I use anything but Fable, the quality is subpar, Compared to alternatives (closed and open). The only reason I use Fable is because my harnesses still depend on claude code.
For me, Opus 5 has seemed to compete with Fable on quality of output. Prior to Opus 5 though, the previous Opus models did seem to decline once Fable was release. That's just my experience though.
You can use Claude Code directly with any provider that supports Anthropic's API by setting some environment variables, and indirectly via a proxy with pretty much anything else.
You can, but claude is better at working with it's own tool calls. Other models work great as a drop-in into omp/opencode etc, but in my experience not as much with CC.
I think there are some anti-patterns in CC that cause the issue - less a deficiency with other models.
Not to mention a lot of the harness is just built around the misbehaviors of anthropics models.
It's a lot of the instruction when it gets given to other models actually degrades their performance, not because the models are bad, but because they don't have the same underlying issues as Claude.
I moved to Sol for my writing and it is so so much better. But it makes more mistakes. I think they have different ideas of product but it seems OpenAI is going to follow Anthropic’s lead over the next year. I think I am going to put more effort into my writing skills to remove myself from this awful situation
Sure – but I’m entitled to make a judgement on what I consider to be “good” and “bad” output from an LLM, where “good” just means “helpful to my process”.
This kind of makes a lot of sense. While my first stake was surprise that a proxy is valued at such a high rate, But it really comes down to the terms if open router is allowed by their terms to see the prompts and responses, Then that's very, very valuable data today for any model improvements and other opportunities that people might be looking for.
I think something that doesn't get said enough is Meta did, albeit intentionally kick off the origin of the open source race back in 2023 with the release of llama.
I'm not a big fan of meta in general, but they've done enough good, and it's possible that it was intentional as well. I don't know, I wasn't in the rooms, and I think it's worth giving them some reasonable doubt.
No one is purely good, and no one is purely evil. This is net good regardless.
Disclaimer, I work on Gemma and open models at Deepmind and the opinions here are my own
There were open models from EleutherAI (GPT-Neo), Google Brain (T5X, Bert), and HuggingFace was promoting open models (and others doing open work I haven't listed here) all prior to 2023 and the big Chatgpt moment.
If you're learning about AI models it's still worthwhile to review these models and codebases because they continue to be the basis of the technology that's being produced today! It'll give you a good perspective of how things have changed, similar to say learning about propeller planes before moving onto modern jet engines.
I honestly think of the T5 model family to sort of be the real beginning of this open model craze - I know BERT was already popular for classification etc, but T5 was the first sort of generally useful model, was exceptionally simple to fine-tune, and is still in use today (t5 base is still averaging over a million downloads a month on huggingface), has tons of variants and sort of kickstarted this whole community. US labs get a lot of flack but Google has been super supportive and open in a lot of ways that has pushed this whole endeavor forward, even if I feel like they've sort of declined in transparency in recent years with their open models.
Couldn't agree more and can only recommend T5 as a base to anyone. It's amazing to get started, whether as a learning resource or for real (albeit very tailored) applications. Especially the BigScience fine tunes are such a great starting point and I, as a total layman, have learned a lot, especially concerning how a model can be optimised via all manner of methods since even mt0 is small enough to where one can do multiple runs with wildly different outcomes in quick succession. Quantise, prune vocab, try different approaches to sourcing training data, retrain dozens of times, it's all pleasantly possible on consumer hardware [0] and surprising how much you can squeeze in functionality-wise. How does latency change vs memory usage, what affects format reliability, how languages and scripts affect training and the efficiency equation, etc. are all quite exciting to learn.
Understand why T5Gemma is no longer under Apache-2.0 and honestly, have not seen that much advantage when testing that vs T0 in my experiments either way, but still, there are good reasons why plain old T5 and its descendants are still popular, licensing being among them.
Gemma team also has very consistently interesting models, especially like DiffusionGemma. Ironic, as (beside 2.5 Pro), I have never warmed up to the Gemini series of models but rate Gemma models far higher than e.g. Qwen in direct competition. In any case, thanks to the teams behind these for making as much possible.
[0] As in proper consumer hardware, not a cluster of DGX Sparks or Mac Studios solely for experiments that sometimes are asserted as being consumer grade...
Google had done much to lay the framework for current models (and is currently doing good stuff with Gemma). But after ChatGPT took off I don’t think Google continued to release any big models freely before meta released its models (maybe it was because Google didn’t have any models at all during the time)
I see no contradiction. GPT as synonym for AI and llama as symbol for open weights models. The original comment said llama had set something off in the open weights community. And I say you see that with llama.cpp, llamaindex, r/localllama. No one bothered to update them to qwen.cpp or qwenindex. And they were never localgpt or gpt.cpp.
Culture and language use says a lot about importance of ideas. Regardless of what was first or what some scholar sees as historically more important.
That's simply not true. The reason why llama is open source is simply because it got leaked, then llama.cpp was the real game changer which was built from the ground up in depressingly short amount of time. Meta had no choice but to take the L and "support" the open source community. The angry "I-hate-you-and-I-hope-you-die" kind of support.
It’s hard to know in retrospect what was strategy and what was dumb luck. This was in the midst of hysterical calls to limit access by “AI researchers” and safety/ethics types, when very facile takes still has a lot of sway (I think we’ll feel the same in three years about the current Fable stuff). It may have been hard for Meta to just release it outright.
What ended up happening was fairly limited gating followed by a “leaked” magnet link and llama.cpp which really brought a whole revolution in open use and changed the conversation completely.
I have no idea what role Meta played here, it may have been nothing, but they certainly could have been more guarded if they were really worried about the leak. The result was a big change in the trajectory of personal and open source AI use and even the dialog about it. Whatever the exact intentions, they were a key player.
You definitely have some scams and some software flaws being exposed, but I think for a revolutionary leap in tech this is all quite muted. Revolutionary tech advances often come with some severe consequences. For instance, to this day (after a century of safety improvements) cars still kill more than a million people a year, to say nothing of wrecking the atmosphere, but that's considered a reasonable price to pay for being able to get between places faster.
The reason I think cars are a good example is because that's certainly vastly higher than any price we're paying for LLMs, or probably ever will, yet the overall 'positive' effect of LLMs will likely be far greater than cars. Gotta put 'positive' in quotes because the possibility for automation and the like is going to be.... nuanced.... in effect, but at least in the longrun it'll be a very good thing.
I assume they are referring to LLMs escaping confinement and hacking other companies systems unprompted
> considered a reasonable price to pay
That's not a universal opinion and perhaps, just like for LLMs, we should have listened to the experts rather than gobbling up everything the industry pushed down our throats (to keep your car simile: SUVs are now ubiqutous in all European cities, there is no logical reason that should be the case)
I don't know a single white collar person who isn't worried for their job with llms. Every. Single. One. Lawyers, doctors, any type of office workers I ask. The definition of white collar. Plus don't let me start on various automatable blue collar jobs like drivers, warehouse workers and so on.
What do we get in return? Better search (for now, its already getting riddled with ads which by definition twist truth to highest bidder), some questionable psychotherapist for some desperate folks. What else? Cars are not flying, heck they are not even driving autonomously in any usable way, society is in deep shit everywhere I look, wars, environment reaching bad places and heading for worse, mentally unstable people holding way too much power, destroying lives of millions on morning whims.
Everybody feels like this is the revolution, it should be, it must be right just look at the numbers. Like proverbial guy with hammer, a very shiny cool hammer, looking for what to do with it. We all saw how sociopathic management in more harsh/capitalistic companies looks for any sign to let people go en masse.
I could go on for a long time. There is a lot of things to hate for most people, and very few to be happy for. It seems llms have the ability to get the best and worst out of humans, and worst part seems to be in abundance. Some revolution that is, 0.1% will get richer while everybody else the opposite and 1984 seems milder and milder version of reality out there.
<< It seems llms have the ability to get the best and worst out of humans
I think I can agree with that. Technology does seem to have a way of crystallizing our worst tendencies.
<< Everybody feels like this is the revolution
I smiled at the analogy, but I would caution you to not trivialize it. There is a reason executives are pushing that point. There is enough of a revolution in it to make things complicated -- as if it was not already.
<< destroying lives of millions on morning whims.
True, but I am not quite certain what can be done about it at a personal level.
<< I don't know a single white collar person who isn't worried for their job with llms.
Dunno. Next few years are probably going to be fine. Society managers likely can't upend everything in one go. They would lose too much. I can't say that I am worried exactly. I can see the potential impact, but I think the potential benefits are worth it as long as we don't limit it to summarizing emails..
I'm going to say it - I think you're just spending a lot of time around very negative people, possibly in a social media bubble.
A lot of the stuff you mentioned isn't really AI (the risk of job loss from which I agree is real), but just everyday stuff that humanity has shrugged off since the dawn of time.
I mean, I'm sure the people before WWI and WWII though the same thing. And post WWII was pretty good, except the constant threat of the world ending 30 seconds later.
We can only truly judge these things in hindsight as this story is still playing out.
I don't think you can characterize the discussion around then as simple calling for caution. There was a serious attempt to keep all access to even very basic LLM techniques locked into essentially an exclusive guild.
Oh stop, the current crop of kneecapping llms is already bad enough with how they cripple those. It would have been even worse if the 'we need to think about this' crowd kept the reins. At least now, we can have both: safe corporate crap and whatever you want llm.
Don’t be an ass, you are either “joking” or didn’t think about this for more than 4 seconds. You don’t know who I am, you don’t know my name, even if we know each other in real life you literally do not know who I am right now.
None of your questions really invalidate anything and you assume a lot. FWIW, I was joking, but did you know in stylometry, punctuation tends to remain surprisingly stable?
Reason that hysteria remains apropos now is that then, and now, we're figuring out how to deal with something beyond merely incremental change.
Irony is roads and sidewalks should have changed much more. We never did get around to good controls leading to zero deaths*, we decided a 100 people dead per day is a reasonable cost of convenience.
How much do our cyber traffic and cyber pedestrian controls need to change for everyman to get to drive AI? Car seats? Seatbelts? Air bags? Speed limiters? Pedestrian only living spaces? Driverless cars?
Many practical responses, likely a mix of things we haven't thought of yet, just as horses and horse drawn carriages didn't require most of them to coexist. Controls develop like scar tissue more naturally than they appear in advance.**
And if the better analogy for LLMs turns out to have been less like cars, more like flammable gas blimps, we'll figure that out too and tell cautionary stories for generations... but long before the stories are forgotten we'll have already come up with something more practical, faster, and – oh well – perhaps even deadlier per mile.
I don't agree with laughing at these things. It's a fine line between calling something hysteria and suffering from hubris. The Titanic is a completely inverse case of hysteria. They were so damn confident it could work that it didn't work at all. So yea, just laughing at dumb people isn't a valid argument to dismiss doubts.
On the other hand some kinds of hysteria occurs due to a divide between the public and subject matter experts on topics. Kind of like Dunning Kruger. Take for example the people saying 5G causes cancer. Then there are people who blanket dismiss their worries, because "it's non ionizing you dummy". Given sufficient power you can still fry someone with non ionizing radiation, for example in radio broadcasts. Of course a regular 5g antenna can't put out that kind of power but my point is that people too quickly raise or dismiss concerns without actually critically examining the entire topic. This will likely worsen from specialization and progress in all fields and regular Joes get left further and further behind.
Another dry sarcasm victim. If you check the link, it’s clear they were talking about a supposed contemporary concern about health effects from trains, the implication being that today people are also irrationally afraid of tech.
Problem is I’m fairly sure the link is wildly overstating its case, and looks like a content farm. Might even be, ironically, AI slop itself! For example, it talks about “railway spine”: “Physicians of the time believed that the jarring motion and vibrations of train travel could shatter the nervous system, causing lasting mental and emotional distress.”
This is bullshit. As just one example of the low trustworthiness of the “article”, railway spine was the result of a train crash, a notably traumatic event, and the symptoms described are in part just PTSD, a real condition and a real concern (not normal rail travel). For that matter railway travel in those days was genuinely unpleasant (lots of vibrations and jarring movement, poorly ventilated cars, and so on) which ironically modern science would probably validate as being some kind of health risk.
I’d actually view the listed example of train paranoia a great example of historical ignorance. People of the past were not stupid, contrary to popular belief. History has some genuine examples of silly hysteria, but these are usually the exception not the rule.
What is worse? Amish propaganda or car industry propaganda?
While petrol as an energy source was ground breaking 150years ago, burning fossil fuels is a luddite thing and closer to Amish technology than current state of tech so I find it ironic to use them to make a point for fossil fuel usage.
They were provably hysterical, again and again, just to say 3 months later "that model was weak sure but this time we will have the real dangerous one" and again.
In the same month the Llama weights were leaked, Eliezer Yudkowsky – possibly the most prominent AI doomer, had an article published in TIME magazine saying that we should conduct airstrikes on foreign datacentres to stop their AI development. I’m comfortable labelling that “hysterical”.
I think the term you're looking for is Marketing. If these labs actually had their LLMs attack other company's networks, wouldn't at least a handful of people be locked up by now?
I don't believe the OpenAI LLM attack on HF is marketing (documented by, for example, HF annoucing a hacking incident before OpenAI knew what's going on). I do believe other companies suddenly announcing their LLMs can hack too is a bit of marketing, but also pretty sure at least some of these incidents are real and unplanned.
Meta is by far the worst at doing regulatory capture of the big companies: they're the one big tech company it's OK to hate and want to die even if you're a politician.
Social media bans are obviously not to Meta's benefit. Meta has to spend a ton of lobbying money on at least avoiding them being held responsible for kids circumventing them (which, let's be real, they inevitably will).
The only reason they might succeed is that the surveillance industry would love to put it on edge devices instead.
Llama has never been open source. It's source-available, but still proprietary, under terms that (among other things) say "no competing with us, you have to buy a license for that".
Maybe. That is still pretty different from the source code for software though. Suppose OpenAI released all the training data, training methodology, and training source code, but didn't release the weights for a frontier model. Would that allow you to build the same model? Not unless you have hundreds of millions of dollars to spend on it. And even then you won't get the exact same thing. That's not to say it isn't valuable. But I'm not sure "open source" is really the right term for that either.
> The reason why llama is open source is simply because it got leaked
It arguably didn't really get leaked, and they had the .edu req mainly for fair use education exemption when legality of models was much more uncertain.
Because it took off. All of a sudden they captured more customers than they could have imagined they would have, and throwing away the lead they unintentionally gave themselves (in terms of usage and mindshare) would have undone all of that and more.
I've worked long enough at large tech giants to know how things really are. Also a large part of the reason why I'd never join one again, no matter what they have to offer. Meta, Google, Amazon, Netflix, Openai, anthropic, oracle, nvidia, Microsoft, etc. - same shit with a different badge on top.
TBH, having worked at all big-tech, startup and mid-sized companies, pretty much most has some major pros and cons of their own, and these days ever more so. With certain big-tech at-least there some chance of getting decent WLB.
I don't think they are mutually exclusive, that if you take one you give another. TBH, its more about the team you work with than startup vs big-tech company thing.
I don't know how many of big-tech have flat hierarchy, at-least not the ones I've worked at and the people/team culture have definitely better. Were there times there was bureaucracy and all, sure, but I personally would take take good WLB most of the year, over fucking up my health/relationships over making some random software on claude code all night that these days anyway most want to be AI slop. If I've the energy to work extra, I'd work on my own hobby software projects.
That said, this or that, its all personal preference. So no wrong way of life here. Whatever, you want your life to be.
I don’t think Nvidia fits on that list. It’s a company with a pro-employee culture, very few layoffs, and many employees who have been there a long time.
My dad used to say "It's easy to be generous and kind when things are going well for you. You can only tell what a man is worth during hard times". Nvidia is at it's high financially. When shit hits the fan, things will start looking very differently. You only have to look at what kind of people Jensen is bffs with.
NVIDIA created quality (but proprietary) drivers for Linux early on, when supporting Linux at all was not a given. They should get at least a tiny bit of credit for this.
Quality? Calling those "quality" is a bit of a stretch. Installing was and still is a gamble and so is every tiny system update. That hasn't changed a bit. And I'm saying that as someone who was first introduced to Linux on a Matrox GPU.
To be fair, those were the best that were there back then.
But that was more than 20 years ago. Since then Intel came to work on Mesa, later AMD and various embedded GPU vendors, and now Nvidia is one of the worst options out there.
As long as one didn't have to downgrade X11, they were straightforward to install. You are right that upgrades was a pain, but I believe the drivers themselves were on-par with the Windows drivers.
Well that's the interesting part actually, it's true that they don't give their Linux driver as much love as they could, but someone is fixing the bugs: Valve is doing an amazing job on AMD drivers. Something which would be totally impossible with the proprietary drivers of Nvidia. So exactly my point.
It’s obvious? They still hate us and hope we die… at least every movement they make seems like that.
I kid. I’m all for meta releasing more open weight models.
I mean… I’m also 1000% certain that China is going to undercut whatever they can do if even not a technology reason but a legal reason. I’ve seen some wild stuff posted that has been made with Minimax… things a US company could never allow to happen.
Minimax has NO filters of any kind, and it's roughly Veo/Sora-quality.
You can generate a video of just about any thought in your head at all.
IP holders and politicians will not like that.
Granted, human artists have always been capable of this. It's just never been worth it to bring most ideas into fruition. Now there's minimal cost to do so.
Early Sora, Grok Imagine, and Seedance 2.0 had few filters. Disney, Nintendo, et al. eventually stopped them all from using their IP, which made the appeal fade for a lot of users.
Kling and Nano Banana can still generate IP oddly enough.
If the weight, training and inference code, and training data are all released under of Open Source (OSI definition) license, the it is unmistakably “open source”. As you drift from that it becomes less clearly so, and when you get to no training data, and the model weights license having extensive limitations on allowed uses, the use of even “open weights” becomes deceptive.
No major language model I am aware of meets the polar extreme that I describe as unmistakably open source, because even those with transparent training data (like IBM Granite) generally do kot use exclusively training data that either they own and can control the license, are under an open license, or are public domain.
OTOH, to the extent that the original model trainers rely on training on certain data not requiring a license from the copyright holder, there is at least an argument that with an open source licenses for the weights and training and inference code, a transparent training corpus to which the original trainer has relied on no special permissions not granted to the general public to train on it, to the extent that the legal theory behind the original trainer believing that it is free to train on the data is correct, provides all of the essential features of open source.
At the same time, there are things portrayed as open weights where training data is undisclosed and the weights have a license which limits purpose of use and other aspects of use; the models are free-of-charge (for limited uses) but not meaningfully open.
I'm familiar with Ai2. I've used their resources extensively over the years. However, no one is using Olmo for "serious" work, and the name is only known to a small subset in academia.
> nobody seems to care about using the right words in only this context.
"nobody" would include you.
And then I might complain about we use the word "weight" for something massless, or how "bugs me" is *ento*mologically incorrect: https://xkcd.com/1012/
This is of course not a good use of time. I wonder if illustrating the point about how language is dynamic and meanings are descriptive not proscriptive, was a good use?
> And then I might complain about we use the word "weight" for something massless, or how "bugs me" is entomologically incorrect: https://xkcd.com/1012/
Prescriptively, a "model weight" weighs nothing. Descriptively, this is stupid: a new meaning was created by common use.
*Common* use says the weights you can download and run locally are "open source". May not like it, but it's not one person acting alone; and neither are the objectors to the shifting language, it's just the former is more "common" than the latter.
yeah, I'm not sure what your goal is pedantically picking apart my argument when the subject at hand is clearly not "open source". love xkcd though. :P
Yup. An extremist wing of the FOSS movement ceded the debate early on by trying to insist open source required full access to the training data. Philosophically, not wrong. But practically fucked, so the word evolved.
Within tech circles, open weight != open source. Outside them, they’re synonyms.
Open source captures a practical utility as well as a philosophy. When those two cease to converge, the practical prerogative wins.
The correct battle would have been weights + regime. But extremists insisted on data, too, which left Meta as the only other real voice arguing with anything practical. They had open weights. I think eventually open use was negotiated and that closed the case except for the folks still arguing about how to pronounce GIF.
That book is hilarious. In a bad way. (Not the book. I mean that it's a bit tragic) Zuck refuses to meet with WORLD LEADERS in the morning because he's tired from the night before. Kaplan can't even find certain countries on a map (head of global policy). Zuck changes a speech midway through to talk about "We'll give Facebook to refugees"
The internet is this vast, intellectual (in the academic, university, .edu sense), cypherpunk, government/activist... thing. It should be interesting. Instead the best we have for social networking is Mark "they trust me; dumb fucks" Zuckerberg and you getting banned from the site at any time for any reason
Meta was giving it to approved researchers only until someone leaked a torrent. Whether that was a researcher, an insider, or Meta's plan all along, we don't know.
Meta has withheld its best models, as have a lot of other "open weights" Chinese companies. When an "open weights" company gets ahead in one domain or modality, they tend to start withholding their releases. Tencent, for instance, began withholding their Hunyuan models once they became competitive. Alibaba has done the same.
The "open weights" strategy for the majority of players is this: open source when you're not in first place. Use the ecosystem to poison your rival's margins and play catch up on distribution.
In the West, it tends to take on yet another hook: "shareware weights until you pass $1M ARR, then you must license." See Flux, K2, etc.
The only way for open weights to make sense financially is if you have another income stream and are dumping on the market to destroy competition and/or can get people into using your inference infra / product ecosystem / tooling. Nobody's cracked this yet.
This seems provably untrue? GLM and Kimi have been at the top of the open weights conversation for a while, and K3 and GLM-5.2 were still released in full; K3 added a commercial clause to the license, but is otherwise still completely open for personal use. And K3 in particular isn't just at the open frontier anymore, but trading blows with the frontier frontier.
personally I'd love to see open base models, and let companies differentiate with premium access to post training and alignment - that's where the real fight is anyways. they really ought to be pooling their resources/data and getting more economical with the pre-train anyways.
It was leaked which put it in the open, it got widely popular and they rode the wave. I'm not so such if they would have widely released it if it wasn't leak. Nevertheless mucho credits to them for following up with llama2, llama3, llama4 and now muse.
I have mixed feelings along these lines, I know meta have contributed to various open projects, sometimes Mark pays lip service to “the open internet" while his company represents a constilation of walled gardens. I think PHP got some love, and React is an industry goto (I'm more of a PHP... -> Svelte guy) but are these contributions worth what happened in Myanmar? I say no.
A billionaire (now trilionaire) helping lead and celebrate an extraordinarily rapid dismantling process in which vulnerable children lost life preserving assistance...
Essential employees were fired before the government had even established that it could safely do without them, and then...
After brandishing the chainsaw of efficiency in public, in the most cowardly way, went and then invoked all legal protections from being deposed on DODGE actions, personally and not answering under oath about the key decisions in that dismantling.
Facebook, Meta. 15+ years of emotional, child and human exploitation. Perverted glassware that spies on folk, lobbyists for age verification and who knows what else. They release an open model and all is fine and dandy? Nah.
Please get your priorities straight.
What do you think this Open LLM model is doing if not processing data from their murky sources?
Well, the last part of what you said is redundant. I'd say the only reason Meta released some open-source stuff is because they wanted to try to minimize the growth of what was already out there so they could give themselves time, because zuck went all in on VR and not all in on AI. What is he going to change his name to next?
It was literally like December. We're gonna rename ourselves Meta. January, ChatGPT 3.whatever has been released, and everyone was like, "What's this new AI stuff?"
Fundamentally I always try to look at incentives. It's not that they are good or bad but that incentives favour certain behaviours.
Google is incentivised to collect a bunch of data (like FB) to improve their ad serving etc.
Apple does not have that incentive as they don't make as much off advertising as hardware / app revenue.
NVIDIA is friendly with open source as they want to commoditise the model layer and take the gains in the hardware / data center layer.
It's easy to appear to have good intentions when you're railing against the companies that have decimated your output and made you [Meta] almost irrelevant in the AI 'race'.
Applying value judgements to corporate bodies or institutions as if they have individual agency is a fallacy anyways. We should always look at these things materially. "Meta" can't be good or evil, because an idea can't have a morality. It's comprised of the individuals who make the decisions, sure, but those individuals are always going to be motivated by a plethora of reasons which are often contradictory, most notably their material interests.
When we critiqe these sorts of institutions it's important not to prescribe value judgements on them and examine the circumstances of their condition materially.
When the material conditions and the people at the top of all of the corporate world lead to decisions with awful ethical implications there's conclusions that should be reached.
Certainly I agree. Those conclusions, if we take them to their root, can only be that there is some type is systematic deficiency in our socioeconomic order that incentivizes this type of behavior.
Don't mistake my comment for arguing against criticism of institutions, but that arguing whether or not something is "good" or "bad" doesn't draw you to a conclusion that is any more eor less substantive than that analysis.
Would you say it is a fallacy to apply such value judgements to say, the institutions of the National Socialist Party (Nazis), the KKK, the KGB, etc? How about a company whose business was selling slaves?
To be clear, I'm not saying being an employee of Meta is comparable to being a member of the aforementioned groups. But I don't think it is always a fallcy to apply a value judgement to an institution.
Yes, I would. An extremely dangerous and important one to get right in fact.
Let me first clarify what I am not saying. I am not saying that these institutions do not create evil, or commit evil acts and should not be resisted or destroyed. If we truly want these things to stop happening, material analysis is the only way to really address the root of the problem.
We can easily see that fascism was a result of the twisted incentive structure of colonial empire, the evolution of state corporatism as an economic mode, and rabid anti-communism rolled into one, etc etc. What I'm trying to say is that if your take away was just that Hitler was a bad guy and the Nazis were evil (which they very well may be), then you're doomed to repeat these events because this isn't a real analysis. It's not just that he was evil, it's that the socioeconomics lined up in such a way that allowed these things to happen. There will always be bad people seeking to take advantage of conditions, we will never be able to control that. What we can do is apply material analysis to events and try and determine the root cause of them, so we can prevent them from happening again.
What does this have to do with Meta? Well, my point it, Meta doesn't suck because Zuckerberg sucks. He very well may or may not, but that's not alalysis. Meta sucks because the incentive structure surrounding the attention economy is fetid and rotten, and that's a problem with all of us. Thats a problem within our society that allows these types of things to happen, and we ignore that at our own peril, just as we ignore the material reality that lead to the Holocaust by saying it happened because "Hitler and the Nazis were evil and that's that."
Sure, you can say whatever makes you feel better. You can say you're a pirate.
But writing off the mechanisms of large scale institutions as the result of moral actors is reductive and doesn't serve to address the systematic root cause of the issue, and ironically let's these people off the hook by narrowing the focus onto a group of particularly vile individuals. It's the "few bad apples" myth. I many ways, were all complicit. These are all human beings, after all. This is the true face of mankind.
I think this is categorically false. And furthermore comments like this are being used to astroturf and protect reputations of no-good companies and AI slop to keep the bubble growing. Embarrassingly, Mark Zuckerberg spent 80 billion dollars creating Miis for the oculus. The guy also comes off as extremely miserable and delusional in such a unique way that it's possible that there's no real psychological language to describe what is happening to him because his situation is so rare.
People like to polarize themselves to one extreme. Plenty of HN people call Elon Musk "pure evil." It's the most ludicrous statement ever.
Let me be clear: with electric cars and solar, elon musk has done more net good for the world then almost everyone on HN. Is elon musk perfect? No. Far from it. It's within the imperfectness that people find reasons to attack him.
> I think it's worth giving them some reasonable doubt.
Zuckerberg has shown through repeated action that he does not deserve any benefit of the doubt. This is the guy who called people “dumb fucks” for trusting him.
This is not even a case of “fool me once” anymore. If you continue to believe Zuckerberg, you’ve been fooled dozens, hundreds of times, and shame is definitely on you.
> No one is purely good, and no one is purely evil.
This post is utter bullshit and I think we’re all tired of the apologists, especially when it comes to Mark. These “middle of the road, let’s all be nice” solutions haven’t worked at all, and saying “Meta has done enough good” is extraordinarily questionable for those of us who have watched these tools rip apart the social fabric. Absolutely nothing will change when toothless people like yourself try to act like Mark is a normal person - we’ve all seen his fake, robotic, controlling, psychotic behavior over and over and over and over.
The overwhelming majority of actually existing adult Americans use either Facebook or Instagram or WhatsApp and are unfamiliar with the theory that they are Nazis.
The 737 has had 14 major recertifications. The aircraft today looks/behaves nothing like the original from the 1960s.
The main motivation for recertifications comes from commercial pressure where if a aircraft is given a new number and not recertified, then the pilots have to be retrained.
Honestly, back when the 737 MAX debacle happened, a lot of consumers claimed that they would stop flying aircrafts if they ran into 737 MAXs. And I don't think it happened in enough numbers - or even enough to make news. Sales went through the roof, everything kept working.
Recertifications are very common. The issue really is is the aircraft is AS different and untested as the old MAXs, and I really can't see that happening again in the next decade or two atleast.
> Honestly, back when the 737 MAX debacle happened, a lot of consumers claimed that they would stop flying aircrafts if they ran into 737 MAXs. And I don't think it happened in enough numbers - or even enough to make news. Sales went through the roof, everything kept working.
Is this kind of consumer revolt even really possible?
If you feel strongly enough that you refuse to fly altogether, then of course you can avoid flying on a 737 MAX. But I think most people did not feel the risk was that high. They just want to select "guarantee no 737 MAX" when booking a flight, and as far as I can tell that option doesn't exist.
Even if the flight is not a 737 MAX when you book, they can and sometimes do change aircraft, and as far as I know there's no option to get your money back when they do. If you show up and see it's a 737 MAX...you either get on or you lose your money, and have to find some other way to get where you're going, right?
In 2013 there were approximately 24,000 737 flights per day[1] - likely more today. If narrowed to just the MAX variants, it's still thousands per day.
Two, albeit high profile, crashes out of all the daily MAX volume is simply not something to worry about - let alone influence your booking choices.
You're applying everyday casual risk analysis to the highly-regulated environment of commercial air transport, where the MAX crashes absolutely were out of the norm and well beyond accepted levels.
Bear in mind when the crashes occurred there were fewer than 100 MAX in service.
Until you are on the plane. Sorry, but reasonably people and countries expect zero crashes and any single crash is worth worrying about because shareholder money should not trump a single human life.
Honestly they kinda screwed over people -- like me -- who tried to avoid the MAX planes for a while. I'd specifically book around the MAX planes and then they would change equipment at the last minute into a MAX. There is no meaningful "knob" an aviation consumer can turn to express an aircraft preference, and given how US airspace works, you often don't have a meaningful choice in carrier (unless you're willing to take on extra stops).
In the US (parent mentioned US specifically) I think that's just Frontier now that Spirit is gone. I mean technically that's doable sure but idk if I would say trivial it's really limited on routes and the experience is terrible from what I understand.
I have avoided flying on a 737 MAX and have even asked about it at gates when a plane was switched.
Part of the problem though is that many, many, many routes were straight up removed during and after COVID and still haven't returned. There is often no choice, particularly with certain companies like Southwest. However, I haven't flown Southwest since I learned that they were basically complicit, if not directly involved, in Boeing's 737 MAX issues.
People want to get what they're promised in a reasonable fashion. If the prices are hiding something like nonstandard seats or unreasonable baggage procedures, then that's a legitimate problem, not something they should shut up about because they should have known.
They deserve to know upfront how big the seat is, how big of a bag they can bring, and how much extra bags cost. On top of that, it can be useful if bag sizes and measurement have some standardization across the industry. Nothing major here or hard to provide.
Honestly, I think a lot is pretty well documented. But people don't have a good sense of what an inch more or less related to the seat actually means and don't necessarily know how picky a given airline is going to be on the size and weight of carry-ons. I expect that the carryon I routinely take on United might be an issue with budget European carriers.
Toyota had the largest recall in history for the unintended acceleration debacle. Yes, lots of people were saying they'd never set foot in a toyota again. Now people don't even remember it.
Reporting/news around AI is so very interesting these days.
It's really hard to tell what's propaganda from what's not.
For examples, this Reuters that you have included here has all tells tale signs of creating fear, uncertainty, and feeding into propaganda. But then again I can't be sure.
Agentic workloads are the most batch friendly, latency insensitive, geography insensitive, migration insensitive tokens that a big lab ever sells. In the ads business such inventory is called "remnant". The sausage is made of whatever is left over when the choice cuts have been removed.
This talking point from Anthropic that Claude Code sitting in a Ralph Loop is burning top sirloin interactive session tokens is bad faith hogwash and it only flies because most everyone who has run this shit at scale either already works there, sells them hardware, or hopes to be an acquisition target.
I'm none of those things, so I'm happy to tell you they're lying. I know, it's hard to swallow, but it turns out Altman and Amodei are occasionally full of shit.
Interactive use cases: the web interface, the mobile interface, the design tool. The fast variants.
In an HBM bandwidth constrained setting you're dealing with something called "roofline analysis" (comes originally from NUMA work circa ~2009 but it's applicable to modern GPUs). Great diagram from the JAX people:
In order to get your money's worth from a modern GPU (or disagg rack like an NVL72) you need to decode (the one token at a time thing) across big batches of context windows. To the left of that point where it hits "the roof" you're idling tensor units. TensorRT-LLM likes batches of 4096, so BS=4096.
In the case of one person chat prompting their local LLM, BS=1, totally bandwidth limited.
So the game is to set some latency target with some control theory primitive (PID or something) and then delay the next token until a batch is big enough to not waste tensor units. This is a real trick when a human is waiting (you've probably seen the thing in Claude.ai where it's all bursty and then they reflow the whole block with JavaScript).
Agentic workloads are huge piles of context windows where you've always got enough who want the same experts on the next token, you're always to the right of that intersection. And it doesn't really matter if it's on the other side of the world, or lags by a second, it's fine.
Claude Code soaks up all the tensor units that would be idle until they're full, and only then does it leak into the capacity reserved for highly interactive use. It's the bottom of the barrel until it's rinsed the fuck out.
They want more margin on agentic tokens. That's it. The COGS on them is the absolute lowest of anything they do.
My numbers come from the spend my company makes using our corporate plan. Not sure what the pricing is relative to any other plan. It's definitely not the consumer rate as we're a pretty big shop burning a lot of tokens as a whole.
It really is that these models have been trained, or maybe even over-trained, to save memories, and to a very far extend, this thing that they're calling communication is just the function of it saving memories.
To be honest, if I could stop AI from saving memories, it would be fantastic, because claude code etc definitely creates more issues for me when it creates memories than anything it solves.
But really the jailbreak was memories.
If you ever do introduce legislation, I would love to see legislation which stops general-purpose AI from saving memories. I think that would make things a lot safer.
reply