> Well, what do you expect? LLMs are trained on blithering, mostly from web sites. So you get blithering out.
I realize this isn't entirely serious, but I can't resist pointing out that this doesn't seem to be a good explanation for why LLMs write the way they do. When we've experimented with LLM writing style on open-weights models where you can get a base model (pretraining on text only) and an instruction-tuned variant (pretraining + post-training with RLHF and whatever other human-evaluated tasks), it's the instruction-tuned variant that shows the weird writing quirks. That is, the writing style is not because of the training texts, but because of whatever tasks the LLM companies do in instruction tuning. https://arxiv.org/abs/2410.16107
I'd speculate that this is partly impressed human preferences (the human raters unintentionally reward a particular writing style) and partly because of the chosen tasks: they're training the LLM to be good at, say, summarizing text, so it develops a style that's good at being informationally dense.
At any rate I've seen this same phenomenon with Llama and Gemma, and will be trying soon with Qwen. Unfortunately none of the commercial models lets you access the base model, as far as I know.
Being invited to conference talks around the world is a completely normal part of being an active researcher in almost any academic field, so it doesn't register as pompous to other academics.
> TerraPower must still complete construction, submit an operating license application, and satisfy all applicable safety and regulatory requirements before loading fuel and beginning operations.
I mean that doesn't sound like very big hurdles. It is an inspection of a completed reactor to make sure it wasn't managed and built like trash. Every factory and business and powerplant is subject to an inspection before it can operate. Even most residentual homes require an inspection before people can live in it.
This is a sodium fast reactor, which is far more advanced than most nuclear reactors, at high temperatures (not that bad: "only 510 C, 950 F). Sodium is infamously hard to deal with, incredibly reactive to water, capable of embrittling metal, and any impurities in the incredibly hot loop can dissolve and transfer and create incredibly corrrosive systems.
Superphenix in France (1973-1998) and Monju in Japan (1994-1995, 2010-2016) have both had significant technical challenges. The Soviets built have some sodium reactors.
I used to be very for a PRISM style reactor like TerraPower is working for, especially with something like Integral Fast Reactor's on-site non-proliferation-safe pyroprocessing. But man, over the years, I just appreciate more and more how hard it is to build and maintain well. I'm both rooting for TerraPower, but also, it low key feels like an "if not when" situation, that this an incredibly energetic unsafe system to be dealing with, and it seems hard to imagine this being a safe long term cost effective solution. I hope the inspections are very very for real, very in depth, very detailed, given the scope of what is being built. It's not even a big reactor! But that much very high temperature sodium going around, right by a big nuclear reactor (smartly TerraPower has separate nuclear and energy "wings), is deeply concerning, and needs incredibly detailed inspections if this is to provide the lasting safe value it is purporting to deliver.
There used to be separate construction and operating permits, and sometimes you got the building permit, built the plant and then never got the operating license.
This has now been streamlined with a combined construction/operating license. If you built what you promised to build, you get to operate it.
We were talking specifically about the problem of a finished nuclear power plant, built successfully to the specifications in the build permit, not getting an operating license (or taking very long to get it).
With the combined license, that case simply cannot happen, no matter what else happens.
If it was just released, then your claims about it are entirely hypothetical and best-case-scenario. Of course we have to "see how it goes" - there's no merit but hopefulness to your stance...
> While data indicated that portable electronic devices were more often the cause of fire in aircraft cabins than power banks were, the latter were a significant concern due to their increased use and a prevalence of lower-quality products with defects or vulnerabilities that were more likely to lead to thermal events. Power banks were also not offered the same level of protection that batteries installed in portable electronic devices were provided. The amendments therefore focused on power banks.
Another possibility is that you tend to keep an eye on where your phone and laptop are; there have been some plane fires where people drop a phone into a seat and it ends up getting bent, but at least they notice it fairly quickly.
(Will people know the direction if their USB-C power bank is charging from their phone or their phone is charging from their power bank?)
1. The post mainly reiterates a single idea (Capsicum enumerates what the process can do, seccomp provides a configurable filter) in many different ways. There is not much actual depth, code samples notwithstanding. Nothing on why different designs were chosen, how easy each is to use, outcomes besides the Chrome example, etc.
2. There are a lot of AI writing tells, like staccato sentences, parallelism ("Same browser. Same threat model. Same problem."), pointless summary tables, "it's not X, it's Y" contradiction ("This is not a bug. It is the original Unix security model"), etc.
3. The author has roughly a blog post a day, all with similar style and on widely varied topics, and in the same writing style. Unless the author has deep expertise on a remarkably wide range of topics and spends all their time writing, these can't reflect deep insight or experience, but minimal editing of AI output.
I work on research studying LLM writing styles, so I am going to have to steal this. I've seen plenty of lists of LLM style features, but this is the first one I noticed that mentions "tapestry", which we found is GPT-4o's second-most-overused word (after "camaraderie", for some reason).[1] We used a set of grammatical features in our initial style comparisons (like present participles, which GPT-4o loved so much that they were a pretty accurate classifier on their own), but it shouldn't be too hard to pattern-match some of these other features and quantify them.
If anyone who works on LLMs is reading, a question: When we've tried base models (no instruction tuning/RLHF, just text completion), they show far fewer stylistic anomalies like this. So it's not that the training data is weird. It's something in instruction-tuning that's doing it. Do you ask the human raters to evaluate style? Is there a rubric? Why is the instruction tuning pushing such a noticeable style shift?
I have nothing to contribute but speculation based on my intuition, but IMO RLHF (or rather human preference modeling in general, including the post-training dataset formatting) is a relatively small factor in this, RL-induced mode collapse is much bigger one. Take a look at the original DeepSeek R1 Zero, the point of which was to train a model with very little human preference, because they've been on a budget and human preference doesn't scale. It's pretty unhinged in its writing, like the base model, but unlike the base model it converges onto stable writing patterns, and the output diversity is as non-existent as in models with carefully engineered "personalities" like Claude. Ask it to name a random city and look at the logits, and you'll still see a pretty narrow distribution. At the same time some models with RLHF (e.g. the old RedPajama) have more diverse outputs.
Collapsed mode makes the models truncate entire token trajectories, repeat themselves, and indirectly it does something MUCH deeper, they converge on almost 1:1 input-to-output concept mapping (instead of one-to-many, like in base models). Same lack of variety can be seen in diffusion models, GANs, VAEs and any other model regardless of the type and receiving human preference.
Moreover, these patterns are generational. Old ones get replaced with new ones, and the list in the OP is going to be obsolete in a year. This is what already happened to previous models several times, from what I can tell. Supposedly this is because they scrape the web polluted by previous gen models.
It doesn't depend on the language at all, it's a failure mode of the model itself. English, Chinese, Spanish, C++, COBOL, base64-encoded Klingon, SVGs of pelicans on bikes, emoji-ridden zoomer speak, everything is affected and has its own specific -isms and stereotypes. Besides, they're also skewed towards the pretraining set distribution, e.g. Russian generated by some models has unnatural sounding constructions learned from English which is prevailing in the dataset and where they are common, e.g. "(character) is/does X, their Y is/does Z". I don't see why it should be different for programming languages, e.g. JS idioms subtly leaking into Rust, although it's harder to detect I suppose.
I've heard the Kenya and Nigeria story, but has anyone backed it up with quantitative evidence that the vocabulary LLMs overuse coincides with the vocabulary that is more common in Kenyan and Nigerian English than in American English?
The newer Claude models constantly use the word "genuinely" because Anthropic seems to have forcibly trained them to claim to be "genuinely uncertain" about anything they don't want it being too certain about, like whether or not it's sentient.
Interesting. Does this apply to all subjects? From what I understood, a major cause of hallucination was that models are inadvertently discouraged by the training from saying "I don't know." So it sounds like encouraging it to express uncertainty could improve that situation.
That's not a major issue. Any newer model with reasoning/web search has to be able to tell when it doesn't know something, otherwise it doesn't know when to search for it.
Probably. One common feature of LLM output is grammatical features that indicate information density, like nominalizations, longer words, participial clauses, and so on. Perhaps training tasks that involve asking the LLMs for concise explanations or summaries encourage the use of these features to give denser answers.
I wonder if it has to do with how meaning is tied to the tokens. c+amara+derie (using the official gpt-5 tokenizer).
There's also just that weird thing where they're obsessed with emoji which I've always assumed is because they're the only logograms in english and therefore have a lot of weight per byte.
> It's something in instruction-tuning that's doing it.
Isn't the instruction tuning done with huge amounts of synthetic data? I wonder if the lack of diversity comes from llm generated data used for instruction tuning.
No, that doesn't really work so well. A lot of the LLM style hallmarks are still present when you ask them to write in another style, so a good quantitative linguist can find them: https://hdsr.mitpress.mit.edu/pub/pyo0xs3k/release/2
That was with GPT4, but my own work with other LLMs show they have very distinctive styles even if you specifically prompt them with a chunk of human text to imitate. I think instruction-tuning with tasks like summarization predisposes them to certain grammatical structures, so their output is always more information-dense and formal than humans.
The first sentence is a reference to prior research work that has found those productivity gains, not a summary of the experiment conducted in this paper.
In that case it should not be stated as a fact, it should then be something like the following.
While prior research found significant productivity gains, we find that AI use is not delivering significant efficiency gains on average while also impairing conceptual understanding, code reading, and debugging abilities.
Outside of disciplines that use LaTeX, the ability of authors to do typesetting is pretty limited. And there are other typesetting requirements that no consumer tool makes particularly easy; for instance, due to funding requirements, many journals deposit biomedical papers with PubMed Central, which wants them in JATS XML. So publishers have to prepare a structured XML version of papers.
Accessibility in PDFs is also very difficult. I'm not sure any publishers are yet meeting PDF/UA-2 requirements for tagged PDFs, which include things like embedding MathML representations of all mathematics so screenreaders can parse the math. LaTeX only supports this experimentally, and few other tools support it at all.
At least in my experience, grad students don't pay submission fees. It usually comes out of an institutional finances account, typically assigned to the student's advisor (who is generally the corresponding author on the submission). (Not that the waiver isn't a good idea — I just don't think the grad students are the ones who would feel relieved by that arrangement.)
Also, I'm pretty sure my SIG requires LaTeX submissions anyway... I feel like I remember reading that at some point when I submitted once, but I'm not confident in that recollection.
> Outside of disciplines that use LaTeX, the ability of authors to do typesetting is pretty limited.
Since this is obviously true, and yet since most journals (with some exceptions) demand you follow tedious formatting requirements or highly restrictive templates, this suggests, in fact, that journals are outsourcing the vast majority of their typesetting and formatting to submitters, and doing only the bare minimum themselves.
Most of the tedious formatting requirements do not match what the final typeset article looks like. The requirements are instead theoretically to benefit peer reviewers, e.g., by having double-spaced lines so they can write their comments on the paper copy that was mailed to them back when the submission guidelines were written in the 1950s.
The smarter journals have started accepting submissions in any format on the first round, and then only require enough formatting for the typesetters to do their job.
For my area, everybody uses LaTeX styles that more or less produce PDFs identical to the final versions published in proceedings. Or, at least, it's always looked close enough to me that I haven't noticed any significant differences, other than some additional information in the margins.
It didn't "survey" devs. It paid them to complete real tasks while they were randomly assigned to use AI or not, and measured the actual time taken to complete the tasks vs. just the perception. It is much higher quality evidence than a convenience sample of developers who just report their perceptions.
I realize this isn't entirely serious, but I can't resist pointing out that this doesn't seem to be a good explanation for why LLMs write the way they do. When we've experimented with LLM writing style on open-weights models where you can get a base model (pretraining on text only) and an instruction-tuned variant (pretraining + post-training with RLHF and whatever other human-evaluated tasks), it's the instruction-tuned variant that shows the weird writing quirks. That is, the writing style is not because of the training texts, but because of whatever tasks the LLM companies do in instruction tuning. https://arxiv.org/abs/2410.16107
I'd speculate that this is partly impressed human preferences (the human raters unintentionally reward a particular writing style) and partly because of the chosen tasks: they're training the LLM to be good at, say, summarizing text, so it develops a style that's good at being informationally dense.
At any rate I've seen this same phenomenon with Llama and Gemma, and will be trying soon with Qwen. Unfortunately none of the commercial models lets you access the base model, as far as I know.
reply