There was a good paper documenting this in broader industry, and highlighting how it may account for previous research suggesting good ideas have been getting harder to find. Here's the abstract:
> Relatively flat US output growth versus rising numbers of US researchers is often interpreted as evidence that ideas are getting harder to find. We build a new 45-year panel tracking the universe of US firms’ patenting to investigate the micro underpinnings of this claim, separately examining the relationships between research inputs and ideas (patents) versus ideas and growth. We find that average patents per R&D input are increasing, the elasticity of patents to R&D inputs is flat or rising, and there is no systematic evidence of a secular decline in patenting after controlling for research inputs. We then document a positive, significant, and fairly steady relationship between firms’ growth in ideas (patents) and labor productivity. Average firm growth after controlling for idea growth, however, declines. Together, these results suggest that innovative efforts play a key role in sustaining growth that has not diminished over the last four decades.
It's not the step change in time that's the gratest danger of DST, it's the entire time people are waking up early, increasing cancer rates and all-cause mortality: https://pmc.ncbi.nlm.nih.gov/articles/PMC6436388/
Multiple resources on the internet put the startup costs at $500K or less. That's not much more than fast food or other high touch retail startups by comparison.
There are certainly a few steps on the learning curve but it's mostly high school and undergraduate math that is needed. That puts it out of reach for some but the field is pretty flat once you get there, is my perspective having considered and opted against getting into it.
It is a commodity product but, again, the regulatory and capital hurdles to get started are significant. As you scale you need more and the exact reserve requirements are often at the discretion of regulators. Opening a McDonalds franchise it is not.
A touch of nuance from someone in the industry (CTO at a startup that sells home and umbrella insurance):
Most of the issues here stem from problem the sales as opposed to the claims process. Customers are, structurally, under-educated on what their policies actually cover and this produces unrealistic expectations about what they should file claims for.
Emphatically, this is not their fault but an industry-wide issue that has complex causes like brokers/salespeople getting increasingly squeezed to produce as well as the intense time pressure under which many folks purchase their homeowners insurance policy (folks are often sprinting to check this box to secure a mortgage). Sadly, there is also a strong correlation between consumers experiencing worse socioeconomic conditions and failed claims. This, to me, generally suggests that some behavioral insights should be brought into the sale and management of the financial product to better protect these buyers' interests.
It is important to highlight and understand this point of failure because: 1) claims (even failed ones) are one of the easiest ways to get your rates jacked up as this is a category that insurers are allowed to price on. 2) The common response to articles like this is that "of course insurance should cover more things" but this counterintuitively risks creating more dead-weight loss for consumers broadly in the form of coverage for things that we simply shouldn't be insuring (and instead should be maintaining and replacing).
All that is to say: ask your broker / agent what's in your contract. Do it BEFORE you have a claim to file as in some cases speaking to them (especially captive agents) may in and of itself trigger a claim.
AI-native brokerages (like what we've built) are part of the solution to this problem since well-constrained LLMs can help buyers and users get a much better sense of what their contracts actually cover, and whether or not they should submit a claim.
What do you guys do when the LLM hallucinates? Someone puts a claim in on the basis of what the LLM instructed and its false, are you still taking responsibility? Geniune question, interesting to see where the liability is resting in a regulated industry as opposed to just chat bots.
My understanding is that this remains unsettled law. We've added a disclaimer after seeing the big players (OAI, Ant) do the same and focus on ensuring LLMs get things right / outperforms those less specialized tools.
There is significant value to reducing pollution (in human health and flourishing).
That said, GP comment is intellectually dishonest. It doesn't account for the negative externalities of his choices/politics that, as you correctly identify, are tied to his values.
Great to see more insurance hype! We've been working on AI to solve the consumer search problem in the industry for the past 3 (almost 4) years and it's great to see the big labs getting their hands dirty and building tools for practitioners in the space.
More industry exposure to well-managed agentic experiences will create oodles of opportunities to reduce premiums for consumers and offput some inflation-driven increases in cost of coverage.
The advice here is good, and I'm a big believer that the cream (e.g., sincerity and real opinions) rises to the top for writing. Still, think folks dunk on these types of writing automation tools too much when, for many, they can be a gateway drug to consistent posting and finding your online voice.
That is to say, the whole post is a bit of an internet old-head complaint. Reminds me of baby boomers complaining about a "decline" in homeownership and having children without acknowledging the massive shifts in the economic accessibility that support these milestones.
It's easy to write a post like this when you've already built a following because you started when social media was a greenfield experience. It's much harder when you have to compete for signal while being pressured to build a brand and perform at your day job.
The obsession with constant content production combined with algorithmic, feed driven consumption frontends with terrible discoverability and intense bubblification lead to today's screaming contest that ruins our sanity. On average I find it much worse than the old infosphere (TV+print+radio) used to be, for producers and consumers. It's quite tragic, really.
Though I also notice awareness around this issue is rising (e.g. smartphone bans in school, initiatives like bluesky), which is good, I guess. All of this is still a society-wide experiment without control group.
Agreed. Discussions like these always remind me of some great research on how the destruction of the old, more averaged, and less targeted infosphere used to support significantly more political cohesion.
If you want to lose all hope, just read the top selling romance novels on the Kindle app. These people are raking in millions a year and it’s just absolutely awful.
"In further analyses, the researchers found that, for cancer patients, whether their HDHP had a health savings accounts (HSAs) did not make a difference."
HDHP are a federal requirement if you want a HSA. Full stop. Everyone I work with and nearly everyone I know has one, and no one is dying. The headline is hyperbolic but given the source I'm not surprised.
It doesn't have to be malicious. If my workflow is to send a prompt once and hopefully accept the result, then degradation matters a lot. If degradation is causing me to silently get worse code output on some of my commits it matters to me.
I care about -expected- performance when picking which model to use, not optimal benchmark performance.
The non-determinism means that even with a temperature of 0.0, you can’t expect the outputs to be the same across API calls.
In practice people tend to index to the best results they’ve experienced and view anything else as degradation. In practice it may just be randomness in either direction from the prompts. When you’re getting good results you assume it’s normal. When things feel off you think something abnormal is happening. Rerun the exact same prompts and context with temperature 0 and you might get a different result.
This has nothing to do with overloading. The suspicion is that when there is too much demand (or they just want to save costs), Anthropic sometimes uses a less capable (quantized, distilled, etc) version of the model. People want to measure this so there is concrete evidence instead of hunches and feelings.
To say that this measurement is bad because the server might just be overloaded completely misses the point. The point is to see if the model sometimes silently performs worse. If I get a response from "Opus", I want a response from Opus. Or at least want to be told that I'm getting slightly-dumber-Opus this hour because the server load is too much.
The question I have now after reading this paper (which was really insightful) is do the models really get worse under load, or do they just have a higher variance? It seems like the latter is what we should expect, not it getting worse, but absent load data we can't really know.
Explain this though. The code is deterministic, even if it relies on pseudo random number generation. It doesn't just happen, someone has to make a conscious decision to force a different code path (or model) if the system is loaded.
Its not deterministic. Any individual floating point mul/add is deterministic, but in a GPU these are all happening in parallel and the accumulation is in the order they happen to complete.
When you add A then B then C, you get a different answer than C then A then B, because floating point, approximation error, subnormals etc.
It can be made deterministic. It's not trivial and can slow it down a bit (not much) but there are environment variables you can set to make your GPU computations bitwise reproducible. I have done this in training models with Pytorch.
For all practical purposes any code reliant on the output of a PRNG is non-deterministic in all but the most pedantic senses... And if the LLM temperature isn't set to 0 LLMs are sampling from a distribution.
If you're going to call a PRNG deterministic then the outcome of a complicated concurrent system with no guaranteed ordering is going to be deterministic too!
No, this isn't right. There are totally legitimate use cases for PRNGs as sources of random number sequences following a certain probability distribution where freezing the seed and getting reproducibility is actually required.
How is this related to overloading? The nondeterminism should not be a function of overloading. It should just time out or reply slower. It will only be dumber if it gets rerouted to a dumber, faster model eg quantized.
Just to make sure I got this right. They serve millions of requests a day & somehow catastrophic error accumulation is what is causing the 10% degradation & no one at Anthropic is noticing it. Is that the theory?
There's a million algorithms to make LLM inference more efficient as a tradeoff for performance, like using a smaller model, using quantized models, using speculative decoding with a more permissive rejection threshold, etc etc
The primary (non malicious, non stupid) explanation given here is batching. But I think you would find looking at large-scale inference the batch sizes being ran on any given rig are fairly static - there is a sweet spot for any given model part ran individually between memory consumption and GPU utilization, and generally GPUs do badly at job parallelism.
I think the more likely explanation is again with the extremely heterogeneous compute platforms they run on.
I checked the link, it never says that the model's prediction get lower quality due to batching, just nondeterministic. I don't understand why people conflate these things. Also it's unlikely that they use smaller batch sizes when load is lower. They just likely spin up and down GPU serves based on demand, or more likely, reallocate servers and gpus between different roles and tasks.
> Relatively flat US output growth versus rising numbers of US researchers is often interpreted as evidence that ideas are getting harder to find. We build a new 45-year panel tracking the universe of US firms’ patenting to investigate the micro underpinnings of this claim, separately examining the relationships between research inputs and ideas (patents) versus ideas and growth. We find that average patents per R&D input are increasing, the elasticity of patents to R&D inputs is flat or rising, and there is no systematic evidence of a secular decline in patenting after controlling for research inputs. We then document a positive, significant, and fairly steady relationship between firms’ growth in ideas (patents) and labor productivity. Average firm growth after controlling for idea growth, however, declines. Together, these results suggest that innovative efforts play a key role in sustaining growth that has not diminished over the last four decades.
https://faculty.tuck.dartmouth.edu/uploads/teresaFort/files/...