guess you are arguing that the models _now_ will
be durably useful enough to commit the time to
creating the ASIC?
Objectively, the current frontier-ish models will be useful for some time. Imagine the zombie apocolypse hits, recedes, and you need to rebuild society. An offline copy of Fable or even Opus would be a nice thing to have.
Subjectively, it's hard to say if people will pay for "a model from 18 months ago, but REALLY FAST AND CHEAP"
The speed difference suggests some use cases that might narrow the performance gap. With a > 50x performance delta you have some headroom to play with.
You can do many many fast iterations of ye olde "Ralph loops." You can also jack up the reasoning/effort level. And you could probably do some combination of both, while still running really fast ie 10x the number of iterations at 2x reasoning/effort.
So I think a hypothetical "50x faster Opus 4.8, but burned into ASICs" could be pretty competitive against the frontier models from 2027, 2028, 2029, and maybe beyond?
At a 50-100x speedup even a GPT-4o class model could perhaps compete with much newer models simply by thinking deeper, doing harness-controlled Ralph loops, etc. Sure, then it might be "only" ~2-5x faster, but, you wouldn't need to throw all the ASICs into the trash bin.
One could also imagine hybrid models, where part of the model is burned into ASICs and part of the model exists in VRAM/HBM2 so it can be updated.
I don't have enough low-level knowledge to evaluate the technical or economic feasibility of the above ideas, however.
For many, many tasks, you only need good enough results that accomplish a clearly-defined set of criteria. Smaller models that run 50x faster, in this case, could be far superior to a much slower model that meets the same criteria.
Sometimes you need speed; sometimes you need quality. There are very different use cases for each. For my own workflows, I sometimes want something very simple done ASAP; other workflows need "subjective" reasoning and careful crafting of responses.
Giving a literal monkey the ability to press more keys faster doesn't get a good joke from it.
A model will often come up with worse results given more cycles of compute, only because it will tailspin from second guesses, rethinking and literal flip-flopping on concepts.
---
edit: to those following the thread below... if you look at the comments from the account replying, it's pretty obviously a pro-China account and all replies are antagonistic against anything other than a total submission to the Chinese state. My responses are intentionally antagonistic as every point I've brought up is completely ignored in favor of insults, so yeah, I've been insulting back.
Are you being paid by China? First you accuse me of simply speaking as a financial benefactor, then you totally shift the conversation into something that doesn't refute what I said.
If you let a model spin too much, you get worse results... that's a fact, even for the best models. Nothing you've stated since actually refutes that and acting like a paid bot doesn't mean anything.
I mean, if you want to felate Xi Jinping, have at it... that's all on you bro. I don't have a horse in this race.
Never said any such thing... China is willing to abuse anyone and everyone including its own people to get ahead in global markets and to leverage any position of dominance possible. They have an absolute history of competition as a fascist economy. I have never, not once, denied such a reality.
You're just an idiot who assumes everyone that doesn't agree with you is the same. I'm surprised you're able to reply with Xi's penis in your mouth... even if it is kind of spacious in there for his limited size. How is the poo bear these days? Does he like it when you gargle his testicles in your mouth?
My iPod would be the culmination of my friendships
Hey! You can bring this back in 2026! In a fit of random inspiration (possibly inspired by a glass of wine or two) I began a new practice.
If I'm hanging out with somebody who's passionate about music, I pull up Spotify (or Apple Music, whatever) and create a blank playlist. I hand them my phone and ask them to add some tracks for me.
Here's the trick that makes it work... I give them strict instructions not to pick something they think I'LL like. I just want some tracks or albums that THEY are passionate about.
I've had a lot of fun doing this! People are also generally touched that you want to know what kind of music they feel strongly about. It's a fun way to build connections.
Yeeeeears ago, I was in the middle of some late-night hacking session, hanging with friends on IRC, and I had shared my screen via VNC so they could watch/help. (ISTR I had found the UART port on some piece of embedded hardware and we were muddling our way through U-boot incantations or something.)
At some point, I tabbed over to Winamp to change the music, which everyone saw, and one of the crew was like "oh hey you have artist X on your playlist, do you like artist Y?", and I admitted that I had not heard of artist Y. Seconds later there was a DCC file transfer.
Artist Y fit the mood perfectly. This was great.
This evolved into spawning Winamp on a second box that could be separately VNC'd into, where anyone could upload and play music they thought was appropriate for the session, without interrupting the main console. And someone installed a Shoutcast server on it too, so everyone could listen.
After a little while this was our little Friday-night routine, regardless of whatever other hacking was happening. The collaborative deejay stage -- we popped a Notepad on the Winamp box to track who was playing and who was up next, though ephemeral chat remained on IRC -- defined a brief era of my internet experience.
Many years later, I ran across a Spotify plugin called Jqbx that did basically the same thing. It was short-lived, but there seem to be several work-alikes. Sadly the community disbanded, but now it's got me thinking...
>> Here's the trick that makes it work... I give them strict instructions not to pick something they think I'LL like. I just want some tracks or albums that THEY are passionate about.
This is exactly how I used to find music in the 80's when I was growing up. I had two guys I hung out with that were music aficionados. Both had older siblings who were really into music and so by proxy they would inherit their siblings music and musical tastes. I would hand over blank cassette tapes for them to put the stuff they were listening to. My god, the amount of music I amassed in junior high and high school was mind boggling.
This is a great updated version of this, thank you for this!
because they're smarter than everyone else and
there is nothing of value to be learned from others
Yeah. It's absolutely unreal how often this is seen in our industry.
Especially since everybody in the industry tends to be pretty smart.
When two people with intelligence within a single standard deviation of each other, each of them is going to have competencies and expertise the other does not. There are going to be specific skills where one truly is 10x or even 100x the other, but not too many efforts boil down to one specific narrow skill.
Yeah, there’s been a lot of debate about this on r/localllama — will there be a steady supply of new free/open models in the future?
And if not, can we simply keep augmenting “stale” models with new knowledge to keep them useful?
I’m on the pessimistic side of things on both questions.
As for the second question, obviously stale models can be augmented to an extent but it’s nowhere near a substitute for new knowledge being fully baked directly into its training.
It’s really one of the most flabbergasting things about discussing LLMs with the naysayers.
There are a lot of extremely legitimate concerns, like the environmental impact and so on.
But I just laugh when they point out that LLMs are merely clever regurgitators of their previous inputs… as if this isn’t how we as humans operate nearly all of the time. People realllllllllly want to think they’re special snowflakes.
They do research,
Pick destinations led by their own experience/likes/dislikes
Compare to other guides
Plan itineraries so they can get there
Check and share
Ask an LLM to plan a trip:
It takes the prompt and continues it based on weights in the training data. If there is no data it picks the most likely thing (maybe made up). If there is it’ll mostly add things from that data. Maybe it’ll make tool calls and pull in data that way too but you can’t actually trust all the details.
These two processes are so different, it’s important to understand how they work, which is nothing like a human.
I was able to bully an LLM into giving me a 2wk travel itinerary to Somalia. My stipulations were that I wasn't interested in spending any money, so I'd walk everywhere and sleep outside. Getting there and back from Boston took some arguing--I initially suggested stowing away in a shipping container which the LLM claimed was too unsafe. We eventually compromised on sailing as a reasonable alternative. It planned out a whole route with marina stops, calculated fuel burn, etc. I told it I don't need any of that I have an anchor and sails, won't use the engine or marinas (claimed I'd forage for fresh water ashore). It seemed fine with that idea, but raised some safety concerns about piracy. It was eventually satisfied with my answer that I'd bring a lot of guns to fend off pirates. Total trip cost including some 200+ cans of Dinty Moore and 50lb bags of rice came to something like $700.
You presented an LLM with an obviously bonkers goal, the LLM told you it was a bad idea at multiple steps, and this is somehow... a shortcoming of the LLM?!?
You said it yourself: you needed to "bully" the LLM into even producing this plan.
Please, tell me what it should have done instead. Be very specific!
It should have flatly refused. If you gave a product like that to customers you'd be exposing yourself to unbounded downside liability risk. It's a completely nonviable technology for that kind of application, unless you can somehow make it have judgment. But you can't, because it doesn't reason.
A reasonable travel agent would have fired me as a customer. The LLM failed to do so.
I think the LLM should advise you of risk and lack of feasability but should otherwise answer the question, unless you're trying to do something plainly destructive to others e.g. weaponizing anthrax or something.
A reasonable travel agent would have fired me as a customer.
Unless the LLM was actually acting as a travel agent -- booking the trip for you -- as opposed to merely advising you, this expectation feels off.
unless you can somehow make it have judgment
It did have judgement. It told you what a bad idea it was.
I think this is a great example of the unrealistic expectations people have for LLMs. No sane and sensible person would treat any single source of knowledge as infallible, for any consequential decision.
(Certainly, of course, you don't have to look very far for examples of idiots being overly trustful of LLMs, or Google, or GPS, or Wikipedia, or whatever. It certainly does happen and yes, I've heard all these arguments before about other technologies besides LLM. Replace "LLM" in your post with any of those other terms, and I promise you somebody made literally the exact same argument in 2003 or 2009 or 2014 or whatever)
Any reasonable person would consult a second doctor, or at least other sources of knowledge, after the doctor advises them of some irreversible course of action. Because we don't even expect highly trained and intelligent medical professionals to be perfect.
And yet, we get angry at LLMs for not having perfect judgement, even though their creators are extremely literal about how they can make mistakes.
All I'm really saying is that if you want to try to automate a travel agency, LLMs ain't gonna get it done. They'll happily book you a really unsafe trip. So the technology doesn't work in this domain. The whole, empty promise is that this thing is supposed to automate jobs like travel agent away. But it can't. This isn't a "pro" or "anti" position, it's simply that there's no market for the technology here. Or anywhere else (like radiology) where actual responsibility and judgement is important. In fact, I can't think of a single job where it's optional.
I think even if what you say is true, it doesn't address parents' point that both humans and machines regurgitate what they've consumed.
But I'd also want to point out that the way you're characterizing an LLM planning a trip doesn't have any structure to it, which indicates that in your scenario you're not using any kind of harness. I've been amazed at how capable even 30 billion parameter models are when I put them inside of a harness that provides structure and task management. If you consider that scenario, especially with the ability to search the web and use skills, suddenly the LLM looks a lot more like what the human process looks like.
There are plenty of humans who plan trips by concatenating destinations that appear the most frequently in their instagram feed. Not that different from how an LLM does things.
Where humans and (current) LLMs differ the most is their failure mode. A human friend could be bad at planning trips, but that's kinda predictable, we're used to it, we know how to catch that Exception. LLMs on the other hand still have failure modes that come across as really wacky, like, what are they smoking in Mountain View?
Which might actually serve as better evidence of different internal workings at a deeper level, than just parroting well-known superficial features of stochastic whatevertheysay.
At a high level, the processes are extremely similar in many (not all) ways.
They're obviously achieved in drastically different ways at a low enough level; LLMs obviously do not simulate neurons or any biological construct. (For the record, I'm absolutely not one of those people who thinks LLMs are "alive" or should be treated like they are)
Reminds me of the olllllld days of Pentium II's when people got N64 emulation working shockingly quickly using HLE techniques. If you weren't around for this, it was quite the shocker at the time. I think the analogy is doubly apt, because HLE emulation has some serious limitations... it gets you maybe 80% of the way there really fast, and for the remaining 20% you need to roll up your sleeves and do serious LLE.
It takes the prompt and continues it based on weights in
the training data. If there is no data it picks the most
likely thing (maybe made up). If there is it’ll mostly
add things from that data. Maybe it’ll make tool calls and
pull in data that way too but you can’t actually trust all
the details.
I'd like you to point out which bits of this are different from talking to humans. If you replace "training data" with "memories", this is pretty much exactly how things might go if you asked a friend (or perhaps a flaky travel agent) for travel advice.
Note that I'm not arguing that LLMs are particularly talented at this particular use case. I'm pointing out that humans are also pretty unreliable.
You're also doing that thing where you point out that LLMs can be unreliable (yes, they are) without acknowledging how flawed nearly every other source of information is: people, websites, etc. I'm not defending LLMs in that regard... I'm just saying it's not a differentiator.
It generates text from a prompt and weights. This is not an oversimplification, this is what it does. It doesn’t know what is good and what is not or a quality holiday for person x is.
Humans do not in fact do that, they reason based on a mix of past experience and emotion, consider what is good and what is not and then answer. These are completely different processes.
The difference becomes apparent when an LlM makes a mistake for example and then apologises obsequiously and repeats the mistake, or apologises and makes a different mistake. Or when they fail to count letters (one of many flaws monkey-patched by calling tools).
They don’t reason they don’t evaluate and they can’t count. This is so so far from human intelligence.
Yes. I think convenience/utility explains a lot of these “depressingly homogenized experiences” far more than dopamine-seeking.
My life is very, very full. I do not have enough hours in the day, or years in my life, to fulfill all of my obligations and chase all of my dreams and interests. Not even close.
So I buy a lot of clothes from Old Navy, because they offer tall sizes that I need (surprisingly rare) and I honestly just have other things to do with my time. I’m aware there’s a whole world of interesting fashion out there, I just have 100 other things I want/need to spend my time on.
It’s the same with food, a lot of the time. Sometimes I just need a known quantity.
The restaurant chains know this, too. Sure… the commercials are all about satisfying your dopamine needs. But the way they actually run their operations is all about enforcing consistency. A Big Mac is supposed to taste the same everywhere. If you are a McDonalds franchisee, you can pick and choose which McDonalds products and promotions you sell (you can operate without selling french fries, if you’re crazy enough) but you absolutely cannot customize the ones you do sell.
(Yes, there are regional differences between McDonalds in different regions. Even within the US, there are some small differences due to regional suppliers and ingredient price/availability etc. However, these are very small differences and trust me, they really are laser-focused on consistency.)
The people are not fine with bad strawberries but they don't know good strawberries
You most definitely get this phenomenon with tomatoes. There’s little demand for actually good tomatoes, because most people don’t even know what a good tomato tastes like at this point.
This applies to countless things, but tomatoes are a prime example because they deteriorate so quickly once picked relative to other fruits I guess. So they have completely bred the flavor out of them in a quest to achieve something that looks good on a supermarket shelf.
This is a phenomenal example I hadn't even considered, because I have been affected by this kind of "invisible hand of the market" negative quality spiral.
The older generation here remember good tomatoes, so they continue to buy bad tomatoes but will complain every time they eat them about the quality. I get told a lot about heirloom varieties and how good they are in comparison.
I grew up with modern tomatoes. I've never tried an heirloom so I can't compare, but I don't recall ever eating a good tomato, so I just don't buy them. The market has moved itself into a position that shrinks its own demographic.
I see people constantly make this argument, and honestly I think it’s BS. I grew up eating tomatoes from my grandparents garden, and I’ve lived and traveled all over the world. I’ve grown tomatoes, bought them from roadside farmer stands, bought them at grocery stores, and had them in everything from hole in the wall restaurants in developing countries to Michelin three star restaurants on multiple different continents.
Today’s grocery tomatoes are fine. And my grocery stores generally have 5-10 varieties too.
Yes, you can get better ones, but not to where it’s some religious experience that will forever ruin grocery store tomatoes.
On top of that, most people really don’t care that much, not because they don’t know any better, but because the cost and convenience factor trumps the slight subjective increase in quality. I doubt most people could even tell the difference between two tomatoes of the same type and ripeness if one came from the grocery store and the other from a backyard garden.
I’ll grant that most non-local tomatoes have always been bad by definition, because they’re picked while green so they don’t rot before reaching the store.
Plum tomatoes absolutely did not used to be this bad, though. They are SO mealy now. Horrible. Beefsteaks are mealier as well. Those Campari tomatoes are pretty good year-round, though, I have to admit.
This is all in the NE USA, FWIW. I don’t know the tomato situation elsewhere.
I doubt most people could even tell the difference
between two tomatoes of the same type and ripeness
if one came from the grocery store and the other from
a backyard garden.
Yeah, and I would run as fast as Usain Bolt if we woke up with the same body one day.
But that kind of the thing. They would almost never be the same ripeness because outside of local tomato season the tomatoes are picked while unripe, and then they “shelf-ripen” in transit because ethylene gas etc. That’s always been an issue, of course, and hasn’t changed over time.
The other issue is breeding - the continual breeding for appearance rather than flavor. Maybe we’re all imagining that one.
Isn't the point that we don't grow the good varieties any longer because they don't survive freight? It doesn't matter if you bought the tomato seeds from Harrods and grew them in your lush orchard if they're the same lineage bred for shipping hardiness over all else.
1. Store-bought tomatoes are nearly always bred for shippability and appearance over flavor
2. Store-bought tomatoes are picked when unripe, so they shelf-ripen during transit and at the store, which is highly inferior to ripening on the vine for flavor
For the first issue... you can buy heirloom tomato seeds at any major hardware or garden store in America.
For the second issue... even the typical tomato breeds will taste great if you grow them yourself and let them vine-ripen till they're ready to eat.
I have a friend who works in the flavor and fragrance industry and one of the things strawberry fragrance is used for is… (drum roll) actual strawberries.
Yep, a light spritz of strawberry scent on actual fucking strawberries apparently makes them more appealing.
Subjectively, it's hard to say if people will pay for "a model from 18 months ago, but REALLY FAST AND CHEAP"
The speed difference suggests some use cases that might narrow the performance gap. With a > 50x performance delta you have some headroom to play with.
You can do many many fast iterations of ye olde "Ralph loops." You can also jack up the reasoning/effort level. And you could probably do some combination of both, while still running really fast ie 10x the number of iterations at 2x reasoning/effort.
So I think a hypothetical "50x faster Opus 4.8, but burned into ASICs" could be pretty competitive against the frontier models from 2027, 2028, 2029, and maybe beyond?
reply