No.

But do you need to run every small problem through a 10B-30B model?

We're smashing ants with hammers most of the time. We're asking frontier Opus/Fable models to classify text and build frontend code.

Once we start dissecting these problems into smaller discreet tasks and having the big reasoning models do the tough stuff, we suddenly have an economical system. Not for the company hoping for a big IPO, but for the end user.

I do about 10 google search queries for every 1 opus/gpt prompt. For google, I don't actually open pages anymore 9 out 10 times; I rely on the AI summary. It's fast and accurate; the trick is that you learn where the boundary is of what you can ask it. Querying information the small model is great at.

Then there might be slow, batch tasks. I can see myself getting 1T of slow RAM one day (in a few years?) and having a slow onsite GLM5.2 doing batch jobs that would be wasteful of my subscription limits, plus sensitive but boring things, such as bookeeping and general admin.

I'd like to to read all my email and al quarterly reporting. But that would have to be a good local model, probably a model simmilar to whatever google search uses, which seems just correct unless you throw serious challenges at it.

The big hammers buy you more confidence and need less supervision. In pure task execution you _might_ smash the ant with a small surgical hammer, but if you absolutely need it smashed, that's when people still reach for the big hammer. It buys more confidence.

> do you need to run every small problem through a 10B-30B model? ... We're asking frontier Opus/Fable models to classify text

Actually probably yes: text analysis (magazine articles) by LLMs in the ~30b .. ~120b range failed miserably (and also randomly - the rare cases of proper interpretation occurred among the failure cases) with the main public models of around one year ago, tried extensively.

So, yes, you can employ an ~80IQ only if you will expect the related quality.

I don’t think you need a 10-30b model for most smartphone use cases.

But I meant to counter gp’s claim that “I can run a 27b model on an iPhone” is kind of pointless and disingenuous. Yes I’m sure someone will come up with a way to run a “27b model” at 0.1 bit quantization on an Apple Watch pretty soon misses the whole point of saying a model is “27b” in capability.

Achieving a parameter count is not the point. And is almost meaningless

> But they first need adoption.

I think they've got that. Short of budget stuff or the Slate truck, most new cars have some big dumb screen in them at all times.

But advertising poses a new problem for both advertisers and mfgs not unlike the mid-90s ad sale issue. There was no consolidated ad server, so everyone was trying to build their own agency and advertisers had to navigate that.

Which probably means some sort of Google or Google-like player in the space.

Obviously they read it, but it's 2026 and entirely possible to publish an article without having read a word of it :)

All that said, I think initially it was a mix of a few things coming together.

Yes, auto mfgs always want to reduce parts for cost and supply chain control. But there was also this moment of New Wow where the impractical nature of touchscreens was overshadowed by the holy crap I've got a tablet in my car. It implied a break with the last generation of cars, where you might have gotten a 4-inch screen (touch or not), and it became desirable at a surface level to users.

Although I greatly dislike touchscreens for the obvious usability issues in a motor vehicle, I still kind of widen my eyes when I'm in a car with some new, ridiculous multi-screen dashboard setup.

Mazda was mentioned in this thread, and I think they do a great job of separating the concerns here; you've got a big buttons of various sizes that do different things that can be memorized without sight.

Provide financial incentives to make it worse.

> Everyone constantly does!

In the aggregate, I agree, but in tech things are pretty loose outside of California.

It was purely a hype/media play.

Elon wanted precisely the same model.

You can't buy the OS directly, but it certainly isn't free.

How do you figure a price of 0 is a high price?

I think this is a good point, but it ignores the idea that a human form is not the ideal (or even closet to) one to do specific and generalized jobs.

Maybe not human form, but my house is designed for humans. It needs to be able to handle stairs. It needs to handle my doors and halls. It needs to reach upper shelves that are human height (my wife is short and constantly complains about things on a top shelf she cannot reach). I have corners in my house that a large dog can only manage because their spine bends. Human form in many ways is a constraint.

It is ideal insofar as the world has been created with humans in mind. And it’s easier to acquire training data for human shaped entities than for some hypothetical better shape.

On the one hand I understand this fairly deeply.

I started doing "ML" ~ 20 years ago building classifiers people would laugh at today and even at the time barely impressed people when they were 95% correct.

I moved into NLP and built NERs that missed 2-10% of named entities per document routinely. Best of breed approaches and models rarely fared better.

Learned the cornerstones in school for ML; linear regression, ANNs, traditional RL, image classifiers, A* bots, etc, most of which got baked into transformers later on.

Then the transformers went from interesting novelty to useful. I couldn't build a useful one locally, but the toys versions were still fun to play with.

Then the novelty LLM went from useful to generally applicable. Then they became a silver bullet.

I still can't build one locally. I can distill or build or fine tune if you give me some rented GPUs. But to call this ML is very much a stretch.

I still use the traditional ML a lot, but mostly for evals and analysis.

I get being naturally bummed by this but I can't justify feeling anything but vaguely nostalgic about it. Someone with a $20 subscription can mog anything I can build with the skills I picked up.

If someone hands you a silver bullet you'd be a fool to decline it and spend your time hand casting a crude piece of brass. If the difference between 95% and 99% means you know how to aim or oil the gun, that's the world we live in.

Building a good RAG pipeline or prompt optimization or LLM consensus is dumb stuff that produces a better result than anything I could do from my 2010 ML/AI textbooks. I don't lack the knowledge or capacity to compete, I lack the compute.

That's the job now for 99% of companies.

An acquaintance who works at FAANG says he still builds non-LLM ML systems because of the cost of running LLMs at that scale

Exactly. Besides cost, domain-specific models (which can still be very large) encode our biases (i.e., our knowledge about the domain) in their architecture. Because of that, we have ways to calibrate their accuracy trade-offs over an in-domain sample.

For LLMs, there is a disconnect between the perceived domain (anything a human can think and verbalize) and actual domain (word sequence prediction). We only know how to sample from the latter, not the former.

This "silver bullet" idea sounds a lot like "free lunch". There has been a lowering of the bar for ML practices to make way for this onslaught of prototype-level productivity. Teams that used to do their best to run uncorrelated evals are now having the prompt engineers manually inspect ~100 model outputs before launch.

People were shocked by the accuracy of n-gram models and user interaction data to "read their mind" and complete their searches. Now we're obviously all impressed by the emerging abilities of LLMs (integrated with lots of business logic by the LLM providers). Hopefully in time we'll get desensitized a bit and have the right mental model when using these tools.

P.S. I love your username :)

> I get being naturally bummed by this but I can't justify feeling anything but vaguely nostalgic about it. Someone with a $20 subscription can mog anything I can build with the skills I picked up.

Welcome to the data science job market of the 2015-2023 where everybody with a $20 online course could become a proficient data scientist in only 4 weeks!