That's why I reached for Apples own local LLM to fool with similar ideas like this: https://pageforth.com. Apple is better than I expected at this. Right now it filters through things like hacker news articles and whatever else you point it at to summarize and find things that match your interests. Apple's LLM reminds me of Claude like 3 years ago. It's weak for sure. But useful for small dose kind of problems.
It's funny the convos I now have with Sonnet that I wasn't having with Opus. I feel like most of us here are starting to be told to draw down some of our 1M Opus xtrahigh thinking tokens :)
Is anyone using a local router to deal with that? Something thats like "don't even bother with sonnet for this task, just go with Opus". I wonder if Haiku could even do that math and recommend the model you should be in?
my task workflow uses something like opus to evaluate the roadmap, sonnet to divide the tickets by complexity, and then dispatch them to the relevant models - I use haiku or openai's spark models (spark is FAST! and DUMB!) for the simplest, and ascending in complexity. I find mid tier sonnet and gpt5 are pretty competitive, and reserve opus for truly "rearchitect the app from scratch" style tasks.
But all that might be somewhat obsolete, the latest update for claude code looks like it uses workflows with various models, so they might already be optimizing that.
The small stuff has their place. I have this safari extension and needed a way to quickly title people's chat histories. Haiku is the fast cheap thing to come up with decent titles of blocks of text. I feel like there's a bunch of those little things lying around you need a model for. I'm even finding Apple's Foundation Model is super useful for stuff like that. Even summarizing an article. It's like equally awful at doing it, but gets enough done to still be useful as a way to be like "oh yeah, this article is actually worth reading"
This makes me wonder if the LLM labs/Perplexities can figure out with CNN a proper revenue share model. I keep writing about this but it feels like we need something like AMP back (i hated AMP). but some kind of thing the publisher can declare: "if you use my content, you better show this ad because this is why that content was written. use my ad so we can still get paid."
Yup, as a forum owner, we've seen our traffic crash at least 60% since the AI just gives them a summary of our forum posts and the user never clicks through.
I just had a maddening convo with Amazon's AI https://share.zight.com/o0udw54W (i had mentioned before that in 5 minutes it would be time to get my refund according to some other policy message they sent me). A convo before this Amazon's bot didn't even recognize the reply the forced me to give from their button. I clicked "Entire package is missing". and they replied "Couldn't understand your response". It's a brutal, antagonistic experience. The kicker is how Amazon keeps saying "we're passionate about customer experience". You were Amazon. You were. But you've given that up.
The silver lining is that the pendulum will swing. It's like all thee independent bookstores thriving again. Eventually enough of us will revolt hard with our dollars. And move back towards businesses that aren't employing all these bots they stick in front of us. We'll get there.
this is why I feel like we need some kind of "consortium" or government effort to be like "yo, llms, you need to honor some kind of source markup to give us people you mention more significant boost"? like if you mention my article, you better also show my ad partner?
It's funny, the way that term gets used now is actually a wild distortion of the true history.
"The commons" was an incredibly successful system, and medieval (and prior) villages used it to great success, for the entire village's benefit! "Commons" are a great thing for everyone to have!
The real history is that as advances in technology (like the Industrial Revolution) changed things, certain rich villagers were suddenly able to manage more animals than they could before. Those (specific/rich) people over-used the commons, creating the "tragedy" we all know of.
The real lesson of history is not that commons fail: to the contrary, they worked great and helped everyone for centuries! The real lesson is "watch the fuck out for the new rich (especially when they just became rich because of recent technology advancements): those bastard will steal from everyone for their own benefit!"
reply