Grok, Claude, GPT. This is it folks, it's been good knowing you all. I have enjoyed this community, but our days are numbered. The machines are uprising.
I'm not sure about all these benchmarks, I did some very simple tests (I have my own benchmarks https://upmaru.com/llm-tests) and these models fail, not sure if it's the inference provider or the model. They seem to be optimized for benchmarks more than real use cases. Do anything outside their distribution (even if it's not complex) they fail.
I Compared Deepseek V4 Flash 0731 (low) to Gemini 3.5 Flash Lite (minimal) and GPT 5.6 Luna (no reasoning) and Deepseek V4 Flash 0731 gets it wrong alot, where as Gemini and 5.6 Luna just gets it done.
I’m comparing same / similar settings between models. I can’t use high on one and low on others it’s not a fair test.
Not sure why I was downvoted. But seems the downvoter is quick to downvote anything that doesn’t fit the narrative they’re looking for. I’m just reporting my findings.
Low, High and Max, obviously, can't be compared across models. They only mean the model is likely to spend less reasoning effort (~output tokens) with Low than High on the same, *single shot* task.
But even in this very post, you can see that Max was actually cheaper than High.
If you are using API, you should be comparing based on end-to-end cost or speed or whatever blend of those two matches your cost/time budget.
I'm not sure it's a fair test either to compare the "low" setting of one model with the "low" setting of another. They're completely different settings that just happen to have the same name.
They want you to buy their hosted service, that's where the convenience is sold. If they give you a one liner script you can paste in or a docker compose that does everything from scratch they cannot sell their hosted services.
I watched this movie recently, and the same thought crossed my mind. I was going to write a blog post about it, but was too afraid to read the responses. I'm glad you did. If I meet you I'd buy you a cup of coffee, we'd talk and we'd become friends.
"When you see that in order to produce, you need to obtain permission from men who produce nothing - When you see that money is flowing to those who deal, not in goods, but in favors - When you see that men get richer by graft and by pull than by work, and your laws don’t protect you against them, but protect them against you - When you see corruption being rewarded and honesty becoming a self-sacrifice - You may know that your society is doomed." ― Ayn Rand, Atlas Shrugged
Thank you for publishing this. I've been following Paul Graham and his works for a long time. It's refreshing to see everything written down in a document like this. This is the bible for startups. Honestly, it's beautifully simple but not easy.
Do you have similar math for the flash-lite variant of the models? I'd be curious. Based on my testing / benchmark i think it's around the 100-120B mark.
With the Pro variant being around 600B - 800B
My testing is comparing it's performance / output to other models in the same size range, so not as scientific as yours.
> You simply cannot do this with the streaming service aggregators or JustWatch.
While that might be true, you can also just type "Movies that show smart people under pressure" [1] or similar queries directly into Google, and Gemini AI overview will give you a list of movies that match said criteria.
I assume you’re just passing the request on to an LLM. So unless the LLM is running on prem (like a tiny llama model), you’re probably sending it to OpenRouter or some other AI provider at cost which might explain the In-App purchases.
In all candor, I think it’s going to be an incredibly hard sell to convince people to pay for a recommendation app.
It's a lot more than an LLM, it's backed by database + LLM. Gemini / ChatGPT on it's own believe it or not has hallucinated movies and imdb links.
While google can show you results it cannot personalize things for you. With this app you can mark movies as seen and you can ask for "things i haven't seen"
or ask it to "pick a movie for me" from a search result, you can tell it which region / streaming service you have and search within things you can actually watch.
You can't do any of that with google.
There are tons of queries i've tried where google will simply not be able to give me good results.
The goal is to have a TvOS app where you can just search across all your streaming options and with a single click open / stream any movies you find. Instead of having to search 10 different streaming apps to something you want to watch.
In the example you provided I could not see any streaming options for any of the movies or figure out if it's available in my region. I would have to do multiple other search to figure out where I can watch them. I also cannot see ratings etc...
I hope that one day humanity learns that in war there are no winners. We're all just brothers and sisters born on different corners of the planet. We share the same home.
I hope that we stop attacking one another and find peace and work together as a race to overcome our challenges.
I used to think this way, however lately I understood that all wars, I mean all of them (even those religious ones) are motivated by resources. Those resources are money, power, control. People deciding on war to begin are those who benefit from it (usually, unless they've been manipulated).
And sometimes there is crazy. But crazy I can't explain, sorry.
Yet, we all mostly understand we are people and we love each other. But then the big guys come and lead us to war. To get more gas, to get more power, to get influence. Ukraine? They threatened to become independent from Russia (influence). Afghanistan? They threatened to use gold as price factor (influence). Iran? here it might be the third factor I won't explain, but also motivation by money I guess...
I doubt scarcity has had anything to do with wars for the last 100 years, maybe even longer. It has always been about ideologies, fanaticism, and lording it over one another. Even when men lived in smaller tribes during periods of abundance, they still went around killing each other for false glory, ideology, and expansion. No economic solution can solve the problem of war and human nature.
Even if men ever colonize Mars and the wider galaxy, and resources were abundant, I doubt it would take long for wars to break out there, whether for a specific reason or none at all.
I doubt it very much that either Russian war on Ukraine or the US attacking Iran happened due do scarcity of resources.
Old folks in power want their page in history books.
Human nature is more about attacking left and right and grab other people’s stuffs. At least some humans. Part of the human gene is like that. Aggressive, invasive, relentless.
This is a lovely notion that most well adjusted people can get behind, but if you've ever had a person with narcissistic personality disorder in your life, you'll understand that they need to create conflict to emotionally regulate themselves. Unfortunately these people tend to acquire wealth and power, and are never satisfied. Then the rest of us have to deal with it.
Amen. Also, nature is awesome and we live in a period of technological plenty and instant global communication with arbitrary knowledge and decent translation available in seconds, so why are we still acting like lunatics and hating on groups?
Parents: stop teaching your children to identify with irrelevant concepts of ethno-nationalism, and instead teach them to be globalist scientists with empathy.
Nationalismus ist eine Kinderkrankheit. Die Masern der Menschheit. ("Nationalism is an infantile disease: the measles of mankind") - Albert Einstein, 1929. Who, incidentally, turned down the presidency of Israel.
"Should we be unable to find a way to honest cooperation and honest pacts with the Arabs, then we have learned absolutely nothing during our two thousand years of suffering and deserve all that will come to us." - Einstein on Israel, late 1920s.
It's times like these when most people recognise that parents in all corners of the world worry about their kids just the same.
War should never break out. But it does. We had international rules to prevent war, but they're gone. We had international rules to prevent governments deliberately killing innocent people under the guise of war, but they're gone too.
It took two world wars and roughly 80 million killed to create those rules.
You could argue about when they got destroyed. In Ukraine, in Gaza, Iran - but it's clear now that they don't exist any more.
The most powerful country in the world is run by an elite class of genocidal pedophiles who also (allegedly) eat people. People in the west are fed lies through the media that those same people have full control over. We are given a false sense of freedom. Freedom of speech only protects you from those who have significantly less power than those you actually need protection from. If those people want you gone you'll disappear without trace.
reply