I think there is some value in asking these questions. Maybe a better question would be how much effort and time was put into a project.
1. Maintenance: Projects that (seem to) have the same quality and complexity but were built over months or years by hand versus built with an AI tool in a weekend will probably experience different amounts of maintenance in the future. This is, of course, not a black-and-white rule. But simply knowing that people think a project is worth spending months or years on gives me an indication that they probably want to keep it alive in the future.
2. Testing: Projects developed by hand probably underwent a lot more hidden manual testing—maybe on different hardware, maybe by more than one person. Since my personal projects have gotten significantly more complex with the availability of AI coding assistance, I spend much more time writing prompts and thinking about integration testing. But this is not the case for all projects.
206 binary blobs = 15MB, so not crazy but i built for this use case where you can declare the registry of languages you want to load and not have to own all the grammar binaries by default
Cost, debt, difficulty forming a moat, gap between what the product promises and what it can do, and the difficulty actually raising capital required.
His style is acerbic and (imo) excessive sometimes. But he's also one of a minority of journos actually looking at the numbers and adding them up. Which seems to be a rarity
That doesn't matter if the free models are as performant in 6 months. I will never personally pay for a model I can have for free. ChatGPT 5 used to be my preferred model as a DMing help tool, now deepseek and LeChat are the one I use, and are better at what OpenAI model use to be better at. And I think the models hit their limit for my usecase, I don't need better one. I never 'reprompt' anymore, and just roll/improvise with what I got.
The only way for openAI to get my subscription back would be my country making open-weight ai or deepseek illegal. It was worth the price tbh, but they can't compete with free.
Disagree. He's cherry picking an extremely limited subset of numbers, based on a weak understanding of the industry and a lack of access to a lot of private data, and taking advantage of vulnerable people.
Well from my point of view. When they talk about gigawatt datacenters, then yes it is economically nonviable. You just need to know the scale of a gigawatt to realize that we need to start building power plants and fortifying the power grid to ship a gigawatt of power to a single location. Until the build out which takes years mind you, it is competing with other consumers of power. Lets take another huge consumer of power like a large steel mills use 100 megawatt. So if that power becomes more expensive because of datacenters, then the price of steel will go up. And if the price of steel goes up it affects a lot of things in the economy.
We are facing a situation that the short term effects are on memory and storage prices going up and lack of jet engines. Long term we wont be able to build actual buildings and ships without financing it with even more debt than today and everyone in the economy is going to service that debt through the price.
but the costs of inference have been going down 20x to 30x over the years. so how can you tell it is nonviable? unless you are saying they are not paying market rate for the inference
So, they still booked up all the ram and ssd in the world and still going to use gigawatts of power. The price of energy production is not going to go down 20x and 30x it just means that they can cram in more inference on the same energy consumption if the cost goes down. But they aren't paying the market rate for inference because everything is subsidized with debt and investors money to scale as fast as possibly. They are flushed with money and that is why they can book up all silicon production.
This claim sounds extremely fancy when AI companies bleed money, and will keep bleeding money in the foreseeable future.
I don't pretend to know the future. Maybe LLMs become economically viable and are the future, maybe not. I don't really care either way, to be frank.
And I use LLMs, btw. I pay for a ChatGPT account, but I find it only moderately useful. I always sort of question myself upon renewal date if it is worth the 20 bucks I spend monthly on it.
In no small part I keep using it to keep myself up to date on the best practices of using them in case it becomes standard.
The graph you linked seems to compare different OpenAI models in terms of "price per million tokens".
I am very skeptical of any financial information that comes from OpenAI. I have no idea how truthful those numbers are, or how creatively they can be collected to paint a rosier future for them.
Even if the numbers are truthful, I have no idea how the calculate price there. Is it in terms of cost of compute they rent? Is this cost subsidized or not?
Also, I don't know this "epoch.ai" website, I don't know their stance. The website name itself does not inspire my confidence on their reporting of anything related to AI. "Eat meat, says the butcher" vibes and all.
You can claim that the AI bleeds money because training is expensive, but inference is cheap. So it will only be financially viable when they stop training models? So they would need to stop improving their capabilities entirely for it to make any sense, is that your claim?
Even if I take this claim at face value (and that would take a lot of faith I don't have to give), it doesn't sound as good as you think it does.
>To analyze the decline in LLM prices over time, we focused on the most cost-effective LLMs above a certain performance threshold at each point in time. To identify these models, we iterated through models sorted by release date. In each iteration, we added a model to the set of cheapest models if it had a lower price than all previous models that scored at or above the threshold.
Can you look at the analysis? It will make it clear. I mean its so obvious because GPT 4 costs way more than GPT 5.2-mini but much worse performance.
>Even if the numbers are truthful, I have no idea how the calculate price there. Is it in terms of cost of compute they rent? Is this cost subsidized or not?
Do you think they are subsidising 900x or simply that the costs have gone down?
Overall you have shown what I feel is extreme skepticism in something that is obvious. You can literally run a model in your laptop that matches an older closed model. Costs are obviously going down, I have shown data. Use your own anecdotes and report.
Extreme skepticism in such a way doesn't do any help.
> Overall you have shown what I feel is extreme skepticism in something that is obvious.
I think you show extreme faith in something that is very obscure.
For me to believe in the analysis I would need to trust the numbers that the analysis is based upon. I see no reason why I should trust this. What sort of regulatory body or neutral third party inspects those numbers to ensure they are not a fabrication?
But you can claim I am a hater if it justifies your worldview. Skepticism is sinful for the believer.
> For our language model benchmarking, we note that we consider endpoints to be serverless when customers only pay for their usage, not a fixed rate for access to a system. Typically this means that endpoints are priced on a per token basis, often with different prices for input and output tokens.
Okay, correct me if I am wrong, so this is measuring the inference costs for clients of AI services, not the the inference costs that the AI service itself has when they offer the service?
I mean, the other guy's claim is that inference costs had come down 20x-30x. But the analysis, if I understood correctly, is based on how much clients are paying for it, not how much it actually costs.
I can charge you 20x less for a service and have massive losses for it.
It could be that OpenAI is subsidising their models by _fifty times_. Do you really think they are doing that? In some cases the costs went down by 200x. Do you really think OpenAI is subsidising their models by 200??
Its easier to just admit that technological advances helped decrease the cost instead of coming up with more complicated reasons like VC funding, subsidies and so on.
For instance take Deepseek and other opensource models - even they have reduced their costs by a huge margin. What explanation is there for opensource models?
> It could be that OpenAI is subsidising their models by _fifty times_. Do you really think they are doing that?
Possibly. I don't know.
It could be unfeasible to increase prices so much whenever a new model was released.
Any assumption made here is based on vibes. I see no reason to drop my skepticism.
> Its easier to just admit that technological advances helped decrease the cost instead of coming up with more complicated reasons like VC funding, subsidies and so on.
They raised an absurd amount of cash, and still bleed money to an absurd degree.
VCs make money when they exit. OpenAI only needs to "make sense" until an IPO happens. Once private investors have their exit, the markets can be left to handle the resulting dumpster fire.
> For instance take Deepseek and other opensource models - even they have reduced their costs by a huge margin.
Chinese companies are very opaque. I don't pretend to have insight into it.
Is the company behind Deepseek profitable?
> What explanation is there for opensource models?
What opensource models have to do with inference?
Your argument is that training is expensive but inference is cheap (something I see no evidence of). Why would a company give away the expensive part of the work?
>It could be unfeasible to increase prices so much whenever a new model was released.
This means you have no idea what I have been saying. A new model is costlier, but they release mini versions of old models that are way cheaper and compete with older models.
GPT 5 mini is way cheaper than GPT 4 but around the same performance
GPT-5 mini:
Input tokens: ~$0.25 per 1 M
Cached input: ~$0.025 per 1 M
Output tokens: ~$2 per 1 M
-----
GPT-4 (legacy flagship):
Input roughly $2.00 per 1 M
Output roughly $8.00 per 1 M
>Chinese companies are very opaque. I don't pretend to have insight into it.
False. The models are not opaque, you can literally download it and host it yourself. They have also released papers on how they reduced cost in certain areas.
This is literally them documenting the cost-profit ratio theoretical at 500%
>The above statistics include all user requests from web, APP, and API. If all tokens were billed at DeepSeek-R1’s pricing (*), the total daily revenue would be $562,027, with a cost profit margin of 545%.
Not only that, there are other providers hosting these opensource models, there are so many companies - just go to openrouter.com
So this is your skepticism
- openai is subsidising their models so much that each year the keep doing it 20x and eventually reached 100x reduction
- all the investors are stupid and they still invest in openai despite unprofitability
- employees of openai and anthropic who have claimed that the unit costs are not high are also lying
- all other providers are in on the lie
- the chinese models like Deepseek is also in on the lie by posting research that is not plausible
- the fact that you can run models in your laptop today that beat previous years models is also not enough
> openai is subsidising their models so much that each year the keep doing it 20x and eventually reached 100x reduction
If that's the truth, then originally they were subsidizing their models by the same factors.
This is not a great argument no matter how you cut it. And even then I would need to see evidence that this is true.
> all the investors are stupid and they still invest in openai despite unprofitability
Much to the opposite, those people are very smart. OpenAI can be extremely unprofitable and they can still profit massively through an exit event.
> employees of openai and anthropic who have claimed that the unit costs are not high are also lying
Possibly? Especially if they are in the position to profit in the case of an exit event, they would have every incentive to paint a rosier picture about the company.
> all other providers are in on the lie
I have no idea who you are talking about.
> the chinese models like Deepseek is also in on the lie by posting research that is not plausible
As I previously stated, I have no idea if Deepseek is profitable. By the looks of things, neither do you. Mentioning Deepseek's research is a non-sequitur.
> the fact that you can run models in your laptop today that beat previous years models is also not enough
I don't think Ed doesn't comment about the actual tech. Here are some things he has said before and please tell me if these still hold in the spirit?
> You cannot "fix" hallucinations (the times when a model authoritatively tells you something that isn't true, or creates a picture of something that isn't right), because these models are predicting things based off of tags in a dataset, which it might be able to do well but can never do so flawlessly or reliably.
ChatGPT is fairly reliable.
>Deep Research has the same problem as every other generative AI product. These models don't know anything, and thus everything they do — even "reading" and "browsing" the web — is limited by their training data and probabilistic models that can say "this is an article about a subject" and posit their relevance, but not truly understand their contents. Deep Research repeatedly citing SEO-bait as a primary source proves that these models, even when grinding their gears as hard as humanely possible, are exceedingly mediocre, deeply untrustworthy, and ultimately useless.
This is untrue in spirit.
> You can fight with me on semantics, on claiming valuations are high and how many users ChatGPT has, but look at the products and tell me any of this is really the future.
Imagine if they’d done something else.
Imagine if they’d done anything else.
Imagine if they’d have decided to unite around something other than the idea that they needed to continue growing.
Imagine, because right now that’s the closest you’re going to fucking get.
This is what he said in 2024. He really thought ChatGPT is not in the future.
There are so many examples and its clear that he's not good faith and has consistently gotten the spirit wrong.
> With the amount of talent working on this problem, you would be unwise to bet against it being solved, for any reasonable definition of solved.
I'm honestly not sure how this issue could be solved. Like, fundamentally LLMs are next (or N-forward) token predictors. They don't have any way (in and of themselves) to ground their token generations, and given that token N is dependent on all of tokens (1...n-1) then small discrepancies can easily spiral out of control.
To solve it doesn't mean we have to eliminate it completely. I think GPT has solved it to enough extent that it is reliable. You can't get it to easily hallucinate.
It depends on how much context is in the training data. I find that they make stuff up more in places where there isn't enough context (so more often in internal $work stuff).
Typical shakedown tactic. I used to have a boss who would issue these ridiculous emails with lines like "you agree to respond within 24 hours else you forfeit (blah blah blah)"
One of the great benefits of AI tools, is they allow anyone to build stuff... even if they have no ideas or knowledge.
One of the great drawbacks of AI tools, is they allow anyone to build stuff... even if they have no ideas or knowledge.
It used to be that ShowHN was a filter: in order to show stuff, you had to have done work. And if you did the work, you probably thought about the problem, at the very least the problem was real enough to make solving it worthwhile.
Now there's no such filter function, so projects are built whether or not they're good ideas, by people who don't know very much
People who got "enabled" by AI to produce stuff, just need to learn to keep their "target audience of one"-projects to themselves. Right now it feels like those fresh parents who show every person they meet the latest photos / videos of their baby, thinking everybody will find them super cute and interesting.
Yeah, I think it's sort of an etiquette thing we haven't arrived at yet.
It's a bit parallel to that thing we had in 2023 where dinguses went into every thread and proudly announced what ChatGPT had to say about the subject. Consensus eventually become that this was annoying and unhelpful.
My manager at work still does this in work chats and it drives me a bit crazy. If I want to k own what an LLMs take on the subject is I can just go ask it.
>went into every thread and proudly announced what ChatGPT.
That is what Show HN has become. Nobody cares what code Claude shat in response to a random person prompt.
If I cared, I would be prompting Claude myself.
>People who got "enabled" by AI to produce stuff, just need to learn to keep their "target audience of one"-projects to themselves.
This is what I do. I have tons of cool (to me) shit I have built with LLM assistance. I only wheel my dumb stuff out if its specifically relevant to someone.
But I am also not doing it as a resume hobby, just as a hobby. A lot of people are trying to jump from hobby to career. The recognition is the point for some.
I get that, and mostly agree but some people will really have the cutest kitten the most beautiful sunset photo. Hopefully we figure out how to discern as fast as we can churn.
You get diminishing returns up there though, the cutest cat photo in the world would look remarkably similar to the next 2,000 photos of cats in the cutest cat photos leaderboard. I feel we should filter on diverse topics rather than best by metrics, perhaps we'll want to discern on concern.
The other element here is that the vibecoder hasn't done the interesting thing, they've pulled other people's interesting things.
Let's see, how to say this less inflamatory..
(just did this) I sit here in a hotel and I wondered if I could do some fancy video processing on the video feed from my laptop to turn it into a wildlife cam to capture the birds who keep flying by.
I ask Codex to whip something up. I iterate a few times, I ask why processing is slow, it suggests a DNN. I tell it to go ahead and add GPU support while its at it.
In a short period of time, I have an app that is processing video, doing all of the detection, applying the correct models, and works.
It's impressive _to me_ but it's not lost on me that all of the hard parts were done by someone else. Someone wrote the video library, someone wrote the easy python video parsers, someone trained and supplied the neural networks, someone did the hard work of writing a CUDA/GPU support library that 'just works'.
I get to slap this all together.
In some ways, that's the essence of software engineering. Building on the infinite layers of abstractions built by others.
In other ways, it doesn't feel earned. It feels hollow in some way and demoing or sharing that code feels equally hollow. "Look at this thing that I had AI copy-paste together!"
To me, part of what makes it feel hollow is that if we were to ask you about any of those layers, and why they were chosen or how they worked, you probably would stumble through an answer.
And for something that is, as you said, impressive to you, that's fine! But the spirit of Show HN is that there was some friction involved, some learning process that you went through, that resulted in the GitHub link at the top.
I knew i could do better so i made a version that is about 15kb and solves a fundamental issue with web gl context limits while being significantly faster.
AI helped do alot of code esp around the compute shaders. However, i had the idea of how to solve the context limits. I also pushed past several perf bottlenecks that were from my fundamental lack of webgpu knowledge and in the process deepened my understanding of it. Pushing the bundle size down also stretched my understanding of js build ecosystems and why web workers still are not more common (special bundler setting for workers breaks often)
Btw my version is on npm/github as chartai. You tell me if that is ai slop. I dont think it is but i could be wrong
I have yet to see any of these that wouldn’t have been far better off self-hosting an existing open source app. This habit of having an LLM either clone (or even worse, cobble together a vague facsimile) of existing software and claiming it as your own is just sort of sad.
I actually came to this realization recently. I'm part of a modding community for a game, and we are seeing an influx of vibe coded mods. The one distinguishing feature of these is that they are entirely parasitic. They only take, they do not contribute.
In the past, new modders would often contribute to existing mods to get their feet wet and quite often they'd turn into maintainers when the original authors burnt out.
But vibe coders never do this. They basically unilaterally just take existing mods' source code, feed this into their LLM of choice and generate a derivative work. They don't contribute back anything, because they don't even try to understand what they are doing.
Their ideas might be novel, but they don't contribute in any way to the common good in terms of capabilities or infrastructure. It's becoming nigh impossible to police this, and I fear the endgame is a sea of AI generated slop which will inevitably implode once the truly innovative stuff dies and and people who actually do the work stop doing so.
Even before vibe coding blew up I think this problem existed, but was slowly increasing. Vibe coding accelerated the problem.
I think "good for you, you built a mod, but now I'm searching through 1000 entries that are all the same things and mostly shit".
I think it's like many popular forums or Reddit communities. The shitty comments float to the top and all the replies are some meme or "me too". It doesn't help anyone else and it feels masturbatory.
That's the essence of the corporations behind these commercial products as well. Leech off of all the work of others and then sell a product that regurgitates that said work without attribution or any back contribution.
It often is. The concept of "gatekeeping" becoming well known and something people blindly rail against was a huge mistake. Not everything is for everyone, and "gatekeeping" is usually just maintaining standards.
Ideally the standard would just be someone's genuine interest in a project or a hobby. In the past, taking the effort to write code often was sufficient proof of that.
AI agent coding has introduced to writing software a sort of interaction like what brands have been doing to social media.
I'm not sure what distinction you're trying to make, but it seems like you might be distinguishing between keeping out substandard work versus keeping out the submitters.
In which case, I kinda disagree. Substandard work is typically submitted by people who don't "get it" and thus either don't understand the standard for work or don't care about meeting it. Either way, any future submission is highly likely to fail the standard again and waste evaluation time.
Of course, there's typically a long tail of people who submit one work to a collection and don't even bother to stick around long enough to see how the community reacts to that work. But those people, almost definitionally, aren't going to complain about being "gatekept" when the work is rejected.
To be fair, one probably needs at least one idea in order to build stuff even with AI. A prompt like "write a cool computer program and tell me what it does" seems unlikely to produce something that even the author of that prompt would deem worthy of showing to others.
Agreed, and were gonna see this everywhere that AI can touch. Our filter functions for books, video, music, etc are all now broken. And worst of all that breaking coincides with an avalanche of slop, making detection even harder.
There is this real disconnect between what the visible level of effort implies you've done, and what you actually have to do.
It's going to be interesting to see how our filters get rewired for this visually-impressive-but-otherwise-slop abundance.
My prediction is that reputation will be increasingly important, certain credentials and institutions will have tremendous value and influence. Normal people will have a hard time breaking out of their community, and success will look like acquiring the right credentials to appear in the trusted places.
That's been the trajectory for at least the last 100 years, an endless procession of certifications. Just like you can no longer get a decent-paying blue collar job without at least an HS diploma or equivalent, the days of working in tech without a university education are drying up and have been doing so for a while now.
I think the recent past was a respite in very specific contexts like software maybe. Others, like most blue collar jobs, were always more of an apprentice system. And, still others, like many branches of engineering, largely required degrees.
I have a sci-fi series I've followed religiously for probably 10 years now. It's called the 'Undying Mercenaries' series. The author is prolific, like he's been putting out a book in this series every 6 months since 2011. I'm sure he has used ghost writers in the past, but the books were always generally a good time.
Last year though I purchased the next book in the series and I am 99% sure it was AI generated. None of the characters behaved consistently, there was a ton of random lewd scenes involving characters from books past. There were paragraphs and paragraphs of purple prose describing the scene but not actually saying anything. It was just so unlike every other book in the series. It was like someone just pasted all the previous books into an LLM and pushed the go button.
I was so shocked and disappointing that I paid good money for some AI slop I've stopped following the author entirely. It was a real eye opener for me. I used to enjoy just taking a chance on a new book because the fact that it made it through publishing at least implied some minimum quality standard, but now I'm really picky about what books I pick up because the quality floor is so much lower than in the past.
If you've some time to burn, write the author and/or his publisher and let them know that the guy's new ghostwriter sucks shit. If this is very seriously making your consider not picking up the next book in the series, be sure to mention that.
If folks just stop purchasing the new books, they can imagine a reason for the lost sales that's convenient for them, but if folks tell them why they stopped purchasing, there's a lot less room for that kind of nonsense.
People will build AI 'quality detectors' to sort and filter the slop.
The problem is of course it won't work very well and will drown all the human channels that are trying to curate various genres. I'm not optimistic about things not all turning into a grey sludge of similar mediocre material everywhere.
I worry that the focus on AI proofing will lead to a deanonymization of the internet. If we force every interaction to be associated with a real world id, we can kill a lot of the bots.
This is going to happen anyway because 'the powerful' want to track what everyone does and says. AI is going to accelerate this because now they have a much more efficient means to filter and identify the people that are doing and saying things that the powerful don't like. The powerful will also be able to get real world ID credentials for their bots if they wanted or needed them, so this will not stop the problem of bots.
Sure, there's many examples (I have a few personal ones as well) where I'm just building small tools and helpers for myself which I just wouldn't have done before because it would take me half a day. Or non-technical people at work that now just build some macros and scripts for Google Sheets that they would've never done before to automate little things.
I'm in the same boat. I use AI to generate tons of small things for work. None of it is shareable online because it's unique to my workplace and it's not some generic reusable tool, and for the most part the scripts are boring. Their most interesting attribute is how little effort they took, not their originality or grandness of scale.
> so projects are built whether or not they're good ideas
Let’s be honest, this was always the case. The difference now is that nobody cares about the implementation, as all side projects are assumed to be vibecoded.
So when execution is becoming easier, it’s the ideas that matter more…
This is something that I was thinking about today. We're at the point where anyone can vibe code a product that "appears" to work. There's going to be a glut of garbage.
It used to be that getting to that point required a lot of effort. So, in producing something large, there were quality indicators, and you could calibrate your expectations based on this.
Nowadays, you can get the large thing done - meanwhile the internal codebase is a mess and held together with AI duct-tape.
In the past, this codebase wouldn't scale, the devs would quit, the project would stall, and most of the time the things written poorly would die off. Not every time, but most of the time -- or at least until someone wrote the thing better/faster/more efficiently.
How can you differentiate between 10 identical products, 9 of which were vibecoded, and 1 of which wasn't. The one which wasn't might actually recover your backups when it fails. The other 9, whoops, never tested that codepath. Customers won't know until the edge cases happen.
It's the app store affect but magnified and applied to everything. Search for a product, find 200 near-identical apps, all somehow "official" -- 90% of which are scams or low-effort trash.
To play devil's advocate, if you were serious about building a product, whether it was hand-coded or vibe-coded, you would iterate through the work and implement functionalities step-by-step.
But with vibe-coding, you might not give enough thoughts about the product to think of use cases. I think you can still build good software with varying degrees of AI assistance, but it takes the same effort of testing and user feedback to make it great.
Your licensing only matters if you are willing to enforce it. That costs lawyer money and a will to spend your time.
This won’t be solved by individuals withholding their content. Everything you have already contributed to (including GitHub, StackOverflow, etc) has already been trained.
The most powerful thing we can do is band together, lobby Congress, and get intellectual property laws changes to support Americans. There’s no way courts have the bandwidth to react to this reactively.
Yes, but those are directed by humans, and in the interest of those humans. My point is that incidents like this one show that autonomous agents can hurt humans and their infrastructure without being directed to do so.
(Being AI accelerated doesn't make this project low value. But it does mean _you_ didn't do the port so much as prompt it)
reply