> Should we as humanity stop right here? Draw a line in the sand and not do anything new?
Maybe someone with the knowledge of hindsight in a few generations will say that, yes, we should stop and draw a line in the sand.
We are still used to the idea that we can treat externalities to the systems we build as infinite and boundless, while it is getting increasingly clear that they are not. I am not saying we should stop working on LLMs. I just say we probably should factor in (that means: assign economic value to) downstream consequences of said high energy consumption if we don't wanna destroy our own habitats. And that then would probably lead to a world where LLMs are used for actually important problems instead of furthering the goals of the few who truly profit from surveillance-capitalism.
But hey, let's continue wasting a years worth of energy on training LLMs on cat pictures, they are cute after all.
There are unaccounted externalities in many areas, but energy use of LLMs isn't one of them - people are paying full market price for the energy of training or using LLMs, that energy cost is large enough so that people care a LOT about it, and make intentional tradeoffs about what is worth it.
And if people vote with their wallets that they are willing to pay for a years worth of energy on training LLMs on cat pictures, we should respect their choice that apparently this is what they consider a valuable use of the limited resources they have.
Jup, and the market price for energy doesn't reflect where it is going towards, which was my point. Whether you train the LLM to predict extreme weather events or put funny hats on pictures of cats, you have to pay the same — in fact, the latter will drive the price up for the former.
Sure, there are still people out there who believe the invisible hand of the market will regulate everything to give us a good outcome despite every real world evidence to the opposite. But in a market more demand for bullshit reasons drives the prices up for everybody, also for things that are essential for the survival of the poorest, the long term sustainability of environment, health services, etc.
The question I laid out was whether it is really a good idea to treat some random bullshit the same as essential services and externalities? Especially considering we are walking into a climate catastrophe that is the result of too much energy use.
Granted, we live in a society that is kinda built on ignoring these connections — otherwise we would go mad or cynical, but lying to ourselves has a hidden cost that will come due at some point.
But as we don't have mass long-term energy storage, future energy is not exchangeable with current energy, thus the future energy cost shouldn't matter for the current training of LLMs, and whether tomorrow it will be a good use of energy to put funny hats on pictures of cats, it's something that people will be able to choose based on the availability of energy - it's up to them to decide what's essential or not, it's up to them to decide whether they are willing to sacrifice some entertainment just to save energy, and that's what they should be doing with money as the mechanism of measure - if a person wants to do something silly, it would be inappropriate for someone else to decide "no, that's random bullshit" and prohibit them to spend their resources (energy or otherwise) on it. The fact that this decision will affect the price of e.g. energy for others is NOT an externality, it's the core economic mechanism of allocating scarce resources - if some people want to spend the same resources on cat picture and they are fully paying the costs of that, then that's directly reflected in the price they're paying, not an unpriced externality.
Are you sure you aren't deep in cognitive-dissonance land now?
More energy use overall — momentary or over time (=work), doesn't make a difference here — especially with LLMs which aren't exactly producing bursty loads and could be in theory scheduled to any time of the day. And energy corps will just have to produce more energy, which has it's own negative externalities on the world.
That means my stupid project still has an impact on the rest of the world. As of now everybody luckily still has the luxury to figure this out themselves. But we have to realize that depending on how bad things get this won't be feasible forever. And how fast things get bad depends on our collective energy use. If you're in a desert with limited water you won't let your kid pour it on the water because it makes a fun sound wouldn't you?
You would only let your kid do that if you had no mental model of water being a limited resouece and no imagination of the consequences a lack of water would mean for you and your kid.
Energy is a limited resource and producing more of it has it's own secondary costs (e.g. for climate change).
Operating with limited resources doesn't mean we are not allowed to have fun. It just means you don't pour your limited resource on the ground for it.
One man's "pour your limited resource on the ground" is another man's "valuable use of that resource" - if they're choosing to do something with the energy they purchase, apparently they consider that this is a worthwhile use of that resource. If I'm in the desert and want to pour some water on the ground for growing orchids, that's my choice on how to use it - if it's really important to conserve water, then it will be costly enough so that I won't waste much of it on luxuries, but if it's cheap enough to do so, then people are voting with their wallets that it's NOT really (yet?) so important to conserve it so much as to refuse spending that on luxuries.
Maybe someone with the knowledge of hindsight in a few generations will say that, yes, we should stop and draw a line in the sand.
We are still used to the idea that we can treat externalities to the systems we build as infinite and boundless, while it is getting increasingly clear that they are not. I am not saying we should stop working on LLMs. I just say we probably should factor in (that means: assign economic value to) downstream consequences of said high energy consumption if we don't wanna destroy our own habitats. And that then would probably lead to a world where LLMs are used for actually important problems instead of furthering the goals of the few who truly profit from surveillance-capitalism.
But hey, let's continue wasting a years worth of energy on training LLMs on cat pictures, they are cute after all.