I remember back when we had to write mechanize scripts to drive a browser through the renewal process, because if you had dozens, hundreds, or thousands of domains there was no nonmanual way, especially if you wanted extended verification or something silly like that.
So what I'm saying is, agreed and that has always been true.
I was working for a company that had hundreds and thousands of domains. It was a fortune 50 company, and they had a policy generally against wildcard domains.
Instead of registering foo.com, and having bob.foo.com, they would register bobfoo.com - they had hundreds or thousands of certs, as a result, and had to renew them all annually or biannually. We didn't even manage all of the domains, we just had to renew the ones our group was responsible for, and it was in the dozens, and each form was ~30-40 actions to fill out correctly. I forget whether they were quarterly or we just ended up doing it all the time, but it was a maddening task.
Part of the explosion was that domains were not just in one TLD, they were bobfoo.com bobfoo.bar bobfoo.xyz and in many cases regionalized to bobfoo.co.uk or whatever. It wasn't "domain hoarding", because if you registered bobfoo.info you'd just get UDRP'd by corporate anyway, and nobody really wanted domains of the form somelongnamebobfoo.com
This was also before something like LetsEncrypt where you could automatically generate certs for programmatic usage, so they were all done manually.
Should Google be allowed to keep Google dot lol if it doesn't do anything useful with it? What about Google dot wtf? Any person with two brain cells would say these domains should belong to a fervent critic of Google, not to Google.
Sadly neither our world not its legal system is built on common sense.
They're poking at SOX and Dodd-Frank I think, but it's a bit of a specious argument, in my opinion the rise of massive private equity (plus the absurd market orientation around short term results) is more of the story, as you say.
And further, if the goal is to eliminate regulatory frictions in public markets that keep companies staying private, then you're effectively advocating for eliminating public markets.
The entire damn deal we make when a company goes public is that the company can raise money from a much broader, potentially less sophisticated investor base that don't have to be accredited, etc, and in exchange there are more stringent requirements around reporting and so forth.
So sure, we could just throw away that regulatory and social contract, but at that point the public market serves no purpose.
I also think it's moving the goalpost significantly. If you asked me what AGI looks like in 2024, I would have said "smarter on tasks that can be done via text than the average random human you'd meet in a NY bodega", and by that standard, it's long in the rear view mirror.
Now I think people are asking a different question, which is "better than any human at any task that can be done via text", which imo is superhuman, not artificial general intelligence.
It's not moving the goalposts - it's just having a different goal in the first place. This has been Shane Legg and Demis Hassabis' definition of AGI since DeepMind was founded, or before (way before OpenAI appeared), and also for me the goal of AI has always been human level (really a human brain, complete with emotions, etc, but we can start with basic learning/prediction).
I guess I fundamentally disagree, because on general knowledge they beat any human, and on processing speed they beat any human, and on problem solving likewise. I just don't see a domain besides "retain change over time" that they fail on, and that is something you can do in the system rather than the model - coding agents can and do build their own memory system and use it, like an amnestic human writing notes.
I think what I'm objecting to is the idea that if it can't do every task that every human can do better than any human, it's not AGI - I think we have to allow that this is a fundamentally different form of cognition, so requiring a 100% match feels uncharitable. Every human has cognitive gaps that AI doesn't have.
I'm not sure there is anything to disagree on here - just different definitions of what we personally associate the term "AGI" with. There is no magic to the name, and however you choose to define it there will be less and more capable levels of AI that precede and follow it.
Separate from what you choose to call any given set of AI capabilities, learning is generally more than retaining change over time. Learning isn't the same as memorization - it's the difference between memorizing a stack of how-to books and going out and practicing the skill for yourself, and all that entails.
A learnt skill needs to be internalized such that in the future it's what is driving behavior, which means it needs to be in the weights. Imagine a robot trying to learn to play tennis where rather than updating it's connections between perception and action it's trying to store everything as a bunch of notes!
It's not just physical tasks where this applies - the same would be true to trying to teach a multimodal model to recognize different types of mushrooms from photos, or even learning a new language. If a system can truly learn for itself then it shouldn't need to be pre-trained with language, but rather should be able to learn by immersion and practice the same way we do, and the same way a dog learns to follow commands. You can't do this by leaving notes to yourself - the learning mechanism needs to be animal-like and update the systems weights in response to prediction failure - a runtime version of the way SGD-based pre-training updates weights in response to prediction errors.
> If a system can truly learn for itself then it shouldn't need to be pre-trained with language, but rather should be able to learn by immersion and practice the same way we do, and the same way a dog learns to follow commands. You can't do this by leaving notes to yourself - the learning mechanism needs to be animal-like and update the systems weights in response to prediction failure - a runtime version of the way SGD-based pre-training updates weights in response to prediction errors
I am not sure why any of that is necessarily true. I think you are crediting the human brain with a lot more than it actually does - memory is something in the direction of a fine tune on existing neural hardware, it's really not anything special, and I absolutely think that "just taking notes", when taken to a sufficient fidelity, is enough to pass that bar.
You can't take notes before you've learned language, so if that's how you propose to learn then you are doomed to build something with less learning capability than an human brain.
Maybe you don't care - if you just want to push to see how far an LLM can go that's fine. but it's not going to be superhuman-level, or even human-level, if it doesn't have the learning ability of a human. If it relies on language and notes to itself then it is just an LLM.
Okay, nowhere did I say it would be before they learned language. I said we could use models that can trivially build notetaking systems to auto-improve themselves by taking, refining, and updating notes as a memory system to exceed the x token context threshold.
> but it's not going to be superhuman-level, or even human-level, if it doesn't have the learning ability of a human. If it relies on language and notes to itself then it is just an LLM.
Why do you think that? You do a lot of assuming a thing to be true without any clear reasoning or evidence. I'm talking about a system, including a LLM, that acts in a way that is equivalent to or better than an average human at a large range of tasks, such that is generally capable of learning and problem solving similar to the degree of success humans have. All these specific criteria about what is "actually human" is not the point. We know how to make more humans, I'm talking about a form of intelligence that we are creating that is genuinely alien, so holding to "well it doesn't work the same as a human" is an impossible bar. If you reverse the situation, humans are by no means LLM-equivalent intelligences - perfect memory of long form content, rapid calculation, instantaneous tool calling - a hypothetical silicon based lifeform would dismiss us as "really not intelligent at all, they can't even do multiple floating point calculations per second in their head", and I think that would be similarly silly.
OK, so we're basically talking about different things. I'm talking about building something that has animal/human intelligence and learning capability, and would be able to first learn language for itself (no pre-trainiing), and then proceed from there.
You're talking about building an "alien" intelligence, apparently something based on an LLM, where language is baked in from the start and can therefore be used as the basis of some type of learning (or at least memorization).
Sure an LLM can be super-human at specifc things like math where the traditional computer strengths of compute, memory, etc apply, but in the context of AGI (generality, not a bag of narrow intelligences) you can't call something super-human if it can't at least also do the majority of things that a human can do, and not having learning ability as powerful as a human seems to me to be a pretty massive omission.
If WE didn't have ability to learn language, then we'd not be here talking about it, and there would be no LLMs. Having a fundamental ability to learn, isn't just useful to learn language from scratch, or for learning all the non-linguistic skills a human is capable of, but would also support an AI that can learn things that we cannot (e.g. could give it additional "senses" like global pressure/temperature inputs so it could "see" the weather and learn global patterns).
I feel like making a human-proof benchmark is pretty clear evidence that they've exceeded even the highest human capacity in most respects, for things that you can do via text generation (and to a lesser extent image generation)
It's been a while since I messed with serious hardware, what's the cooling noise level? I guess they have to be water cooled or the rack format 8u ones would be about 100db
My 4x 15k rpm 1U fans I use to cool my twin AMD V620s make a horrible racket, especially how they resonate with the rest of the workstation's case fans. Similar in volume and annoyance to running an 8" circular saw. I can only imagine it would probably be as bad or worse than that.
Yeah my experience in the past was that datacenter hardware (air cooled, at least, I've never dealt with water cooled) uses tiny fans at or about 15000RPM, so it matches basically what you're saying or worse.
That was one of those fantastic early google videos/youtube things I probably watched 100 times. This was another from (roughly) that era: https://www.youtube.com/watch?v=ULSA5wktywI
Directionally I think social media is going to be regarded like tetraethyl lead and asbestos. (So is a lot of the ways we use plastic, though maybe not actual e.g. polyethlyene itself)
It's a bit like backtesting, in finance. Quite useful to establish whether your algorithm is sane and coherent, no use at all in determining if it is actually useful, to my understanding.
reply