I don't have the historic numbers at hand, but I would assume that for each of those categories this is a similar proportional increase, so it's similarly notable to mention the increase in spending on humanoid robots.
Androids (humanoid robots) will require loads of ram, and loads of model training under the current paradigm. So it sort of makes sense. At least, I see robots as the top of the pyramid.
Autonomous (non-teleoperated) humanoid robots that can do useful work in an unfamiliar environment do not exist. And nobody's close enough to making them to understand if they're possible with our current level of technology, let alone how.
Most initial work for them would be in familiar, well-controlled environments - replacing humans in existing factories. I think whether they'd be cost effective for that will remain unknown even after a few years in service though.
Factory work that can be straightforwardly automated by robots generally already is, by special-purpose factory robots. The remaining tasks are generally:
* Tasks where poorly-paid humans are cheaper than expensive factory robots. Humanoid robots are more complex / fiddly so will be more expensive than existing factory robots. No help here.
* Tasks where human dexterity has an advantage over state-of-the-art robot actuators (e.g. sewing fabric panels into garments). Better robotics could help here, but the advancement needed is better actuators, not AI and a humanoid form-factor. And if you solve this you'd be better off putting your new end-effector on an existing 6dof platform.
* Supervising robotic equipment and handling exceptions. But then you get to "handling poorly-specified unfamiliar tasks in the physical world" which is not currently a solved problem and there's no guarantee just throwing more compute at it will be sufficient to solve this. So far all humanoid robot demos have either been either known tasks in a tightly-controlled environment, teleportation, or so poorly-functioning as to be obviously not fit for purpose.
I'd also point out that what makes this whole thing smell of being a grift is the fact that what's being chased is humanoid.
Humans are not the pinnacle of dexterity or stability. And freed from biological constraints, it makes no sense why we'd use the human form as a reference.
Making a humanoid robot is a hard thing to do, for sure, but it's also not particularly useful. The routines needed to balance a robot or correct for a slip are interesting to solve, but don't really make for a better robot which is more capable of doing the dishes or folding the laundry.
If I'm amazon, for example, then the most useful form factor for a general purpose robot is 2 arms on a 4 wheel omnidirectional rolling platform.
Our hands are actually amazing, to a degree that’s hard to comprehend unless you’ve tried to build something equivalent. Our finger tips can discern features down to around 10 microns, detect a wide range of vibrations and single shocks, manipulate objects two orders of magnitude smaller than them, grasp an incredible range of weird shapes and materials reliably, repeat pre-learned motions remarkably quickly, and with training carry kilograms, all while weighing about a kilogram and reliably operating maintenance free for tens of thousands of hours. Oh and they’re made of meat.
An artificial manipulator with even half of these features would be the holy grail of robotics. Doing them all at once is a miracle.
Completely agree. My point is more that even though our hands are amazing and even if we decided we want all the features of hands, there's no reason why you'd design such a device with 5 fingers and an oddly placed opposable thumb. There's no reason you'd have more than 2 joints per finger.
That's what I'm calling a grift of the humanoid robots. They are focusing pretty heavily on making manipulators shaped like hands, which don't perform anywhere near hand capabilities. The form is mattering more than the functionality which introduces a lot of unnecessarily hard to solve functionality problems.
The actual hard part of doing something like folding cloths isn't even the manipulation (though it's part of it) you mostly just need a few pincers. The hard part is that fabric changes like crazy in a 3 dimensional way on every motion. Just identifying "this is a shirt" is a hard problem to solve. Further deciding how to fold said shirt is extra difficult. But even further, getting a robot to know "this is where this shirt should be put away" and "this is how I should handle new cloths" is crazy hard. That's the part that I've seen absolutely no evidence that any of these humanoid robots are making any sort of progress on. The most impressive ones are cheating with a guy in the room over doing the task. Not exactly something I'd want in my home. Perhaps in a nuclear reactor.
I get your point about strictly anthropomorphic robots. There's definite scope for improvement there - I think Boston Dynamics' recent videos on their design process for the hands for Atlas is fascinating, and the way the fingers (in fact most/all(?) of the robot's joints) can flex both ways is a definite improvement. I also think that pure legged locomotion is dumb in an urban environment, compared with wheeled legs like a lot of recent robots are using.
I would say that turtles are the pinnacle of stability, and longevity, and so we have Roombas. But, Racoons are at the pinnacle of dexterity, and Octopodes are at the pinnacle of dimensional manipulation...
but personally, I would welcome our cyper-coon overlords.
So far all humanoid robot demos have either been either known tasks in a tightly-controlled environment, teleportation, or so poorly-functioning as to be obviously not fit for purpose. It's possible throwing more compute at the problem will work and they'll be useful in a few years. Or maybe we're actually experiencing a Markov-chain / ELIZA moment and are multiple decades away from anything actually useful.
Self driving cars is far from a solved problem. It's something we've been working on for the last 20 years at least. We are getting closer, some solutions are impressive (like waymo). But even those ultimately need operators in the area of the cars to solve the problem of the car getting stuck.
Self driving cars are an infinitely simpler problem to solve vs a general purpose humanoid robot. You have basically 2 outputs, acceleration and steering. You have rules simple enough that I was driving at 14. With enough input, you'd think self driving could be completed in a snap. But it's not there yet.
Humanoid robots which are useful will come after widespread deployment of L5 self driving cars. Since we don't have that, I have no faith that we are close to useful humanoids.
"I have no faith that we are close to useful humanoids." I agree completely.
We still have competitions about having robots walk up stairs and open doors,
and balance while doing those things...
To replace humans for general tasks. McDonalds would like to use them to flip burgers, take orders, and deliver meals. Amazon would like to use them to pick items from their warehouses for delivery.
I'd argue, though, in both cases the "humanoid" part isn't what they really want or need. Even in the mcdonalds case, so long as the robot is "cute" enough to interact with customers, then it can be a box with an arm.
Of course, a major problem with using robots in the food industry is cleaning and hygiene. Robots tend to have lots of hard to clean cracks that will happily carry around germs. Even in medicine, we are finding that things like laparoscopic devices are beasts that are almost impossible to fully clean.
We said the same thing about Waymo, that it was perpetually in the future. It took them less than a decade. The robots today are functionally capable, they don’t have the right fuzzy intelligence yet. It’s purely a data problem (lack of) and a lot of people are working on it.
From what I've seen with Waymo, autonomous driving in relatively good conditions (light to medium rain, fog, or sunny) is a solved technical problem, even in small, winding streets. Snow is the next big hurdle, and they're actively working on that in Denver and Detroit.
They're being conservative with their rollout mostly because of civic issues. Most places do not have a legal framework for "what if your autonomous vehicle hits someone?" yet. Even if Waymos never were at fault for a collision with a person, you can always have cases like the one in Georgia back in October where a bicyclist wasn't looking where they were going and rammed into one. The shaky legal ground is a pretty big impediment right now, and that's in the US where we have much laxer laws about corporations killing people.
Yeah, I suspected that, but legislation and dealing with who is responsible and what happens if an autonomous vehicles directly kills/hurts someone or is involved in an accident it IS part of the problem, luckily.
Then, there will be concerns about the scaling costs. If it makes economical sense only in densely populated zones, what's the point? I mean, yes, it will be yet another business that works only in cities in that case, but that will not then make it universal as cars are.
If they can pull off year round driving in Manhattan, and they have, it’s a solved problem. Rollout it slow due to bureaucracy, appetite for progressiveness and catering to entrenched business interests but it will show up everywhere it’s profitable eventually.
It's not just a data problem, it's a hardware problem. Transformer-based robots require even more processing power than plain LLMs, as they also need to process visual and spatial/touch input. We don't have GPUs capable of fast inference on a SOTA LLM that would fit in a robot brain form factor, let alone also run fast enough spatial and visual processing. And there's currently nothing even approaching a feasible solution for cooling such a device.
I have been following them since they were Halodi robotics. They're cool, but nowhere near the level of autonomy you need. My theory is that before household robots become a thing we should have self driving cars be a common occurrence, since that is a much much simpler problem.
If there's no unknown unknowns in the brain, it's most likely possible. As the universal approximation theorem and empirical results of scaling SGD+RL suggest. Whether it will be economically viable remains to be seen. The human cerebellum has a peculiar structure and 80% of the brain's neurons after all.
The parameter count equivalent of a human brain is not yet known, but if it was one per synapse then a full human brain replica would need about 1.5e14.
We also don't yet know how to be as efficient with training examples as any living creatures' brain, and we only partially make up for this by training on so many examples it would take you a million or so years to do the same, so we'd still stuggle with something proportionally smaller-brained such as a cat.
That said, remote controlled androids are going to be economically disruptive, as they make every (unlicensed) job open to outsourcing from an office in a low wage country.
Real neurons are orders of magnitude more complex than their artificial pseudo-approximation (it is all based on the century-old understanding of how neurons work). You can think of _individual_ biological neuron as an analog of the small artificial neural network. You can see this simple visual explanation on YouTube[1]. So we aren't even close. It doesn't mean the AI is impossible, it just means people underestimate the "computing power" of real brains, as well as that AI, even the future one might be totally different in how it works from the natural intelligence.
Brain's information flow also isn't just through direct neuron-to-neuron connections. Firing also releases neuromodulators into the extracellular space which affects how other neurons operate. Furhermore neuron connection architecture is very different between brain and feedforward ANNs, with the former exhibiting a lot of recurrent connections.
But for biological neurons to do something that can't be efficiently approximated on a digital computer (but conductive to useful information processing) they need to have unknown unknowns (well, partial unknowns like an unknown quantum algorithm will do too).
We don't know the violations of the physical Church-Turing thesis that are conductive for machine learning. We don't have evidence for their existence in the brain (although, the brain would be the prime candidate for finding them as evolution works directly with the true physical laws).
BTW, large ANNs don't try to model how the brain does things. They are trying to mimic what the brain does. So, using "how many transistors/artificial neurons it takes to model a biological neuron" is not a good approach.
We have no evidence. We even have no solid theories how this can work (Penrose's OrchOR is "OrchOR somehow taps into mathematical knowledge somehow encoded into the structure of spacetime"). But people, for some reason, insist that there should be something there. I can't attribute it to anything else but to deeply entrenched feeling of human exceptionalism.
You're talking about a philosophical debate whether the brain is computable, the other commenters are pointing out that even conservative estimates point to a brain-like NN requiring over a quadrillion parameters.
Let's define the terms. What does it mean "with similar capacity"? As far as I understand xvilka was taking about the number of artificial neurons required to model a biological neuron times the number of biological neurons in the human brain. It is modelling its physical structure (on a neuronal level).
That is we have unknowns that might increase or decrease estimates of computational demands of a functionally equivalent ANN. Not everything that happens in the brain contributes to information processing.
If you're running a massive model for logic you're probably better off not putting it in the robot. And it'll be a long time before there's enough robots to make up a significant share of usage.
More basic movement control doesn't need loads of ram as far as I know.
The same logic for why self-driving cars can't be cloud based, applies for robots. Something cannot be in the middle of a delicate operation and then "oops!", no network, it just stops.
The larger the context window, the better with models. Having a few TB of RAM would be exceptionally helpful.
All this just made me realise something however. Having your robot dormant and charging, is a bit of a waste. You could have robots dormant, but its compute in use to act as a compute node. If the distribution of robots is similar world-wide, we'd need a fraction of the datacenters we have now.
Using such nodes for training purposes would be beyond advantageous. And the company which can slice up the work and having training done in batches would get the big bucks. And actually, with consumer facing products soon all laden with extra ram and gpu for local compute, that applies there too.
Imagine leasing out idle time on your desktop or even laptop for cash. There may be a market here, especially with the cost of new datacenters. Any company able to securely package compute without risking data safety is going to make a mint.
> The same logic for why self-driving cars can't be cloud based, applies for robots. Something cannot be in the middle of a delicate operation and then "oops!", no network, it just stops.
I don't think you understood my post. The equivalent of self-driving is the movement control I was talking about.
Self-driving cars don't have high level logic, except for route planning. Which often is offloaded to the cloud. An extra 30 milliseconds on understanding your speech is nothing.
> Imagine leasing out idle time on your desktop or even laptop for cash.
You're missing the point. The issue is "how is the thing used" not trying to make identical break points in differing tech. A self driving car cannot have object detection, collision avoidance, human detection offloaded to the cloud. Ever. At all. A network drop can't mean it smashes into things. Or stops unsafely.
The same is true with an android. Imagine it turning on an frying pan, cooking dinner, and then going offline part way through. Or turning on a tap to wash something, and going offline while the sink overflows and destroys the house.
There are myriad of such scenarios, but local compute is absolutely, 100% necessary. Anyone betting the farm on putting network controlled devices into homes for any serious task is going to lose their shirt. Local compute is an absolute requirement, and a few TB of RAM and local compute will be nothing over the scope of this discussion (a few years minimum, just to build and kick off all these new fabs).
By the time these fabs are online, expect most smart phones to have 1TB of RAM and significant llm capable compute (gpu or other custom silicon). I would be astonished if flagship model phones in 2030 weren't sold with 1TB RAM. Note I'm saying flagship, there will be of course economy models as always. Certainly laptops will be sold in multi-TB RAM configs.
I'm not missing the point. I'm saying that some amount of local compute is necessary but the really big stuff can be remote.
You don't need terabytes to turn things back off.
Not that I want to trust "not setting the house on fire" and "not flooding the house" to this kind of model in the first place...
> I would be astonished if flagship model phones in 2030 weren't sold with 1TB RAM.
I'll be astonished if they have 50GB.
Have you looked at RAM size/price trends? I'm not even talking about the last year, just the pattern before that. We're not in the 80s and 90s anymore. The most recent price lows were roughly $3.50/GB in 2013, $2.50/GB in 2016, and $1.50/GB in 2023. If we're lucky the cheapest stuff will hit $1/GB in a few years, and the kind that would actually fit on a phone motherboard would be significantly more than that.
Samsung hit 16GB on their top model in 2020 and it's either been 16GB or 12GB ever since. Apple only went up to 12GB in the last year. Google offers 16GB. A couple niche offerings have 24GB. Why would these RAM numbers even double during the next four years?
Well there's only one reason, so you do indeed know why.
Here's the thing, by 2028, maybe 2029, 1/2 the cloud AI providers are going to go bankrupt, or severely downsize. This will likely be due to massive reduced power and processor requirements for AI. GPUs are a stopgap, and they won't be used much longer. When that happens, and the number of servers drops by 90%, along with similar power requirements, a lot of people are going to go tits up due to massive, empty datacentres and compute.
That will absolutely flood the market with cheap obsolete GPU + RAM, used servers, you name it. Current RAM prices will plummet, and those able to withstand the storm will have to compete again, not just ride the wave like they've been doing now.
RAM has been ridiculously high for a long time, it won't be by 2030.
Couple that with local SoC with dedicated compute for LLMs, and you have the perfect storm for local, on phone compute. Why this? Well, because LLMs will be in everything! Certainly in robots, autonomous weapons, and anything the crazies can shove it into. Phones won't be the unique case, it'll be sharing a hardware market with robots, and 100 other things using the same SoCs.
Local compute will be easy... RAM makes a massive difference, huge as you know, difference in local compute. And with the datacentre market in tatters, and people becoming more and more concerned about privacy, people will likely not want to whisper to a cloud LLM, about their fetishes for donkeys or moose or whatever turns their crank.
Not to mention, other aspects of their lives.
In as RAM will be super cheap, that's where I see this going.
I have a sneaking suspicion you don't agree with this assessment. I guess we'll come back here 2030 and see what's what.
What do you think the production cost of RAM is? Even if we perfectly fixed competition and ended the AI bubble right now, I don't see how new RAM prices would drop below half the previous low point by 2030. Those fabs are legitimately ultra expensive, that's why almost all the RAM makers went out of business and we're left with the current cartel.
Also "massively reduced requirements for AI, so much so that we have 90% fewer servers despite the Jevons paradox" is not compatible with "high quality models make effective use of terabytes of memory" and "LLMs will be in everything".
And even if you could perfectly reuse all the memory out of the collapsing datacenters, that's not enough to make many of the phones and laptops you're describing. We'd need a 10x increase in manufacturing on top of the massive price drops, and even if we started building those fabs today they wouldn't be done in time (and we wouldn't have enough equipment to fill them).
So... like vertical markets? Ram for the data centers, data centers for the robots, robots for the Humanoid? or... the other way around? Robots to build data centers, and robots to build Ram.
I see... tech giants and R & D at the top of the pyramid, and I have seen Gumee, the fab that started all this Korean ram prodution.
+1 - Exercise gives the benefit of calories expended for weight loss, but it really isn't enough to matter for the average person. Exercise is great for improving your cardiovascular function, your muscles, your neurological pathways (could be butchering the terminology) for those movements, and it often leads you towards other good things, like spending time with others and getting outside.
I don't think you understand macros if you think a breakfast of yogurt, berries, and milk is avoiding sugar. Berries are mostly sugar, and lactose is a sugar which makes up a significant portion of yogurt and milk's calories. Your breakfast is close to 50% sugar. I would hate to see what the macros look like when you're not "avoiding all sugar".
Also 150 calories of whole milk is 8 fl oz. How big is your morning latte? Milk is a great food for bodybuilding, because it's easy to consume a bunch of calories without feeling that full. This makes it less good when you're trying to lose weight.
> apparently, 10 pounds of weight loss is roughly a clothing size [XL → L → M]
What? This is so wrong I'm confused how it could have possibly made it in. As a 5'7" guy, I was a M at 145, and still an M when I hit 175, though at that point I was close to an L. There is no way I was 3 separate sizes during that time
Something feels off / sloppy with the article in general and that’s gotta be the smoking gun. I cannot think of a single case where this is true, I’ve lost 35 pounds in the last 6 months, and I’m still a medium. 5’9”.
This rule of thumb, when used correctly, is supposed to refer to womens' dress straight sizes. As in, for an average height woman, losing 10 lbs is roughly equivalent to going from a size 8 to a size 6. It doesn't refer to clothing that is sized as "medium", "large", etc. (I did not read the article; it seems from the quote that the author was using this rule of thumb inappropriately and out of context.)
This is the most significant record in running to fall since 1954 when a sub-4 minute mile was run. I think running can be front page news once a century
I get it and agree, but historical significance doesn’t factor into what they put on front page or what is popular at the moment. It’s not a slow news day for sports and they don’t think their viewers care enough. I’m sure if we had their data it would show us that they wouldn’t.
It’s not meant to be malicious they just don’t report on things that don’t get enough engagement. If you look at the long list of sports they cover, there’s nothing running related even mentioned. They do now have an article on it in their Olympics category as of 2 hours ago. But I feel like them not having a breaking news coverage on a Sunday in this sport is to be expected more so than your expectation of them covering it.
Can you be more specific here? In an article about civilization 5300 years ago, where India has had a human population for at least 65,000 years, saying "always" is fairly vague
Sorry I was referring to mechanical techniques of spinning a drill in general, that I'm familiar with, in addition to the bow method described. I was not referring to any time scale.
I want to be as polite as possible, but explain the confusion that timerol was experiencing, just in case it was unclear. Because the article's focus is about the oldest "sophisticated drilling tool", and your comment said "Indian carpenters always had ...", the implication is that you're disagreeing with the article being discussed. The information you share is interesting, but it's slightly off topic from the main point of discussion (ie, the age of sophisticated tooling). You can avoid confusion like this by explicitly acknowledging when you're going to go off-topic. Eg, you could say "Not related to the age of the tool, but I saw similar tools used in India -- Indian carpenters always had..." I hope that explanation is polite and maybe helpful :). Apologies if I over-explained.
A status symbol will easily run you $1000/mo. I currently pay $350/mo (including cost of capital), and I don't know how I would pay less for a car that's not actively falling apart. Chevy Spark, manual transmission, $7k KBB value, averaging 500 miles per month.
There's no shot that number isn't being driven up by people purchasing more car than they need. You can get a used car for $10,000 or less, there's no reason one needs to pay $500/mo.
You sure about that? A $7k car expensed over 18 months plus insurance and road tax is ballpark $450 /mo. That omits maintenance and fuel but conversely also underestimates how long the vehicle will be kept on average. Depending on where you live parking will range from free to potentially more than $100 /mo.
If you manage to stretch $10k cars out to 5 years on average with zero maintenance it's less than $200 /mo but ... no maintenance in 5 years?
I think $300 /mo plus fuel and parking is probably a reasonable estimate for frugal behavior.
