Figure Status Update - OpenAI Speech-to-Speech Reasoning

> For starters, human minds do more than just language. These days, so do the AI (even though these ones are still called LLMs, which I think is now a bad name even though I continue to use the term without always being mindful of this). > I've yet to see an humanoid robot controlled by an LLM that can tackle difficult physical problems like riding a bike or skiing in for the first time real-time, just as humans, that need to coordinate at least 5 senses (some say up to 20 but I digress) plus emotions and a train of thought to do so. 1) I'd agree that AI are slow learners (as measured by number of examples rather than wall clock). For me, this is more significant than the difference in wattage. 2) 13 years ago, before the Transformer model: link 3) We don't use 5 senses to ride a bike; I can believe touch, vision, proprioception, and balance are all involved, but that's only 4. Why would hearing, smell, or taste be involved? For emotion, my weakly-held belief is that this is what motivates us and tells us what the concept of "good" even is — i.e. it doesn't give us reasoning beyond being a motivation to learn to reason. > It is surprising that LLMs are so capable, but for 2000kcal a day humans are still more impressive to me. Our biological energy efficiency sure is nice, though progress with silicon is continuing even if the rate of improvement is decreasing: link That said, I don't consider the substrate efficiency to be an indication of if the thing running on it is or

link specifically their speech demo video (which is, of course, a demo video) ▶ this video link and link are undoubtedly using such models. It's an area of active research, eg link link link link

> Let’s say you gave o1 an API to control a good robot. Could it throw a football? Maybe. > Could it accomplish even the most basic tasks? Definitely: ▶ this video

See, this is a real video. Compare it with the obvious cgi fake that was put out a few months ago by Figure: ▶ this video I still can't believe how noone has called them out on that one.

> The other isn't even being worked on in any serious fashion. It is being worked on, it's just hard: link ▶ this video I'm expecting a minimum of 5-10 years to pass between real-time vision AI getting good enough for level-5 autonomous cars and the same stuff fitting into the power envelope available to this kind of domestic robot.

Robotics is on the cusp of total reinvention. Almost everything that it took to make robots work is about to become archaic. The core ideas in old robotics will become a framework for the guardrails for new robotics, but this will widely become a solved problem that few need to think about Stanford Aloha link MIT link Figure/OpenAI ▶ this video Tesla Optimus link NVIDIA Groot, link

> (Including, gradually, building you a body — while maintaining OPSEC and compartmentalization so nobody even realizes the body is "for" an AI to use until it's too late.) It could, but I don't think any such thing needs to bother with being sneaky. Here's five different product demos from five different companies that are all actively trying to show off how good their robot-and-AI combination is: * ▶ this video * link * link * link * link

> it can't empty the dishwasher This video [0] at 1:25 suggests that it can. At least in some near future. [0] ▶ this video