Robot-Use Agents: Why General-Purpose Models May Win in Robotics

影片總結

Waddle Labs and RoboCurve argue LLMs could control general-purpose robots within two years.

Waddle Labs and RoboCurve describe a shift from robot-specific training toward general-purpose LLM agents that control many machines through tools, code, and reusable skills. RT2 showed how pretrained language models could output robot end-effector poses, while newer systems such as Astra can inspect camera feeds, call robot-control tools, and complete tasks like moving blocks, unscrewing caps, and uncapping pens. The speakers argue that coding data, computer-use interactions, CAD, and egocentric video may teach spatial and physical concepts more effectively than limited robotics datasets. In-context learning adapts quickly but saturates after roughly 20–40 examples, making skill compilation and later distillation essential for reducing latency. RoboCurve reports model latency improving about 2× per month, potentially enabling real-time control by year-end. The panel predicts competent general-purpose robots within two years, while warning that society and the robotics economy may be unprepared.

章節

1. 0:00Intro + The Rise Of Robot Use Agents: Philip Isola’s Robot-Use Agent Era
2. 4:07The Bitter Lesson for Robotics: General Data and Pretraining Beat Narrow VLA Training
3. 7:40From Coding Agents to Robot Policies: Voyager and One-Shot Python Control
4. 10:41In-Context Learning vs Model Training: ICL Saturates After 20–40 Examples
5. 14:22Building a Harness for Robot Control: Camera-Guided Blocks and 2x Monthly Latency Gains
6. 17:50Turning Robot Actions Into Reusable Skills: Deterministic Policy Graphs with VLM Branches
7. 20:55How AI Models Learn the Physical World: Platonic Representations, CAD Data, and Computer Use
8. 26:05How Close Are We To General Purpose Robots? General-purpose robots expected within two years