Why Robotics Still Isn't Solved - But Could Be Soon | YC Paper Club

Video summary

YC researchers say robotics remains unsolved, but memory, self-supervised reasoning, and simulation-trained dexterity are pushing it forward.

A YC Robotics Club host argues that robotics has been declared “solved next year” for a decade, yet teleoperation remains difficult and four barriers persist: the sim-to-real gap, action representation, limited touch sensing, and embodiment drift. Stanford’s Marcel Torne presents MEM, which adds compressed visual and language memory to robot policies so they can track long tasks, avoid burning grilled cheese, and adapt after mistakes. Milan Ganai describes self-supervised bootstrapping that selects concise, action-predictive reasoning, improving policies across manipulation, navigation, and driving, including models from 1B to 30B parameters. Tyler Lum’s Sim2Real trains one dexterous hand-and-arm policy in simulation; it runs at 60 Hz and handles 12 unseen tools and behaviors zero-shot, while Play to Perfect extends the approach to precise assembly. Rerun CEO Nico urges startups to own customer operations end to end, begin with teleoperation and off-the-shelf hardware, and iterate on real data. General Instinct closes with world-action models and infrastructure optimizations that reduce flow-matching inference from 50–100 steps to one or two.

Chapters

1. 0:00Francois Chaubard: AlphaGo, Aloha, and four barriers to solving robotics
2. 7:59Marcel Torne: MEM splits robot memory into compressed short- and long-context streams
3. 12:04Marcel Torne: MEM enables grilled-cheese timing and recovery from robot mistakes
4. 16:32Marcel Torne: MEM uses annotated text in RAM while exploring learned memory
5. 20:21Milan Ganai: Why VLAs need embodiment-specific, action-predictive reasoning
6. 24:23Milan Ganai: RNB Encore selects useful reasoning across robot tasks
7. 29:48Milan Ganai: Action forcing avoids inference-time reasoning latency
8. 33:42Tyler Ga Wei Lum: SimToolReal controls a 22-DoF hand and 7-DoF arm at 60 Hz
9. 38:18SimToolReal generalizes to 12 unseen tools; Play to Perfect extends it to assembly
10. 42:27SimToolReal Q&A: Random pushes teach recovery, but arm twisting remains difficult
11. 47:17SimToolReal evaluation: Pose tracking caused roughly 60% of failures
12. 51:21Niko West: Start robotics companies with teleoperation and a paying customer
13. 56:27Niko West: Build an early teleoperation learning loop and customer-specific evaluation
14. 59:41Niko West: Robotics data tooling must handle new designs, multimodal data, and full-stack operations
15. 1:03:43Niko West: Rerun’s open-source SDK and teleoperation support lean robotics startups
16. 1:08:30Bill Jiao: World action models predict future frames but face DreamZero-scale compute costs
17. 1:13:35General Instinct: Distillation cuts world-action sampling to 1–2 steps and 500 ms per chunk
18. 1:18:33General Instinct: Future-video dynamics, action sampling, and adaptive key-frame detail