Why Robotics Still Isn't Solved - But Could Be Soon | YC Paper Club

Video summary

YC Robotics Club tackles sim-to-real, VLA memory, dexterous tools, and faster world action models

YC Robotics Club argues that robotics is still blocked by four hard problems: the sim-to-real gap, action representation, limited sensorimotor feedback, and embodiment drift. Marcel presents MEM, which splits memory into dense short-term visual context and compressed long-term language memory, helping robots avoid endlessly washing dishes or burning grilled cheese and adapt after mistakes. Milan Ganai’s R&B Encore selectively bootstraps embodied reasoning, showing that action-predictive traces matter more than exhaustive descriptions across manipulation, navigation, and driving. Tyler Lum’s SimToolReal trains a single 60 Hz policy in simulation to control a 22-DoF hand and 7-DoF arm, generalizing zero-shot to 12 unseen tools; Play to Perfect extends this toward precise assembly. Nico promotes robotics application companies that begin with teleoperation and off-the-shelf hardware, while General Instinct explains how distillation can make world action models practical on Jetson hardware, reducing 50–100 diffusion steps to one or two.

Chapters

1. 0:00Ten years of “next year, robotics is solved”: sim-to-real, sensorimotor limits, and embodiment drift
2. 7:59MEM: Multi-Scale Embodied Memory for Vision Language Action Models — long-horizon failures and compressed memory
3. 12:04MEM short- and long-term memory: temporal compression and in-context adaptation
4. 16:32MEM discussion: supervised textual memory, RAM inference, and latent-memory limitations
5. 20:21Self-Supervised Bootstrapping of Action-Predictive Embodied Reasoning: grounding and verbosity problems
6. 24:23Self-Supervised Bootstrapping: embodiment-specific reasoning across 1B–30B models
7. 29:48Milan Ganai: Action-Forcing Reasoning Without Inference Latency
8. 33:42Tyler Ga Wei Lum: SimToolReal Runs Zero-Shot at 60 Hz on 22-DOF Hands
9. 38:18Tyler Ga Wei Lum: SimToolReal Generalizes Across 12 Unseen Tools
10. 42:27Tyler Ga Wei Lum: SimToolReal Recovery Comes from Randomized Object Forces
11. 47:17Tyler Ga Wei Lum: SimToolReal Uses LSTM and 60% of Failures Are Pose Tracking
12. 51:21Niko West (Rerun.io): Teleoperation-First Robotics Application Companies
13. 56:27Why the next great robotics companies will start with teleoperation: evaluating customer replicas and training data
14. 59:41Why the next great robotics companies will start with teleoperation: physical-data infrastructure and rapid iteration
15. 1:03:43Why the next great robotics companies will start with teleoperation: lean application startups and scalable teleoperation
16. 1:08:30World action models and what comes after VLAs: Dream Zero performance versus inference cost
17. 1:13:35World action models and what comes after VLAs: distilling flow matching to two steps
18. 1:18:33World action models and what comes after VLAs: future dynamics, edge deployment, and adaptive latent prediction