Can Open Models Keep Up With OpenAI and Anthropic? | Nathan Lambert

影片總結

Nathan Lambert says compute—not talent—is China’s main constraint, while Trillium will open-source AI training environments and research.

AI researcher Nathan Lambert tells a16z partner Lisha Li that frontier-model progress depends on compute, data, and talent. He sees Chinese labs as technically strong and roughly equal in talent, but far more compute-starved; open releases also help them reach customers and follow frontier advances. Lambert questions whether distillation alone explains their rapid progress: public research suggests it helps, but does not account for all the gains, while access to reasoning outputs and training methods remains difficult to assess. He expects closed labs to lead initially through capital, integrated infrastructure, and early access to lucrative markets, while open models can win where organizations need private data, fine-tuning, and ownership of model weights. At Trillium, he plans to publish open training environments and document experiments to make research—including recursive self-improvement (RSI)—less opaque. He argues that frontier gains increasingly depend on costly, high-quality data and reliable RL environments, and says exponential compute demands make a sudden takeoff uncertain. His bigger concern is society’s weak preparation for cybersecurity and other risks as capabilities spread.

章節

1. 0:00Compute, data, and talent: OpenAI’s reasoning leap and Chinese open-model strategy
2. 4:35Compute, data, and talent: Open models trail the frontier as Chinese labs fast-follow
3. 8:48Compute-starved, and the distillation debate: China’s compute gap and uncertain distillation gains
4. 14:57Is distillation just competition? Jensen Huang’s view and the frontier compute flywheel
5. 19:24Open and closed on different timescales: Cursor’s SQL test and the slower open-model economy
6. 25:00Frontier progress is really about data: RL environments, reward hacking, and data quality
7. 31:29What Trillium will build: open RL environments and T-Max terminal-task data
8. 38:05Demystifying RSI: publish open training stacks, tasks, and model failures
9. 44:32Can open source pre-train? Solid base models, Megatron forks, and hardware constraints
10. 49:32Harnesses and agent swarms: OpenAI and Anthropic may lead 10,000-agent coordination
11. 53:09Harnesses and agent swarms: Inference economics, scaling limits, and AI risk preparation