Why The Harness Matters More Than The Model | YC Paper Club

Video summary

YC researchers argue harnesses can lift ARC-AGI performance from 30% to 95%, while Prime Agent and QM demonstrate self-improving, long-running agents.

At YC Harness Club, speakers argue that harnesses are research, not mere scaffolding: the same model weights can rise from 30% to 95% on ARC-AGI with a better setup, with Nvidia’s AVO reportedly reaching 100%. Seth Karten presents Prime Agent, a self-improving RLM harness that manages persistent, messaging subagents and context like layered caches. Its reported tests include 95.5% on ARC-AGI with Opus, plus long-running research and a seven-day Factorio run using 633 agents and 23 million output tokens. Stanford’s OpenJarvis aims to run personal AI locally, reporting up to 800x lower inference cost while using cloud models to optimize on-device configurations. YC’s Josh and Rean explain QM, its open-source work assistant: it centralizes agent sessions, connects company tools, and supports Slack workflows, while requiring human review for sensitive database writes and careful permissions to prevent leaks.

Chapters

1. 0:00Why harnesses matter: ARC-AGI rises from 30% to 95% with a harness
2. 4:27Building an auto-researcher by accident: eight ideas across eight H100 nodes
3. 7:13A five-minute history of harnesses: from GPT-2 loops to recursive subagents
4. 13:56Self-improving harnesses: DSPy prompt search, Darwin machines, and DAgger learning
5. 17:22Tonight's speakers: Seth on Prime Agent, Jon on OpenJarvis, and Josh on QM
6. 18:35Seth Karten: Prime Agent uses an IPython shell to coordinate persistent subagents
7. 21:41Context as an L1, L2, L3 cache: compaction, IPython RAM, and agentic cleanup
8. 24:51From Turing machine to von Neumann computer: persistent RLM subagents and shared state
9. 28:33Messaging between agents: coordination and fair long-horizon evaluations
10. 30:04ARC-AGI results: Prime Agent scores 78% with GPT-5 and 95.5% with Opus
11. 33:09Emulator Bench, a 633-agent Factorial run, and OpenJarvis’s 6–12-month local-model gap
12. 39:21OpenJarvis’s five primitives for an optimized on-device personal AI stack
13. 42:47Cloud models optimize OpenJarvis configurations so local AI can rival cloud workflows
14. 43:53OpenJarvis targets 800x lower local inference costs and a shift from cloud to devices
15. 45:58Josh France and Regan Bell introduce QM, YC’s Slack-based agent harness for work
16. 47:29YC’s internal agents evolved from a January 2025 general agent to Slack-based coding bots
17. 49:24OpenClaw’s appeal led YC to deploy 50-plus Hermes agents and build a better-managed harness
18. 51:04QM pulls agent context into Postgres and makes sandboxes reusable resources
19. 54:43QM lets agents choose task-specific sandboxes and models while keeping database writes reviewed
20. 57:16QM’s grind tool gives agents time and token budgets to persist on goals
21. 58:50Agents lack social context, making fine-grained permissions essential for safe information sharing

What the full analysis includes

21 chapters · 83 key points · 21 section summaries · mind map

The remaining 80 key points, section summaries and mind map are in the full analysis. Analyze this video to see everything →

This video is 1 hours. The Free plan analyzes videos up to 30 min; plans from Plus cover videos up to 2 hours.