概念

KV cache

1:50:48Why Inference Matters 12x More Than Training | YC Paper ClubY Combinator · 2026-10-24YC’s inference panel says serving can use 12× more FLOPs than training, while SGLang’s cache cuts time to first token by 56%.↪ 提到於 5:49
44:15Inferact: Building the Infrastructure That Runs Modern AIa16z Deep Dives · 2026-01-22Inferact’s vLLM founders aim to build an open-source inference layer for trillion-parameter models across 400,000–500,000 GPUs.↪ 提到於 21:26
1:09:00EP148 - Kimi K3 震撼矽谷！真的這麼強嗎？科技浪 · 2026-07-20月之暗面 Kimi K3 登上 AI 模型第三名，但 Token 效率差抵銷低價，AI 競爭仍利多算力與記憶體。↪ 提到於 32:52