Cerebras explains how it runs massive AI models across multiple wafer‑scale chips. A single WSE‑3T wafer offers 44 GB of on‑chip memory, but trillion‑parameter models need terabytes of weights. Using the open‑source Kimi K3 (~3 T parameters) as an example, the team splits the model across 59 wafers. Dense layers, which require a KV cache, occupy one wafer each, while linear‑attention layers, needing only a small fixed‑size state, allow two per wafer. The compiler maps layers to wafer regions, and the runtime routes activations between wafers via RDMA over Ethernet (≈2 µs per hop). Each wafer acts as a pipeline stage, processing different requests concurrently to keep the pipeline full and achieve 1000‑2000+ tokens per second. This approach maximizes throughput while minimizing weight movement, enabling efficient deployment of multi‑trillion‑parameter models.
Key details: 44 GB per wafer, 16 GB per layer, 69 linear‑attention layers, 24 dense layers, 59 wafers total, 2 µs inter‑wafer latency, Kimi K3 model.
Source: Hacker News · Summarized by HeadlinesBriefing