HeadlinesBriefing favicon HeadlinesBriefing.com

ARGODRIVE Deltafin: Kimi K3 2.8T MoE on Apple Silicon

Hacker News •
×

ARGODRIVE Deltafin runs the full Kimi K3 (2.8T-parameter MoE, 1.45 TB experts) on one M5 Max MacBook Pro with 128 GB, streaming experts from four SSDs. It achieves 1.00 tok/s steady decode over a 512-token answer, 1.13 over 128 tokens, and 0.96 on a 17-token prompt (upstream: 0.68). The honest limit: a 512-token prompt takes ~6.3 minutes to first token.

Cause found: prefill re-reads each layer's experts 8 times; fix planned but not built. Useful findings: one drive gives ~52% of four-drive speed, two ~73%, three ~90% — the slowest drive in each layer's 16 reads sets the pace, not total bandwidth. Every number is one cold run with the exact prompt; per-run logs and placement manifests are in k3-public-bench/.

This fork of gavamedia/deltafin (MIT) includes the engine (see CREDITS.md) and benchmark instruments. Measurements on 2026-09-08 with this fork's configuration of record (k3-public-bench/env.sh). Results: 0.9232 tok/s steady decode (512 tokens), 1.0015 tok/s (128 tokens), 0.92611 tok/s (17-token prompt, median of 3; upstream 0.684).

Time to first token: ~376 s for 512-token prompt. Drive-count ladder: one drive ~52% of four-drive speed, two ~73%, three ~90% (results/SCALING.md). Why prefill is slow and fixes: results/PREFILL.md.

Definitions, identity scope, precision: k3-public-bench/README.md. Run the full, never-pruned Kimi K3 on consumer hardware, as fast as possible. Deltafin is a single native binary, nothing pruned or skipped.

K3 decides every token, all 16 experts, no shortcuts. Quality rule: K3 itself decides, no small draft models. Upstream benchmarks on M1 Max laptop (gavamedia/deltafin, unchanged) show 0.2901 tok/s (3.447 s/token) — 1.9% higher throughput than last update.

Historical M1 benchmarks: 0.2847 tok/s (August 2, 2026), 0.2660 tok/s (July 30, 2026), 0.1311 tok/s (July 28, 2026), 0.0141 tok/s (July 27, 2026). Mission: pure, uncut K3 quality, speed without reducing model quality. Deltafin keeps all 16 routed experts and the full K3 target as sole authority for every token.

Goal: squeeze out every efficiency when running a huge model like K3, all options on the table except reducing quality. This is not a product pitch but an experiment in pushing consumer hardware and learning from the attempt. Kimi K3 targets infrastructure on the scale of 16 nodes with ~4.8 TB aggregate VRAM.

Running it on a single MacBook is an extreme constraint, and every 1% improvement is hard-won. Each gain teaches something; research and exploration are the point. Not everything must be a minimum viable product to impress VCs.

If Deltafin helps make frontier models usable on a $15,000 home setup instead of a $2,000,000 infrastructure, that's worthwhile progress for self-hosted AI. Plus, everything learned could benefit...