Modal unveiled Auto Endpoints, a one‑command solution that lets teams such as Cognition, Decagon, Fathom and DoorDash run production‑grade LLM inference on their own hardware. The service promises the speed and cost profile of managed providers while preserving full control over code, GPU selection and deployment pipelines.

Unlike typical managed APIs, Modal exposes every line of the serving stack. Users can inspect engine flags, regional placement and even apply custom patches. Real‑time dashboards surface speculative decoding acceptance length and per‑replica token latency quantiles, giving engineers the data needed to debug and fine‑tune inference workloads.

The platform runs on Modal’s AI infrastructure, which already powers protein folding, robotics and music generation. Its autoscaling runtime allocates GPUs only when needed, avoiding months‑long reservations. New Modal Servers provide regionalized, ultra‑low‑latency routing with roughly 5 ms overhead, eliminating queueing while retaining reliability and elastic scaling.

Auto Endpoints ship with pre‑tuned recipes for supported models, including GLM 5.2, and expose benchmark results at deployment time. Engineers can click to test latency versus throughput, then adjust engine knobs as workload demands evolve. Modal therefore delivers a self‑serve path to owned, high‑performance inference without the overhead of building a custom stack.