AI startups face runaway bills from LLM APIs like OpenAI. Traditional rate limiting by requests per minute fails because a single prompt can consume thousands of tokens. The new standard is TPM (Tokens Per Minute) to protect both latency and budget. This shift is critical as companies scale their AI operations.

Engineers use three algorithms to regulate token flow. The Token Bucket allows bursts for quiet users, while the Leaky Bucket ensures smooth, predictable processing. The Sliding Window prevents users from exploiting minute boundaries. For multi-tenant systems, Tiered Priority models separate free and premium users, with a circuit breaker to halt spending.

Beyond simple throttling, advanced systems implement Cost-Aware Model Routing. Simple queries route to cheaper models like GPT-3.5, reserving premium tokens for complex tasks. This turns a rate limiter into a financial planner, preventing budget overruns and ensuring system stability as AI applications grow.