Respan, a Y Combinator W24 company, has launched an open-source AI gateway written in Rust. Founder Andy explains that the gateway is decoupled, meaning everything is a plugin: providers such as OpenAI, Anthropic and Gemini, protocols like MCP and voice, modalities including images and video, and gateway features such as fallbacks, PII redaction and smart routing.
The company says most LLM gateways solve the same core problem of calling multiple providers from one endpoint, but adding new features becomes increasingly difficult. Respan began building on LiteLLM in 2024, and as its wrapper code grew past 100,000 lines while serving thousands of organizations, maintenance became unsustainable. A routine LiteLLM upgrade in January caused intermittent connection errors and forced a rollback, after which the team began manually backporting upstream fixes.
Inspired by Nginx, Respan designed a small core responsible only for dispatching events. Developers can add providers or custom features as plugins without touching the core. The team says a provider can be a short file, with its Mistral integration cited at 22 lines. The gateway also translates between API formats, letting, for example, an app built on the OpenAI Realtime API use Gemini Live, though not every feature maps across.
When client and provider share a format and no plugin modifies the payload, original bytes and provider-specific fields pass through unchanged.
Source: Hacker News · Summarized by HeadlinesBriefing