HeadlinesBriefing favicon HeadlinesBriefing.com

litelm: Minimal LLM Routing in 2,900 Lines

Hacker News •
×

litellm's routing + translation in ~2,900 lines and 2 dependencies (openai, httpx). litellm routes LLM calls across providers and translates between message formats. That core is buried under 100k+ LOC of proxy servers, caching layers, cost tracking, and dozens of features most users never touch.

litelm extracts just the call path — model routing, message translation, streaming, tool use, embeddings — and nothing else. No Router class, no proxy, no caching. Install: pip install litelm # openai + httpx; pip install litelm[anthropic] # + anthropic SDK; pip install litelm[bedrock] # + boto3; pip install litelm[all] # everything.

Usage: import litelm; response = litelm.completion("openai/gpt-4o", messages=[{"role": "user", "content": "Hello!"}]). Every function has an async variant: acompletion, aembedding, aresponses, atext_completion. The API mirrors litellm — same function names, same arguments, same response types. Switching is s/litellm/litelm/ in your imports.

What's in: Model routing, message translation (Anthropic, Bedrock, Cloudflare, Mistral), streaming + stream_chunk_builder, tool use, embeddings, text completions, OpenAI Responses API, mock responses. What's out: Router (load balancing, fallbacks), proxy server, caching/budgeting/cost tracking, token counting, image gen, audio, OCR, fine-tuning, agents, guardrails, scheduler. Routes to 19 providers via "provider/model-name" syntax. Any OpenAI-compatible endpoint works via api_base. All provider errors mapped to litelm's exception hierarchy. litelm is human-directed, AI-assisted software.