GitHub released wayfinder-router, a CLI tool that routes queries between local and cloud LLM models without making any model calls to decide. The tool analyzes prompt structure—length, headings, lists, code—and difficulty cues like math or constraints to determine whether to send requests to a small local model or expensive cloud backend. Decisions happen in microseconds and require zero network connectivity during routing.

Most existing routers call a model to classify prompt complexity, adding latency, cost, and randomness to the routing step itself. Wayfinder avoids this by reading structural features only, with lexical analysis available but disabled by default after testing showed poor generalization. The approach prioritizes deterministic, offline operation over perfect accuracy, acknowledging that semantic-difficulty prompts still pose challenges.

Users configure two tiers in a toml file—one local endpoint like Ollama, one cloud provider like OpenAI or Anthropic. The gateway proxies OpenAI-compatible API calls, automatically routing easy prompts locally while sending complex ones to premium models. Headers expose which model handled each request and the routing score.

While not perfect—random accuracy on short-but-hard benchmarks—the tool delivers concrete savings by keeping simple queries off expensive APIs. The tradeoff: deterministic routing versus potentially more accurate but costly ML-based classifiers.