Cactus introduces an on-device AI model, Gemma 4 E2B Hybrid, designed to know when it's wrong by providing a confidence score with every response. This allows developers to use the on-device model for high-confidence queries and seamlessly hand off lower-confidence queries to larger cloud models.

By routing only 15-35% of queries to models like Gemini 3.1 Flash-Lite, Gemma-4-E2B achieves comparable performance on various benchmarks. The innovation lies in a 68k parameter probe layer that analyzes intermediate model states to predict incorrectness, achieving an average AUROC of 0.814 compared to 0.549 for token entropy.

This technique is modality-independent, as demonstrated by its effectiveness on audio benchmarks despite no audio training data. The model and associated code are available on Hugging Face and GitHub, with straightforward integration for frameworks like Transformers, MLX, and llama.cpp. The code is MIT licensed, with Gemma model usage subject to its own terms.