Edge-Veda delivers on-device AI to Flutter with sub-200ms latency for text, vision, and speech models. The open-source runtime runs entirely on-device without cloud dependencies, using 22,700 lines of code and 50 C API functions to manage AI workloads sustainably on real phones. It maintains model persistence across long sessions while adapting to thermal and memory constraints.

Unlike typical demos that fail under real-world conditions, Edge-Veda provides structured observability and runtime supervision to prevent crashes from thermal throttling or memory spikes. The system uses persistent workers that keep models loaded in memory, applying adaptive policies rather than failing when device resources are strained. Performance tracing and debugging tools help developers understand runtime behavior.

Current capabilities include streaming text generation, on-device speech recognition via whisper.cpp, structured output with grammar constraints, and vector search for retrieval-augmented generation. The ModelAdvisor component scores models across fit, quality, speed, and context dimensions, generating optimal configurations per device. Edge-Veda represents a complete on-device AI stack that makes local LLM inference predictable and sustainable for production Flutter applications.