Prism ML has released its Bonsai 27B AI model, claiming it's the first major model of its size capable of running on an iPhone. The AI startup's CEO, Babak Hassibi, told CNBC that Apple is evaluating Prism ML's technology, though discussions are in early stages.

Bonsai 27B reportedly achieves impressive speeds on various hardware, including up to 87 tok/s on an M5 Max chip. Crucially, its 1-bit version is approximately 4 GB, allowing it to fit within the limited memory available to apps on an iPhone, a feat not achieved by conventional 27B models. The model runs natively on Apple devices via MLX and on NVIDIA GPUs via CUDA.

Model weights are available under the Apache 2.0 License, and Prism ML is offering a limited-time developer preview API. While Prism ML is generating buzz by highlighting its communications with Apple, the actual impact of these discussions remains to be seen. The company has successfully garnered attention from publications like The Information and CNBC for its latest release.