Google has significantly updated its Android Bench, a benchmark designed to evaluate large language models (LLMs) for Android app development tasks. The updated benchmark now incorporates eight new LLMs, including OpenAI's GPT 5.4, Anthropic's Claude Sonnet 5 and Claude Opus 4.8, and various Qwen models. This expansion aims to provide developers with a clearer picture of which AI agents excel at coding and related development tasks.

Despite Google's efforts, its own Gemini models continue to lag behind competitors. In the latest Android Bench leaderboard, Gemini 3.1 Pro sits in fifth place. The top spot is claimed by Anthropic's Claude Fable 5, which achieved an impressive 84.5 percent accuracy on the benchmark's suite of 100 Android development tasks. This performance places Fable 5 ahead of other leading models.

The update also introduces new metrics such as cost and efficiency, and supports open-weight models. Google is actively encouraging developers to submit their own test results and feedback, suggesting a commitment to refining Android Bench into a more comprehensive and community-driven evaluation tool. The ongoing performance gap for Google's AI in this specific domain raises questions about its competitiveness against rivals like OpenAI and Anthropic in specialized development applications.