Artificial Analysis Intelligence Index v4.3.2 evaluates leading models across ten benchmarks including AA-Briefcase v1.1, GDPval-AA v2.1, Automation Bench-AA, Terminal-Bench 4.0, Sci Code, Humanity's Last Exam, GDP.pdf, Crit Pt, AA-Omniscience, and AA-LCR v1.1. The updated index measures capability across agentic knowledge work, scientific reasoning, software engineering, and legal tasks. No publicly available API exists for direct interaction with these models.
Evaluations are weighted to calculate a cost per task, factoring in input, cache, reasoning, and answer token prices. Models are categorized by commercial availability, labeled as "Commercial Use Restricted" or "Non-commercial." The index provides capability indexes measuring performance in specific industries and use cases, with AA-Briefcase v1.1 serving as an agentic knowledge work benchmark aggregating analytical quality, presentation, and rubric pass rates. AA-Omniscience Index assesses knowledge reliability and hallucination rates.
While model intelligence generally translates across use cases, specific evaluations may hold more relevance depending on the application. The analysis covers model weights availability and commercial licensing terms alongside performance metrics.
Source: Hacker News · Summarized by HeadlinesBriefing