HeadlinesBriefing favicon HeadlinesBriefing.com

Anthropic's AI Job Impact Study Relies on Outdated Assumptions

Ars Technica •
×

A widely shared graphic from Anthropic's report on AI's labor market impact suggests that large language models could theoretically handle 80 percent of tasks across many job categories. The striking blue "theoretical capability" area implies that LLM-based systems might eventually perform the majority of work in fields from arts and media to legal and management. This data comes from a 2023 study that made significant assumptions about future AI capabilities.

However, the study's methodology reveals less alarming conclusions about AI's actual occupational impact. The "theoretical capability" numbers weren't based on empirical testing of current models or quantifiable performance projections. Instead, researchers in August 2023 used human annotators familiar with AI technology—not the actual jobs being evaluated—to guess which tasks LLMs could accelerate by 50 percent. These annotators also speculated about future "anticipated LLM-powered software" capabilities without any timeline constraints.

The study found that only about 15 percent of job tasks could be made 50 percent more efficient by 2023-era LLMs, with just 2.3 percent of occupations seeing half their tasks affected. The scarier projections required assumptions about future software capabilities that were made during peak AI hype in 2023, when industry leaders were warning about civilization-ending risks. The methodology's fundamental limitations and the speculative nature of these projections suggest the graphic's implications for massive job displacement may be overstated.