Judgment is what sets senior professionals apart from their junior peers. Building judgment requires thousands of hours learning on the job, usually through tedious but formative engagement with routine work under guided supervision. But AI is already changing how on-the-job learning works. On one hand, empirical studies show that when AI tools are thoughtfully embedded into training workflows, less-experienced workers can improve their independent performance. On the other hand, recent experiments demonstrate that while AI serves as a temporary exoskeleton boosting immediate output, those gains often fail to persist once the tool is removed.
In “Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting”, published by the National Bureau of Economic Research, we describe a field experiment capturing changes in short-term productivity and longer-term skill building from AI usage in intellectual property law. We randomized access to a then-unreleased Google Labs AI patent writing assistant tool among 133 lawyers at eleven intellectual property law firms. Two-thirds of lawyers (“treatment” group) were given early access to the tool, while the remainder (“control” group) were withheld from using the tool until after the study.
After 10 days of AI assistance, we sent all lawyers a packet of inventor materials for a hypothetical invention and asked them to draft a patent. We did the same after 90 days. AI tool access raised drafting scores by 0.34 standard deviations at 10 days, equivalent to a 10-point climb in percentile ranking among control group scores; by 90 days, this improvement increased to 0.38 SD, implying an 11-percentile point increase. Better performance came from lower frequency of poor scores and higher frequency of good scores, with no change in the frequency of excellent scores. This pattern was particularly clear for junior lawyers, whose improvements were matched by significant time savings.
But patent lawyers also review each other’s work for critical mistakes that can make patents unenforceable in court. We added another task at 90 days requiring subjects to redline an existing hypothetical patent containing many errors. This task mirrors the kind of professional judgment that more senior workers are expected to exercise, testing whether AI assistance enhances or erodes deeper expertise.
Source: Google AI Blog · Summarized by HeadlinesBriefing