HeadlinesBriefing favicon HeadlinesBriefing.com

AI Adoption Lift Is a Selection Effect

Towards Data Science •
×

A slide claims customers who enabled the AI assistant retain 15 points better than non-adopters. Nobody randomized it. The analytics team compared opt-in accounts against non-opt-in accounts. That comparison is not an effect. It is a description of who opts in.

To adopt an AI assistant, someone at the account must notice the release, enable it, trust it enough to put it in front of their team, train people on it, fold it into a workflow, and keep using it after the novelty fades. Every one of those steps reveals something about the account: administrator engagement, executive sponsorship, technical sophistication, product maturity, organizational appetite for change.

The instinct is to model the customer's choice harder. Add covariates. Match on usage. Build a propensity score. This article argues for a different move: Don't model the customers' choice harder. Find variation the customers didn't choose.

The synthetic dataset has 40,000 B2B accounts. The AI assistant is available only to accounts with 25 or more seats, a seat-count eligibility rule of the kind common in SaaS products. Among eligible accounts, adoption is voluntary.