HeadlinesBriefing favicon HeadlinesBriefing.com

Predictive Models vs. Treatment Effects

Towards Data Science •
×

Standard methods for selecting variables in predictive models, such as stepwise selection or Lasso, optimize for prediction accuracy. However, a model that predicts an outcome well may still yield incorrect estimates of treatment effects because it can miss crucial confounders.

A 2012 paper by Chi Wang, Giovanni Parmigiani, and Francesca Dominici highlighted this issue as "adjustment uncertainty" and proposed Bayesian Adjustment for Confounding (BAC). Similar frequentist approaches include post-double-selection and double machine learning.

The core problem is that predictive models favor variables explaining outcome variance, while effect estimation requires variables linked to both exposure and outcome. A confounder strongly predicting exposure but weakly predicting outcome might be excluded by predictive criteria, leading to biased treatment effect estimates. This is particularly problematic in finite samples where asymptotic properties offer little comfort.

Bayesian Model Averaging (BMA), which weights models by their posterior probability, can also fail if the weights are driven by the same flawed predictive criteria. BAC addresses this by incorporating an exposure model alongside the outcome model to identify confounders more reliably. The strength of this linkage is controlled by a parameter $\omega$, where a higher $\omega$ forces variables selected by the exposure model into the outcome model, ensuring better confounding adjustment.