HeadlinesBriefing favicon HeadlinesBriefing.com

Why Betas Explode: The Geometry of Multicollinearity

Towards Data Science •
×

When a senior director sees Linear TV at +2.4 and Digital TV at +1.8, confidence is high. A week later, the same model reports +0.9 and +3.2. To non‑technical stakeholders it looks broken, but the culprit is not a fault—it's multicollinearity, a geometric collapse that makes coefficients explode.

The regression solution β̂ = (XᵀX)⁻¹Xᵀy relies on XᵀX being invertible. When columns align, the matrix becomes singular; its determinant collapses to zero and the inverse disappears. The model then assigns extreme, unstable values to each feature, as the data offer no unique split.

In a data‑engineering bug, two identical columns—say aggressive_TV1 and aggressive_TV2—cause XᵀX to be singular. The solver throws an error. A near‑duplicate, differing only by floating‑point noise, is technically invertible, but the fitted coefficients become absurd, split arbitrarily by numerical noise.

Marketing datasets rarely duplicate columns; instead, Linear TV and Digital TV are highly correlated, with correlation coefficients around 0.85–0.95. They rise and fall together, driven by the same media planners and budgets. Adding a week of data can shift the split wildly, even though the total contribution remains stable, illustrating how ill‑conditioned matrices produce volatile betas.