HeadlinesBriefing favicon HeadlinesBriefing.com

Robust Linear Regression Against Outliers

Towards Data Science •
×

A simple model with a serious weakness: a straight line can look convincing until bad measurements pull it off course. Linear regression remains valuable for physical interpretation, resource-constrained devices, and computational latency, yet Ordinary Least Squares treats every observation as trustworthy. A faulty sensor, communication error, or biased measurement can produce observations far from the intended relationship.

Because OLS squares every residual, a few such observations can have a disproportionate effect. Robust estimators try to prevent those observations from dominating the fit. Figures 1 and 2 show how quickly the picture can change.

With clean observations, OLS follows the nominal relationship closely. After 30% of the responses are replaced by outliers, the same estimator is pulled sharply away. A representative robust fit, however, stays much closer to the relationship supported by nominal observations.

This article compares OLS as the non-robust baseline with five robust estimators: Huber regression, RANSAC, Graduated Non-Convexity with Geman–McClure loss, GNC with Truncated Least-Squares loss, and Adaptive Selective Outlier Rejecting. The estimators are evaluated using prediction error and runtime to capture both statistical accuracy and computational efficiency. The author developed ASOR in the original study cited, ensuring a transparent comparison with documented settings across Monte Carlo realizations.