Linear regression is popular for its simplicity, speed and interpretability, but real relationships often bend, flatten or change across regions. Braking distance, crop response, wind-turbine power and satellite Doppler all show curvature that a straight line cannot capture.
The previous article in this series compared OLS with Huber regression, RANSAC, GNC-GM, GNC-TLS and ASOR to see how a straight-line fit reacts when contaminated observations pull it away from the nominal relationship. This installment keeps the same robustness problem but gives the model more flexibility.
Two questions drive the approach. The estimator controls which observations to trust, while the basis controls which shapes the model can represent. Polynomial features are a natural first step, but their flexibility is global, so changing one coefficient reshapes the curve everywhere. Splines introduce flexibility locally.
The article illustrates the trust question with a nonlinear example. With clean data, OLS closely follows the underlying relationship. After 30% of the responses are contaminated, the OLS fit is pulled far from the true curve, while a robust fit stays much closer to it. Polynomial regression is then framed as a linear model in a transformed feature space, with coefficients estimated by least squares on a design matrix.
Source: Towards Data Science · Summarized by HeadlinesBriefing