Two models, one number, opposite realities. Imagine a sensor recording seismic background, electrical load, or bridge strain, and you've trained a model to forecast the next value. It gives a single number: 0.5. A threshold τ=1.0 fires an alarm. Should you worry? You can't answer because the model lacks a way to tell you what it knows.

Imagine two models predicting x^=0.5 at the same timestep, with identical MSE to three decimal places. By standard metrics, they are interchangeable. But they're not. Model A sees a tight distribution: x ~ N(0.5, 0.01²). The threshold at 1.0 is 50 standard deviations away—alarm won't fire. Model B sees a wide distribution: x ~ N(0.5, 2.0²). The threshold is only 0.25 standard deviations away—roughly 40% chance of crossing.

Same forecast, same MSE, same test set. But risk is ≈0% versus ≈40%. If deciding to evacuate, reroute power, or flag a detector event, those are opposite conclusions. MSE cannot distinguish them.

The problem isn't broken models; both predicted the correct mean. A single number can't express "I'm sure" versus "I'm guessing." This is a direct consequence of the loss function. We'll unpack why MSE hands you the mean and discards everything else, what to replace it with, and the cost.