If you have ever tried neural network weight averaging by training the same architecture twice on the same data, changing nothing but the random seed, you have probably assumed the two results were basically interchangeable. Both runs converge. Both reach the same loss. So you average the two weight vectors, expecting something at least as good as either one alone. The average is worse. Often much worse. There were no problems during the training process; this is a structural feature of neural networks and can be directly deduced from a property known as permutation symmetry.

The same principle also accounts for why model merging succeeds when it does and fails when it fails, a question which, in 2026, is at the heart of practical LLM engineering, whether one is dealing with model soups or federated averaging. Think of two trained networks as two spreadsheets describing the same report, except the columns are in a different order. Model A's "column 3" might hold what Model B calls "column 7." Both spreadsheets are correct. But average them cell by cell without first lining the columns up, and you're averaging revenue with headcount.

Most introductions to neural networks stay in function space. This article stays in parameter space: what the set of good solutions actually looks like, and what that geometry costs you. Classic models like polynomial regression have fixed basis functions and convex loss. A neural network learns the basis functions themselves, making the loss non-convex. This non-convexity is the price of adaptivity. Fix the basis, and we get convexity. Learn the basis, and we lose it. There is no third option.