HeadlinesBriefing favicon HeadlinesBriefing.com

Widening Neural Nets: Geometry Beats Lottery Ticket Myth

Hacker News •
×

Researchers have questioned the long‑held lottery‑ticket analogy that credits overparameterized neural nets with hidden sub‑networks capable of learning tasks in isolation. The comparison likens extra weights to buying more lottery tickets, implying that widening a model simply raises the odds of finding a pre‑trained path. Yet this view oversimplifies how gradients navigate a network today.

Instead, the authors argue that widening expands the dimensionality of the loss surface, granting optimizers more routes to circumvent fragile basins. Loss landscapes become richer, and as width grows, bad local minima shrink in proportion to the plentiful good ones. This geometric lens explains why larger nets routinely converge faster than their slimmer counterparts.

Critics of the lottery‑ticket view point out that a winning subnetwork can be sabotaged by perturbations elsewhere in the model, undermining the isolation claim. The paper stresses that optimization cannot be reduced to a search; instead, it is a descent that benefits from the degrees of freedom. This nuance reshapes how practitioners think about scaling.

These insights urge the community to abandon the isolated‑ticket narrative and adopt a geometry‑centric view when designing or pruning massive models. By recognizing that width supplies extra escape routes, engineers can better target regularization and initialization strategies that exploit the expanded search space. The paper delivers a clearer map for navigating modern deep‑learning landscapes today.