A walkthrough of using low-capacity neural networks to derive fine-grained scoring from categorical labels.

The problem involves predicting patient outcomes for "Pathogen A" using 8 features. Three categories exist: home recovery, hospital recovery, and death. While standard classifiers achieve 98-99% accuracy, health authorities need a continuous risk score.

A naive approach assigns numeric values (0,1,2) to categories and trains a network with MSE loss. However, this causes the model to cluster outputs around the target values rather than producing meaningful continuous scores.

The solution uses a bottleneck architecture with a linear scoring layer and a trainable category approximator head. The encoder produces a continuous score, while the decoder learns thresholds between categories using softmax. This enables extracting meaningful thresholds (t0, t1) that separate the three outcomes, providing interpretable risk scoring from categorical data.