A tiny GPT memorized every fact it was given, then couldn't answer a single question that required combining two of them. This article tests a one-line fix proposed in a recent paper, and finds that it only partly works.
The problem builds on earlier research. Balesni, Korbak and Evans found that when a model learns two invented facts from separate documents, its accuracy on questions requiring both facts falls to chance. The effect is called the Two-Hop Curse. When the same facts appear together in one training document, the model can chain them. The author compares this to the Reversal Curse, where a model that learns "A is B" cannot answer "B is A."
To study the effect at a small scale, the author built a synthetic dataset of 200 subjects, 200 bridge entities and 200 answers, linked by random pairings with no real-world meaning. The model, a 4-layer GPT trained from scratch, learned every one-hop fact almost immediately, reaching 100% training accuracy. On the two-hop test, however, it answered none of the 200 questions correctly in any of three runs. This is the Two-Hop Curse in its clearest form: the model knows each fact but cannot combine them without being shown how.
A recent paper by Lin, Chen and Xu proposes a remedy called the identity bridge. For each bridge entity, it adds one training example that maps the entity to itself, signaling that the output of the first hop is the input of the second. The author tests this fix under the same controlled conditions to see how far it closes the gap.
Source: Towards Data Science · Summarized by HeadlinesBriefing