Transformers process text in parallel, which is fast but loses word order. Without order, the model can't distinguish "The dog bit the man" from "The man bit the dog." This is the core problem of Positional Encoding, a technique introduced in the original Attention Is All You Need paper to inject sequence information.

Early attempts to encode position with integers or fractions failed because they produced unbounded values or inconsistent distances. The solution uses Sine and Cosine waves to generate unique, bounded vectors for each token position. This creates a deterministic "timestamp" that the model can interpret consistently, regardless of sentence length.

The positional vector is simply added to the word embedding, combining meaning and order. This design allows the model to calculate relative distances between words and even generalize to sequences longer than those seen during training. It’s a foundational component that enables the self-attention mechanism to understand context.