Transformer is a neural network architecture that has fundamentally changed Artificial Intelligence. Introduced in the 2017 paper "Attention is All You Need," it powers models like OpenAI's GPT, Meta's Llama, and Google's Gemini, and is applied in audio generation, image recognition, protein structure prediction, and game playing. Text-generative Transformers predict the next token using self-attention, capturing long-range dependencies. Transformer Explainer is powered by GPT-2 (small) with 124 million parameters, sharing architectural components with state-of-the-art models.

Every text-generative Transformer has three key components: Embedding converts tokens into numerical vectors capturing semantic meaning; Transformer Block processes and transforms input data with Attention Mechanism (allowing tokens to communicate) and MLP (refining each token's representation); Output Probabilities transform embeddings into next-token probabilities.

Embedding converts a prompt like "Data visualization empowers users to" into a numerical representation. Steps: 1) Tokenization breaks text into tokens (GPT-2 has 50,257 unique tokens); 2) Token Embedding represents each token as a 768-dimensional vector; 3) Positional Encoding; 4) Final Embedding. These steps enable the model to process and generate text.