Welcome back! Many of us now try to find time to walk or exercise regularly, even as our lives grow busier. The author decided to make running part of a daily routine, and like many people, chose to buy running shoes online. Searching a shopping app for "running shoes" revealed a puzzle: products were also labeled walking shoes, sports shoes, and sports sneakers, even though the query used only one phrase. That raised a question about how the app knows these terms are related, which led the author to the concept of word embeddings.
Embeddings matter well beyond shopping apps. Large Language Models use them to process prompts, and AI assistants that retrieve web information for responses, a method known as Retrieval-Augmented Generation (RAG), rely on them too. Modern search systems often combine keyword matching, embeddings, and ranking models, but embeddings are central to identifying semantic relationships and retrieving information by meaning rather than exact word matches.
An embedding is a numerical representation of something, and in word embeddings each word is represented as a vector of numbers. This differs from TF-IDF, a statistical method that weighs terms by how frequently they appear in a document and how common they are across a collection. Word embeddings instead learn a vector for each word based on how it is used alongside other words, so relationships are learned from the data itself.
The next question is what a learned vector representation means and how that learning happens. Using an online shopping search system as a simple example, the idea becomes clearer: words used in similar contexts end up with similar vectors, which allows a system to connect "running shoes" with "sports shoes" even without an exact match.
Source: Towards Data Science · Summarized by HeadlinesBriefing