Airtable has engineered a specialized search layer to power its Omni AI and linked‑record recommendations. By storing hundreds of thousands of embeddings in a dedicated vector database, the platform can surface semantically relevant rows in under 500 milliseconds for every query, even when a base holds half a million rows. The result is instant, natural‑language answers that feel native today.

Choosing Milvus as the vector engine let Airtable keep the system self‑hosted while supporting multi‑tenant partitions. Each base received its own partition, eliminating post‑query filtering and simplifying deletions. When partition counts approached 100,000, performance dropped, so the team introduced hierarchical capping: 400 collections per cluster, each capped at 1,000 partitions, keeping latency predictable. The design also supports rapid embedding updates so fresh data feeds the model.

Permission checks remain in Airtable’s primary database, keeping vector search focused on similarity. By isolating data physically, the system guarantees that a customer’s rows never leak into another’s results. The architecture scales to millions of bases while honoring strict privacy and query‑speed targets, proving that thoughtful partitioning can tame the memory‑latency‑recall tradeoff in large‑scale semantic search today efficiently.