Companies like Spotify need vast amounts of data accessible at low latency for online services and AI agents. Online portals and personalization features must look up per‑user data quickly, while agents answering questions such as "what was I listening to last summer?" rely on fast point queries to build context for LLM prompts.

At Spotify, petabytes reside in Bigtable for online use‑cases, but exabytes sit in GCS. GCS delivers 30–100 ms per request, and newer layers can offer single‑digit ms latency. The bottleneck is no longer storage but query engines like Trino or BigQuery, which add seconds of scheduling overhead even for a single‑row lookup. Random Access Parquet (RAP) bridges this gap by using an external index that maps keys directly to file locations, enabling precise ranged reads on the same Parquet files already used by ML pipelines, notebooks, and batch analytics.

RAP replaces dependent reads with a single O(1) lookup, cached page mappings, and parallel ranged reads, dramatically reducing latency and bandwidth on cloud storage. The index builder reads footers and page locations, scans key columns, and writes a multimap that grows by appending fragments. Index size scales roughly as terabytes of data produce gigabytes of index.

Optimizations such as sorting by key, hash bucketing, co‑grouping, and coarser partitioning concentrate key data, reduce bytes per read, and lower the number of reads, making point queries far more efficient.