HeadlinesBriefing HeadlinesBriefing.com

pg_vectorsearch: Fast Vector Search for PostgreSQL

Hacker News •
×

pg_vectorsearch is a PostgreSQL extension for high-performance approximate nearest neighbor (ANN) search, shared on GitHub and discussed on Hacker News. It is currently in alpha and meant for testing rather than production. Upgrading between prereleases may not be possible, so users should plan to drop and recreate the extension and its indexes. The changelog lists known issues.

The extension's PRISM index (Partitioned Routing Index for Similarity Matching) is an index access method conceptually similar to pgvector's IVFFlat, but built around RaBitQ quantization, hierarchical routing and partitioning. It is implemented in C and supports SIMD distance computation (AVX2, AVX512, NEON), dynamic inserts, updates and deletes, and rebalancing of posting lists via prism_rebalance. The project states that PRISM is significantly faster than pgvector's IVFFlat and HNSW for both querying and index building, and that on a 100-million-vector dataset it outperforms HNSW at equivalent recall on an instance with a quarter of the memory.

PRISM uses a hierarchical centroid tree to partition vectors into clusters, relying on the PostgreSQL buffer cache to keep that tree warm. Vectors are RaBitQ-quantized and stored in posting lists. Its error bounds let searches exclude vectors that cannot be nearest neighbors, reducing the number of full-precision reranks needed.

Users can store vectors in native vec32 or vec16 columns, or build a PRISM index directly on existing pgvector vector or halfvec columns without changing their tables or queries. Both paths use the same operators and tuning settings, such as prism.nprobe for trading recall against speed.

Source: Hacker News · Summarized by HeadlinesBriefing