HeadlinesBriefing favicon HeadlinesBriefing.com

Manticore Search Adds Vector Chunking

Hacker News •
×

Say you are building search over your team's internal documentation — guides, runbooks, postmortems. You have a table with auto embeddings: you insert text, Manticore runs the model and fills the vector column for you. You load a 4,000-word document. The insert succeeds. The search works. Except the model has a 512-token input window, and that document is about 5,000 tokens long. The model read the first 380 words and threw away the other 3,600.

Manticore now handles this in the table definition: add chunk_strategy to the vector column in CREATE TABLE, and Manticore splits each document into chunks, embeds every chunk, and searches all of them. Five strategies: truncate, mean, fixed, recursive, sentence. A document is still one search result. Chunks compete individually, and Manticore returns the document once, with knn_dist() reporting the distance to its closest chunk. k counts documents, not chunks.

Measured on the Manticore manual (189 pages, ~298k words): recall@5 went from 55.1% → 83.3% and MRR from 0.44 → 0.70, at ~2.5× the RAM and ~4× the ingest time. Queries are never chunked. A query is short enough to embed as a whole; only stored documents are split.