HeadlinesBriefing favicon HeadlinesBriefing.com

Gemini Embedding 2 Enables Direct Video Search Without Text

Hacker News •
×

A new CLI tool called SentrySearch uses Google's Gemini Embedding 2 to perform semantic search directly on video footage without requiring transcription or frame-by-frame analysis. The tool projects raw video into the same 768-dimensional vector space as text queries, allowing natural language searches like 'red truck running a stop sign' to find matching clips in seconds.

Built on ChromaDB, SentrySearch splits videos into overlapping chunks, embeds each chunk directly using Gemini's video-native model, and stores the vectors for fast similarity matching. The system automatically trims and saves matching segments, making it practical for reviewing hours of dashcam or security footage. Processing costs run about $2.50 per hour of footage, with optimizations like still-frame detection and preprocessing helping reduce expenses.

This approach eliminates the need for intermediate text processing, enabling sub-second semantic search over large video archives. While still-frame detection uses heuristic JPEG comparison and chunk boundaries may occasionally split events, the tool works with any MP4 footage and requires only a Gemini API key and ffmpeg. The technology demonstrates how video-native embeddings could transform how we search and analyze visual content.