This article discusses optimizing video input for LLMs by prioritizing frame selection over transcripts to reduce token costs and improve context. Key challenges include token budget constraints, where images consume significant space, necessitating selective frame retention.\n\nScene detection uses three channels: a global channel with dynamic thresholds for motion, an action channel for small subject changes via pixel grids, and a settled channel for static local changes like text. Deduplication ensures unique frames by comparing against recent frames and applying strict tolerances.\n\nFusion with transcripts precomputes frame-transcript alignment, avoiding misalignment in long videos.

Outputs are simplified files (JPEGs, transcript, manifest) for portability. The MCP server enables seamless integration with tools like Claude, resizing frames for efficiency.\n\nThe approach emphasizes model-driven compression, allowing LLMs to observe raw video data rather than summarized text, enhancing accuracy in tasks requiring visual details.