Netflix editors once lost days manually scrubbing through thousands of hours of raw footage to find specific character moments or scenes. A single season generates over 2,000 hours of material—216 million frames that needed searching. The creative bottleneck became severe enough to stall workflows entirely.

Rather than relying on one general AI model, Netflix deployed an ensemble of specialized models, each excelling at specific tasks like face recognition, scene classification, dialogue transcription, and object detection. This approach outperforms unified models because specialists consistently deliver higher accuracy for their particular domain. The system processes billions of data points from these diverse model outputs.

The architecture employs temporal bucketing, slicing video into one-second intervals where multiple model outputs intersect and fuse. Raw annotations feed into Apache Cassandra, then an asynchronous fusion layer merges different data types—text labels, vector embeddings, and timestamps—into unified records. These enriched buckets flow to Elasticsearch for sub-second query performance.

Netflix chose one-second resolution after balancing precision against scale: 2,000 hours produces 7.2 million buckets. The company also explores MediaFM, a unified foundation model, though production currently relies on the specialized ensemble approach.