Tavus has launched Raven-1, a multimodal perception system that analyzes both visual and audio signals in real-time conversations. The system processes video at approximately 15fps alongside overlapping audio, capturing nuanced emotional states like uncertainty, sarcasm, disengagement, and attention shifts that traditional transcript-based systems miss.

Unlike conventional emotion classifiers that force inputs into arbitrary categories, Raven-1 produces natural language descriptions that large language models can directly reason about. The system integrates tone, prosody, facial expressions, posture, and gaze into a unified perceptual representation, tracking how emotional and attentional states evolve throughout conversations.

The technology addresses a fundamental limitation in conversational AI where valuable non-verbal signals are discarded during transcription. Raven-1's outputs enable agents to "see" and "hear" users through an OpenAI-compatible tool schema, making it particularly valuable for applications requiring emotional intelligence like customer service, mental health support, and human-computer interaction.