Google AI has developed SymptomAI, a conversational AI agent designed for everyday symptom assessment. Current diagnostic interviews, while effective, face accessibility barriers. Existing language models show promise but are often evaluated on curated data, not real-world patient interactions.

To bridge this gap, a large-scale study involving 13,917 participants tested five experimental SymptomAI agents powered by Gemini Flash 2.0. Participants interacted with an AI agent to describe symptoms, receive a differential diagnosis (DDx), and recommendations. Post-interaction surveys and clinical expert annotations were used to evaluate SymptomAI's performance.

Key findings indicate that clinical experts preferred SymptomAI's DDx in over 50% of cases and found it more accurate than other clinicians' assessments. Agent-driven prompting strategies significantly improved diagnostic accuracy compared to user-driven queries. SymptomAI's performance was particularly strong in cases where clinicians felt less confident. Furthermore, SymptomAI diagnoses, especially for infectious diseases, correlated with physiological trends from Fitbit wearable devices, suggesting potential for large-scale physiological data analysis.