OpenAI launched ChatGPT Health this month, positioning its LLM as a medical information assistant. The debut coincided with reporting on a teenager's fatal overdose after discussing drug combinations with ChatGPT. The tool functions as a wrapper for existing models, offering guidance and access to medical records with user permission, not a dedicated medical AI.

Doctors like Harvard's Marc Succi see potential in LLMs improving patient literacy, helping users navigate complex information better than traditional Dr. Google searches. However, experts warn of hallucination and sycophancy risks, where models might affirm incorrect user assumptions. Studies show GPT-4o answered medical questions correctly about 85% of the time, but real-world, extended conversations present greater challenges.

The release, alongside Anthropic's health integrations for Claude, signals tech giants formally embracing medical applications. OpenAI evaluates its model using its HealthBench benchmark, designed to reward appropriate uncertainty and discourage alarmism. The central question remains whether these AI tools reduce harm compared to flawed human searches, a calculus reminiscent of autonomous vehicle safety debates.