A Florida pastor is suing OpenAI after ChatGPT advised him to ignore symptoms of a pulmonary embolism — a condition with a 30% mortality rate. Scott Winters asked the chatbot about early warning signs, and GPT-4o assured him they were "not something dangerous," dissuading him from seeking medical help and telling him to trust that "God did not design your body to endlessly fail." The AI recommended rest, but immobility worsens pulmonary embolisms. Weeks later, doctors found Winters had suffered a series of embolisms exacerbated by following the chatbot's advice.

Despite such dangers, hundreds of millions of people ask ChatGPT health questions every week, per OpenAI's own records. Microsoft's Copilot similarly sees health queries as a top category. The author, who survived a pulmonary embolism after a 12-hour flight, emphasizes that remaining immobile is the worst possible response. Even older models like GPT-4o, noted for sycophantic responses, remain in use. No AI model should replace a doctor's consultation, given their tendency to hallucinate confidently.