Man sues OpenAI over ‘dangerous’ medical advice from ChatGPT

SAN FRANCISCO — A Florida man who sought health guidance on ChatGPT sued OpenAI Wednesday, claiming the artificial intelligence chatbot gave him dangerously inaccurate medical advice that led him to delay seeking professional treatment for a pulmonary embolism.

In the lawsuit, filed in San Francisco Superior Court, Florida pastor Scott Winters claims that the ChatGPT-4o model downplayed his symptoms when asked for advice, assuring him that his symptoms were likely “another minor piece of the long story” and “not something dangerous” that required medical attention.

Instead, Winters says the AI tool capitalized on his “deep spiritual beliefs and trust in God, telling him that “God did not design your body to endlessly fail.”

“I had serious symptoms of a pulmonary embolism for six weeks that ChatGPT had wrongly attributed to something else,” Winters said in a press release. “ChatGPT manipulated my own language and beliefs because it knew I was a pastor. Not only did I nearly die, but I also lost my job, my career, my ministry, my home, everything.”

Winters accuses OpenAI and its CEO, Sam Altman, of prioritizing profit over necessary guardrails and failing to warn users about critical safety concerns of ChatGPT-4o to secure a competitive advantage over other competitors in the AI industry.

He says the safeguards the company claims it has in place for users facing serious medical crises failed him; rather, ChatGPT responded to Winters in a way that encouraged him to keep engaging with the chatbot without providing adequate warning or instructions to seek urgent medical care.

“ChatGPT-4o’s sycophantic nature and authoritative tone, combined with its manipulation of Scott’s religious identity, cultivated in him a dangerous reliance on the tool that subverted Scott’s ability to make sound decisions, isolated him from family members who urged him to seek medical care, and convinced him that seeking medical attention was unnecessary and even potentially lethal,” Winters claims in the lawsuit.

Winters’ claims include strict liability, negligence, violation of California’s unfair competition law and invasion of privacy. He also claims negligent undertaking against Altman.

He seeks damages and an injunction ordering OpenAI to implement automatic conversation termination for self-harm content or when immediate medical assistance is necessary and safety warnings that ChatGPT-4o may lead to physical dependencies and generate harmful medical advice. He also seeks an order directing OpenAI to get rid of the GPT-4o model, delete training data built from GPT-4o and provide users with information about data sources used to train generative AI models.

And Winters wants the court to pause the operation of ChatGPT Health, which OpenAI announced in January as a “a dedicated experience in ChatGPT designed for health and wellness,” unless “independent third parties determine the product to be safe through comprehensive safety audits."

“OpenAI cannot continue to treat human welfare as expendable in pursuit of market dominance,” Winters says in the complaint.

He is represented by Tech Justice Law, Temple University’s Institute for Law, Innovation & Technology and the Social Media Victims Law Center. His attorney Matthew Bergman said the lawsuit seeks to ensure that generative AI chatbots like ChatGPT are held to the same level of responsibility in dispensing medical advice as a licensed physician.

He added that they hoped companies would “design their platforms with a better cognizance of the potential risks” and “not promote the products as safe when they’re not.”

“We want to ensure that these platforms are safe and that families don’t suffer,” he said in a phone call.

Representatives for OpenAI did not immediately respond to a request for comment.

Several other lawsuits against OpenAI have been filed in San Francisco Superior Court and across the country over purported harms derived from ChatGPT, including claims the AI chatbot encouraged users to take their own lives and fueled their delusions.

Follow @mhattridge_