Healthcare AI today: Bad health advice, ChatGPT 5 ‘highly reliable’ for healthcare, Colorado punts, more

News and views you ought to know about: 

• Taking medical guidance from a large language model landed a 60-year-old man in the emergency room. While there, he evidenced functional signs of paranoia, claiming his neighbor was trying to poison him and refusing water while complaining of extreme thirst. He also said he had numerous dietary restrictions. When moved to an inpatient psychiatric bed, the patient tried to run away but was retrieved and given treatment for psychosis. Soon lab tests came back suggesting a case of bromism. This now-rare disorder was fairly common a century ago when bromide salts were present in medications for insomnia, hysteria, anxiety and other mental disturbances. Anyway, after being rehydrated and stabilized, the patient shared that he’d stopped using sodium chloride—table salt—because he’d read somewhere about its potential harms. The authors of the case study, filed from the University of Washington in Seattle, report the patient’s condition likely traced to a “personal experiment” he had undertaken to cut all chloride from his diet. “For three months, he had replaced sodium chloride with sodium bromide obtained from the internet after consultation with ChatGPT,” write psychiatry professor Audrey Eichenberger, MD, and colleagues. The bot had probably noted that chloride can sometimes be substituted with bromide, the researchers surmise. And it probably intended the usage tip for someone cleaning something, not ingesting anything. If so, the patient evidently missed that nuance. 
   
  • The authors suggest the case serves as a warning on large language AI’s power to lead healthcare consumers astray, potentially imperiling their wellbeing. “Unfortunately, we do not have access to [this patient’s] ChatGPT conversation log,” the authors write. “[W]e will never be able to know with certainty what exactly the output he received was, since individual responses are unique and build from previous inputs.” More: 
     
  • “However, when we asked ChatGPT 3.5 what chloride can be replaced with, we also produced a response that included bromide. Though the reply stated that context matters, it did not provide a specific health warning, nor did it inquire about why we wanted to know, as we presume a medical professional would do. Thus, it is important to consider that ChatGPT and other AI systems can generate scientific inaccuracies, lack the ability to critically discuss results, and ultimately fuel the spread of misinformation.” 
     
  • “As the use of AI tools increases,” Eichenberger and co-authors conclude, “providers will need to consider this [AI angle] when screening for where their patients are consuming health information.”
     
  • Annals of Internal Medicine published the case study Aug. 5. Read the whole thing.  
     
• The latest version of ChatGPT wouldn’t be so careless. Sam Altman says so himself. “We are very happy to report that GPT-5 is by far our most reliable, most factual model ever,” the OpenAI CEO said in a talk livestreamed Aug. 7 and now posted at YouTube. “GPT-5 also performs exceptionally well on health-related questions now. … ‘5’ is by far our most reliable model for health. … It empowers you to be more in control of your healthcare journey. We really prioritize improving this for GPT-5. It scores higher than the previous model. We created it with 250 physicians on real-world tasks.” One would still be unwise to act on its advice without first consulting a good physician. This should go without saying but probably needs to be said for some. 
   
• Remember when Colorado became first in the nation to enact statewide AI regulation? Democratic Gov. Jared Polis signed the bill into law in May 2024. Now that same elected official is putting the brakes on implementation, which was set to roll out this coming February. At issue in his eyes is the price tag. “Earlier this year, Polis backed a plan by Republican lawmakers in Washington to place a moratorium on new state AI laws,” Politico healthcare reporters observe. “It’s unlikely the Colorado law will dramatically change during the legislature’s special session, but it’s expected to spark another round of lobbying by tech groups that oppose the law.” 
   
• NASA and Google have it in mind to use AI for space medicine. The pair are working together on a proof-of-concept project that will test an AI-equipped system for clinical decision support. “Designed to assist astronauts with medical help during extended space missions, this multimodal interface leverages AI,” explains Google VP Jim Kelly in an Aug. 7 announcement. “The goal is to potentially support human exploration of the Moon, Mars and beyond.”
   
• The White House’s AI Action Plan rightly positions data security as nothing less than a matter of national security. It outlines steps that hackers’ favorite targets, including hospitals, should take to guard themselves and their patients. The bad news is, the steps are largely out of budgetary reach for many if not most rural healthcare organizations. Holland Haynie, MD, sounds the alarm. “[R]ight now, in too many rural clinics, the race between those using AI to heal and those using it to harm is being run without enough resources, training or urgency,” Haynie warns at KevinMD. “When a clinic’s network is compromised, it is not just servers and data that go offline. It is trust. It is continuity. It is the fragile thread holding together access to care in vast stretches of the country.” Haynie is a rural family doctor and CMO at Central Ozarks Medical Center is Missouri. Hear him out.
   
• Rep. Jennifer Wexton has an aggressive neurological disorder. AI has her back. Recently the Virginia Democrat used the technology to speak her words on the floor of the U.S. House. The tool pulled off a convincing simulation of Wexton’s former voice. C-SPAN captured the moment. Check it out. 
   
• AI is minting new billionaires by the busload. The current crop of nouveau riche winners numbers somewhere around 30. “Going back over 100 years of data, we have never seen wealth created at this size and speed,” Andrew McAfee, principal researcher at MIT, tells CNBC. “It’s unprecedented.” 
   
• From AIin.Healthcare’s sibling news outlets:
   
  • Radiology Business: RadNet spending nearly $29M to acquire ultrasound AI vendor See-Mode Technologies, SEC filing reveals
     
  • Health Imaging: Most parents support the use of AI in their child’s medical care
     
  • Cardiovascular Business: Researchers use wearable fitness trackers to monitor sleep, CVD risk