Healthcare AI today: Whom to sue, invite into AI governance and altogether avoid (unless you want a new religion)
News and views you ought to know about:
- The question of legal responsibility for AI gone wrong in healthcare remains unsettled. The only sure thing is a lot of finger pointing. A decisive precedent may not come until regulation takes shape, stakeholders adapt and the technology matures. That’s according to an overview of AI in healthcare with particular interest in the legal and ethical angles. The piece is on offer from two subject-matter experts at the global law firm A&O Shearman. “Accountability will be a particular challenge given the complexity of the value chain and the interplay of different regulatory regimes,” writes attorney Jieni Ji of the firm’s Hong Kong office with David Egan, assistant general counsel for global digital and privacy issues at the pharma giant GSK. “It will be important for all stakeholders to engage in continued dialogue to ensure that legal frameworks keep pace with technological developments and that patient safety remains a central focus.” Other passages worth a neon streak from a digital highlighter:
- “From the perspective of legislation by sovereign states, the legal landscape for AI in healthcare is still in its infancy. Many countries are currently relying on existing technology-neutral laws, such as data protection and equality laws, as well as industry standards, to address AI-related matters. Additionally, some nations are taking proactive steps to develop approaches to address issues arising from AI technologies.”
- “Leading AI developers are also setting up in-house AI ethics policies and processes, including independent ethics board and review committee, to ensure safe and ethical in AI research. These frameworks are crucial while the international landscape of legally binding regulations continues to mature.”
- “[T]eams reviewing AI systems should consist of stakeholders representing a broad range of expertise and disciplines to ensure comprehensive oversight. For example, this may include professionals with backgrounds in healthcare, medical technology, legal and compliance, cybersecurity, ethics and other relevant fields as well as patient interest groups. By bringing together diverse perspectives, the complexities and ethical considerations of AI in healthcare can be better addressed, fostering trust and accountability.”
- “From the perspective of legislation by sovereign states, the legal landscape for AI in healthcare is still in its infancy. Many countries are currently relying on existing technology-neutral laws, such as data protection and equality laws, as well as industry standards, to address AI-related matters. Additionally, some nations are taking proactive steps to develop approaches to address issues arising from AI technologies.”
- The American Medical Association seconds that last notion. For healthcare organizations diving in with AI, “it is indeed crucial to form robust working groups that include people who represent a broad group of departments,” the AMA states in an article posted July 31. “Physicians, nurses and other clinical staff—as well as representatives from the operational, financial, legal, compliance, technology, data science, pharmacy and patient experience branches of the organization—need to be part of the decisions being made when implementing AI.”
- The item quotes AMA chief medical information officer Margaret Lozovatsky, MD. “[E]very clinician and every administrative user of these tools has different needs, so it’s important to involve those stakeholders in the conversation to understand what the needs are and to ensure that technology is actually solving problems rather than creating them,” she says. “I’m a pediatrician. My needs are different than a cardiologist or a dermatologist. That’s why it’s important to have the right voices in the conversation.”
- The item quotes AMA chief medical information officer Margaret Lozovatsky, MD. “[E]very clinician and every administrative user of these tools has different needs, so it’s important to involve those stakeholders in the conversation to understand what the needs are and to ensure that technology is actually solving problems rather than creating them,” she says. “I’m a pediatrician. My needs are different than a cardiologist or a dermatologist. That’s why it’s important to have the right voices in the conversation.”
- Google’s healthcare AI suite recently made a goof during a test that could have been dangerous in the real world. Med-Gemini verbally blended two parts of the central nervous system, commenting on a section of the brain that doesn’t exist. Google people shrugged off the blunder as a mere misspelling. Medical AI skeptics used the episode as reason to distrust AI in clinical settings. “The problem with these typos or other hallucinations is I don’t trust humans to review them,” Maulin Shah, chief medical information officer at 51-hospital Providence health system in Washington State tells The Verge. “These things propagate.”
- The outlet also spoke with radiologist Judy Gichoya, MD, head of Emory University’s Healthcare Al Innovation and Translational Informatics lab. It’s in the very nature of large-scale AI models, she says, to make things up rather than admitting it doesn’t know something. This tendency, Gichoya adds, is “a big, big problem for high-stakes domains like medicine.”
- The outlet also spoke with radiologist Judy Gichoya, MD, head of Emory University’s Healthcare Al Innovation and Translational Informatics lab. It’s in the very nature of large-scale AI models, she says, to make things up rather than admitting it doesn’t know something. This tendency, Gichoya adds, is “a big, big problem for high-stakes domains like medicine.”
- In the eyes of investors, the healthcare innovation economy is complicated. Fundraising dollars are down—yet, overall, private markets are looking better than they have in years. That’s the word from Silicon Valley Bank, which just released a midyear report. “AI may not change the face of clinical care, but it’s coming into its own behind the scenes as a valuable tool for tackling long-standing industry pain points,” the authors write. The report shows U.S. healthcare venture-capital fundraising totaled $3B in the first half of 2025. The figure represents a plummet from 2024 and may be healthcare’s worst year in more than a decade, SVB notes. And yet. “Trailing 12 month healthtech AI deal activity has grown ~2x since 2022 and accounted for nearly a third of all healthcare investment in the first half of 2025,” the report states. “Healthcare-specific AI models, pure software drug-discovery platforms and myriad new administrative tools have furnished healthtech with attractive investment opportunities.” Download an expanded report preview here.
- ‘Religion is text and story and ritual. All of that applies here.’ Applies where? To the Rationalists, a growing Berkeley, Calif., community whose members seem obsessed with AI. The quote is from a Catholic nun and theology professor with the Franciscan order, which has been observing (and praying for) our changing world since the 1200s. Sister Ilia Delio and others, including insiders, were asked for their take on these capital-R Rationalists by The New York Times.
- A Harvard chaplain goes further than the nun in raising a flag over the group. Noting that some of the most powerful minds in Silicon Valley align their thinking on AI with the Rationalists, Greg Epstein—who wrote a book discussing technology as a new religion—asks: “What do cultish and fundamentalist religions often do? They get people to ignore their common sense about problems in the here and now in order to focus their attention on some fantastical future.”
- Rationalist Alex K. Chen says it’s not like that at all. “It’s a place where serendipity can happen,” he tells the Times. “Some people liken it to a college campus or the MIT Media Lab.”
- Article here.
- Article here.
- A Harvard chaplain goes further than the nun in raising a flag over the group. Noting that some of the most powerful minds in Silicon Valley align their thinking on AI with the Rationalists, Greg Epstein—who wrote a book discussing technology as a new religion—asks: “What do cultish and fundamentalist religions often do? They get people to ignore their common sense about problems in the here and now in order to focus their attention on some fantastical future.”
- From AIin.Healthcare’s sibling outlets:
- Cardiovascular Business: AI company raises $55M with help from leading health systems
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.