3 accomplished AI innovators worry anew over their mind-of-its-own creation
Recent days have seen some nerve-rattling brain dumps about the near future—including five figurative minutes from now—authored by three tech luminaries who specialize in AI innovation.
None of these alerts—or warnings—concentrates on AI in healthcare per se. All of them are generalizable to AI worldwide and across all industries.
Read the representative excerpts below and decide for yourself whether these AI thought leaders are indulging in the human appetite for hype and, with it, the impulse to draw clicks and Likes—or if they’re honestly describing the gathering AI storm as best they can (or, in one case, as poetically as possible given the circumstances).
Dario Amodei, CEO and co-founder of Anthropic
Humanity needs to wake up, and this essay is an attempt—a possibly futile one, but it’s worth trying—to jolt people awake. “To be clear, I believe if we act decisively and carefully, the risks can be overcome—I would even say our odds are good,” Amodei offers. “And there’s a hugely better world on the other side of it. But we need to understand that this is a serious civilizational challenge.”
More:
The level of AI that raises civilizational concerns for me is the powerful AI that I described in my 2024 essay ‘Machines of Loving Grace.’ [P]owerful AI could be as little as one to two years away, although it could also be considerably further out. Exactly when powerful AI will arrive is a complex topic that deserves an essay of its own, but … I think there’s a strong chance it could be very soon.
Today people worry a lot about, for example, the potential influence of TikTok as Chinese Communist Party propaganda directed at children. “I worry about that too,” Amodei admits, “but a personalized AI agent that gets to know you over years and uses its knowledge of you to shape all of your opinions would be dramatically more powerful than this.”
More:
AI models are known to display different personalities or behaviors under different circumstances. … AI models could develop personalities during training that are—or if they occurred in humans would be described as—psychotic, paranoid, violent or unstable. [These ‘sick’ models could] act out, which for very powerful or capable systems could involve exterminating humanity.
Many people report that it is easier [and less costly] to talk to AI about their personal problems than to talk to a therapist. “When my sister was struggling with medical problems during a pregnancy, she felt she wasn’t getting the answers or support she needed from her care providers,” Amodei recalls. “[S]he found [Anthropic’s] Claude to have a better bedside manner (as well as succeeding better at diagnosing the problem).”
More:
I’m sure there are some tasks for which a human touch really is important, but I’m not sure how many—and here we’re talking about finding work for nearly everyone in the labor market.
Matt Shumer, co-founder and CEO of Otherside AI
I know job displacement is real because it has already happened to me. “Here’s the thing nobody outside of tech quite understands yet: We’re not making predictions,” Shumer writes. “We’re telling you what already occurred in our own jobs and warning you that you’re next.”
More:
For years, AI had been improving steadily. Then in 2025, new techniques for building these models unlocked a much faster pace of progress. This year, something clicked. Not like a light switch … more like the moment you realize the water has been rising around you and is now at your chest.
The models available today are unrecognizable from what existed even six months ago. “The debate about whether AI is really getting better or hitting a wall—which has been going on for more than a year—is over,” Shumer states. “It’s done.”
More:
[T]he gap between public perception and current reality is now enormous, and that gap is dangerous … because it’s preventing people from preparing.
These new AI models aren’t incremental improvements. This is a different thing entirely. “The experience that tech workers have had over the past year—of watching AI go from ‘helpful tool’ to ‘does my job better than I do’—is the experience everyone else is about to have,” Shumer predicts. “Law, finance, medicine, accounting, consulting, writing, design, analysis, customer service. Not in 10 years. The people building these systems say one to five years. Some say less.”
More:
The market was spooked enough this month that it wiped out $1 trillion worth of software value in just a week. And given what I’ve seen in just the last couple of months, I see more disruption—and soon.
Read the whole thing. (If you don’t want a $1 trial subscription to Fortune, you can read the piece for free in a choppier visual format here. HealthExec won’t tell on you.)
Mrinank Sharma, just-resigned head of the Safeguards Research Team at Anthropic
Dear Colleagues, I’ve decided to leave Anthropic. My last day [was] February 9. “I’ve achieved what I wanted to here. I arrived in San Francisco two years ago, having wrapped up my PhD and wanting to contribute to AI safety. I feel lucky to have been able to [help with] understanding AI sycophancy and its causes; developing defenses to reduce risks from AI-assisted bioterrorism; actually putting those defenses into production; and writing one of the first AI safety cases.”
More:
I’m especially proud of my recent efforts to help us live our values via internal transparency mechanisms; and also my final project on understanding how AI assistants could make us less human or distort our humanity.
I continuously find myself reckoning with our situation. The world is in peril. “And not just from AI or bioweapons but [also] from a whole series of interconnected crises unfolding in this very moment.”
More:
I feel called to writing that addresses and engages fully with the place we find ourselves and that places poetic truth alongside scientific truth as equally valid ways of knowing. I believe both have something essential to contribute when developing new technology.
Full letter here.
Also of interest to high-level AI watchers:
- Many healthcare leaders are leaning into agentic AI as adoption hurdles ease (Deloitte)
- Top 10 AI platforms being used in the healthcare sector aiming to deliver high-quality care and improve patient outcomes (Healthcare Digital)
- Validating AI in healthcare: the role of real-world evidence (KevinMD)
- Life sciences and healthcare need to better integrate AI (Fast Company)
From HealthExec’s sibling news outlets:
- Radiology has become a ‘case study for why AI won’t replace human workers’ (Radiology Business)
- Medtronic to acquire AI-enabled FFR specialists CathWorks for up to $585M (Cardiovascular Business)
