A novel way to take AI from lab to clinic + AI that admits uncertainty + AI brain rot = All in a week’s watchdogging

Consider three AI developments that caught our eye at HealthExec this week. 
 

1. You can teach an old algorithm new tricks. In fact, you darn well should. 

After all, that’s what continuous learning in real-world deployment is all about. This is crucial in clinical settings where AI-equipped medical devices incorporate adaptive AI. But before you can get there, you’ll need to grapple with AI’s proclivity for falling back on the original training it received in preclinical test settings. A new paper breaks down the problem—namely “generalization uncertainty”—and proposes a fix. 

The solution is something the paper’s authors call Digital Similarity Analysis, or DSA. They define the technique as a “voluntary tool that would compare an individual patient’s medical image against the device’s training and testing data before the AI device is used.” If the patient’s image seems an outlier, the physician could “avoid using the device, seek supplemental validation or treat the AI output with lower confidence.”

The paper is the work of researchers at Paragon Health Institute outside Washington, D.C. The authors, led by Paragon research fellow and healthcare AI specialist Kev Coleman, state the present document sticks with AI specific to medical imaging. However, they note, “should the DSA proposal prove successful, the learnings could be extended to analogous AI challenges related to the interpretation of non-image inputs (e.g., EKGs), serial images in video or devices analyzing sound patterns (e.g., AI-enabled stethoscopes).”

Coleman and colleagues acknowledge their DSA proposal cannot completely eliminate generalization uncertainty or algorithmic bias. However, they state, its approach “provides a valuable direction for AI medical device safety and avoids alternatives that inadequately address the problem” while also “extending the discussion of algorithmic bias from broad demographic categories to individual patient characteristics.” 

More: 

‘By shifting evaluation from population groups to individuals, the DSA approach may enhance safety across demographic segments. [Further], an additional benefit of DSA is its integration of image characteristics that arise from differences in radiology equipment and technician technique, a subject too often ignored in the discussion of generalization uncertainty.’

The paper is posted in full for free.

2. Someone has built an AI assistant that admits it doesn’t know everything. Finally. 

Stump it with a hard question or intricate prompt and it just may come back with “I’m not sure.” 

The progress toward a knowingly fallible—and thus more humanlike—digital helper comes from researchers at KAIST, aka the Korea Advanced Institute of Science and Technology in Daejeon, South Korea.

In a peer-reviewed paper describing the work published by Nature Machine Intelligence, Professor Se-Bum Paik, PhD, and graduate student Jeonghwan Cheon suggest they undertook the project on a shared conviction: 

The ability to estimate uncertainty and recognize the unknown is crucial for real-world intelligent systems. “Human intelligence naturally exhibits meta-cognition—the ability to assess probabilistic confidence and distinguish between what is known and unknown,” they write. “Without this meta-cognitive ability to estimate uncertainty, intelligent systems may make critical errors in real-world scenarios in which incorrect decisions can be costly.

The news operation at KAIST applies this thinking to real-world scenarios in medicine, for example. The Paik-Cheon advance, the news team reports, is expected to penetrate “not only fields requiring high reliability—such as autonomous driving, medical AI and generative AI—but also the initialization methods of nearly all deep learning models, making it a key technology for improving overall AI reliability.”

To this Prof. Paik adds:

‘By incorporating key principles of brain development, AI can recognize its own knowledge state in a way that is more similar to humans. This is important because it helps AI understand when it is uncertain or might be mistaken, not just improve how often it gives the right answer.’

KAIST news item here, journal article here.

3. If big things in healthcare AI have you feeling simultaneously wowed and bored, you’re not alone. 

Jared Dashevsky, MD, MEng, can relate. And probably speaks for you when he writes of himself at HLTH.com: “Every day, almost every hour, I get announcements or emails about AI companies launching revolutionary solutions that promise to reduce burnout or boost productivity and patient engagement.” And yet: 

“Every AI announcement sounds the same, and my reaction has become reflexive: That’s great. Next.” From there Dashevsky, a resident physician at Mount Sinai Hospital and founder of the online outlet Healthcare Huddle, diagnoses himself as suffering from AI brain rot—and gives himself some physician solidarity, workplace advice and curative talk therapy.

Three standout passages: 

• There’s a lot of attention on problems that attract investors and headlines—ambient documentation, clinical decision support, predictive analytics. “Less focus on the boring workflow problems that, when compounded, cause the burnout and inefficiencies we actually experience day to day. This includes inundated inbasket messages, redundant data entries, prior auth forms, reviewing faxes … the list goes on. While these aren’t sexy problems, they’re the ones that drain us.”
   
• AI is everywhere now. But somehow, from a physician’s perspective, it feels like nothing has really changed. “Much of the decision-making around onboarding AI tools systemwide is out of our control, but we at least have a voice in what we want. We should focus on tools that make our days feel lighter—hard to define objectively, but we know the feeling.” 
   
• When possible, we should hold vendors to higher standards. “This includes demanding evidence of sustained impact—not just a pilot study with ten hand-picked users. I know a couple of AI companies partnering with health systems to run real studies of their platforms and present results at conferences. That's the bar.”

Read the rest.