Mind games: Real physicians more accurate than symptom checker apps
Patients relying on symptom checker apps and websites aren’t getting as accurate of a diagnosis as they would by visiting a real physician, according to a study published in JAMA Internal Medicine.
The study, led by Harvard Medical School student Hannah L. Semigran with researchers from Brigham and Women’s Hospital and the Human Diagnosis Project, asked 234 physicians to diagnose 45 hypothetical patients based on clinical vignettes that included medical history but no physical exam, lab test or blood test results.
Some 90 percent of the physicians participating were trained in internal medicine, while 52 percent were fellows or residents. Their diagnoses were compared to 770 vignettes evaluated by 23 symptom checker apps, including tools from major medical organizations like the American Academy of Physicians and the American Academy of Pediatrics.
The apps weren't in the same ballpark as the real-life physicians' opinions. The doctors listed the correct diagnosis first 72 percent of the time. The symptom checkers only did so only 34 percent of the time. When listing the top three possible diagnoses, physicians had the correct diagnosis in their list 84 percent of the time, compared to 51 percent accuracy from the apps and websites.
While the flesh-and-blood doctors won what researchers called the “first direct comparison of diagnostic accuracy,” the study noted there is room for improvement.
“Despite physicians’ superior performance, they provided the incorrect diagnosis in about 15 percent of cases, similar to prior estimates (10 to 15 percent) for physician diagnostic error,” Semigran and her coauthors wrote. “While in this project we compared diagnostic performance, future work should test whether computer algorithms can augment physician diagnostic accuracy.”
The study also noted the clinical vignettes used aren’t likely to match the complexity of real-life patients. It’s also possible online tools other than the 23 which were used could be more accurate.
John joined TriMed in 2016, focusing on healthcare policy and regulation. After graduating from Columbia College Chicago, he worked at FM News Chicago and Rivet News Radio, and worked on the state government and politics beat for the Illinois Radio Network. Outside of work, you may find him adding to his never-ending graphic novel collection.