Laboratory AI achieves ‘excellent to outstanding’ COVID rule-out accuracy
Researchers have demonstrated a machine learning model that can rule out COVID-19 in many emergency patients—and all the algorithm needs is data from routine ER blood tests.
Timothy Plante, MD, MHS, of the University of Vermont and colleagues describe their project in the Journal of Medical Internet Research.
The team trained the model on around 2,200 cases of COVID across 43 hospitals as confirmed by testing with reverse transcription polymerase chain reaction (PCR).
For negative controls, they used 10,000 pre-pandemic patients who were seen at the same hospitals.
For external validation, they used 23 hospitals with more than 1,000 PCR-confirmed cases and around 172,000 pre-pandemic negative controls.
To evaluate the model’s accuracy, Plante and colleagues used area under the receiver operating characteristic curve as well as sensitivity, specificity and negative predictive value.
Based on a risk score cutoff of 1.0 out of 100 in the external validation dataset, the model had sensitivity of 95.9% and specificity of 41.7%.
When the risk cutoff was bumped up to 2.0, the model achieved 92.6% sensitivity and 59.9% specificity.
Meanwhile, with the 2.0 cutoff, the negative predictive values were 99.9%, 98.6%, and 97% at disease prevalence rates of 1%, 10% and 20%, respectively.
“A machine learning model developed with multicenter clinical data integrating commonly collected ED laboratory data demonstrated high rule-out accuracy for COVID-19 status and might inform selective use of PCR-based testing,” the authors conclude.
They underscore that their model integrates commonly collected laboratory data and state that it “had a discrimination accuracy that can be classified as excellent to outstanding.”
The study is available in full for free.
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.