Factoring workflow into big data tools

CHICAGO—Media reports indicate big data is going to cure everything but “we have major care institutions that have employed very high-end EMRs and cannot extract their data,’ said Gabriel Escobar, MD, speaking at the 2014 Healthcare Leadership Forum. Escobar is a research scientist at the Kaiser Permanente Northern California Division of Research; director of the Division of Research Systems Research Initiative; and regional director for Hospital Operations Research for Kaiser Permanente Northern California.

"What concerns me the most is that there are literally hundreds of predictive models published each year but show me a study where the predictive model actually changed outcomes or physician behavior. That is a much smaller body of literature."

The disconnect, he said, is due to unstable platforms; EHRs designed for billing, not clinical care; extraordinary data density; and workflow issues.

He cited a National Institutes of Health-funded study he conducted that looked at the incidence of sepsis in newborns. Even though the incidence has fallen—the rate is .3 per 1,000—the number of babies worked up and put in NICUs may have increased. “In some places, 20 percent of babies get an evaluation for sepsis and stay in the hospital for 2 to 4 days. Almost all of them will have sepsis ruled out.”

He and his team developed two components for predictive modeling. One performs prior probability—the risk of sepsis the instant the baby is born. The other is likelihood ratios for how the baby looked. He cited an example of a women whose water broke nine hours ago, had no fever and had been given no antibiotics. In the example, the baby’s risk is zero and the provider need not do anything. “That’s a big deal because, in the past, they might have put the baby in the NICU.”

When he changed the example to a mother with a fever and membranes that ruptured 28 hours ago, “now we can see the baby’s risk has soared.” Even so, if the baby looks good, the clinician only needs to check the baby a little more carefully. Had the mother been treated with antibiotics, the risk drops dramatically. The use of predictive models could incentivize obstetricians to better manage the mother's health.

The calculator was implemented into the EMR system but it was hard to get people to use it. There was a massive effort to get the word out and “a lot of negotiations on the screens and getting people to buy into the project.” It’s worth the effort, Escobar said, because full deployment of the NICU calculator resulted in an estimated 5 percent decrease in the NICU census. “That’s a lot. If we applied it to all of the U.S., we would reduce the sepsis rule out rate by 80,000 to 240,000 babies a year.”

A good workflow is essential to making these models work in practice. Escobar said workflow refers to the integration of data delivery in the EMR, particularly the issue of the display of the information. “If the information is buried, it won’t do anybody any good.” Also, in some facilities, the “entire practice of medicine centers around that EMR. It’s really critical that you put energy into defining workflow.”

Escobar worked on another study focused on early detection of impending deterioration of ward patients. While much attention goes to patients in the ICU, “we found that the most important subset of hospital patients were those who deteriorated once in the hospital” and account for a significant portion of deaths. Kaiser Permanente hospitals found that crashing while in the hospital led to 12 extra days in the hospital with half being ICU days, if they survived.

Escobar and his team developed severity of illness tools using about 400,000 Kaiser hospitalizations. Designed as a continuous variable, he said it can replicated by anyone with a high-end EMR. Every month every Kaiser member has their record scanned for the preceding 12 months and grouped into hierarchical categories. Then, clinicians can go into the second model to get the point score. “The higher your score, the higher your comorbidity burden in the previous year.”

The predictive model for early detection was based on 652,000 hospitalizations of whom 20,000 crashed. A current pilot involves two hospitals using electronic clinical decision support through their EMR interface. Clinicians get a 12-hour warning that a patient might crash. In November, Escobar said new functionality will be deployed so that when a user assigns an order to consult, the system will fire a severity of illness score in real time.

The pilot warning system is accompanied by multiple workflows including automated checklists and protocols for involving palliative care. It’s also disease-agnostic. “What does a ‘presponse’ look like?” Escobar posed. “We had to spend time with our physicians to develop scripts.” While patients hate the idea of computers telling doctors what to, he said, they love the potential of technology. If the system issues a warning, the rapid response team is triggered and clinicians can do preemptive rounding and make sure a document care directive is in place.

An audience member asked about the business case for use of these types of tools. Escobar said he is going to make the equations used in these projects publicly available. An upcoming issue of the Journal of Hospital Medicine will include a complete toolkit including all of the code.

The business care is complicated, though, he acknowledged. “You can’t use traditional metrics because they are contaminated by the fact that a lot of hospital patients won’t make it no matter what you do.” Part of the project will involve comparing end-of-life care in sites with and without the alerts. The baseline mortality already is extremely low but “if we can make the end of life more human and more respectful of human dignity, I think it’s worth it.”