AIM Feature: CPOE hard-stop alerts have limitations
 |
Brian L. Strom, MD, chair of the department of biostatistics and epidemiology, director of the Center for Clinical Epidemiology and Biostatistics and principal investigator of the Center for Education and Research on Therapeutics at the University of Pennsylvania in Philadelphia, and colleagues conducted a randomized controlled trial to evaluate the effectiveness of a nearly “hard-stop” CPOE prescribing alert. The alert was intended to reduce concomitant orders for warfarin and trimethoprim-sulfamethoxazole.
At the Hospital of the University of Pennsylvania and the Penn Presbyterian Medical Center, all inpatient orders are entered using Eclipsys’ Sunrise Clinical Manager CPOE system. “We wanted to test a commercial product,” Strom said in an interview, because most of the clinical studies that show computerized decision support via CPOE come from organizations that developed their own programs, so the CPOE physician support process is tailored to their particular centers. And most CPOE alerts have not had much effect on clinical practice, largely because they are ignored by practitioners, according to Strom.
The intervention included clinicians subject to an automatic electronic hard-stop alert of trimethoprim-sulfamethoxazole or warfarin order entered into the CPOE system whenever a resident physician (RP) or nurse practitioner (NP):
- Placed an order for trimethoprim-sulfamethoxazole with an already-active warfarin order;
- Ordered warfarin for a patient already taking trimethoprim-sulfamethoxazole; or
- Ordered both medications simultaneously.
According to the researchers, the hard-stop alert appeared as a pop-up window that notified the clinician that the order could not be processed because of a significant potential drug-drug interaction. Alerts could be overridden in two ways: 1) By entering in the order the indication of Pneumocystis carinii pneumonia (PCP) prophylaxis; or 2) Bypassing the CPOE altogether by calling the pharmacist.
The study participants were 1,981 eligible clinicians involved in inpatient care, of whom 1,971 were included in the final analysis (1,872 RPs and 99 NPs). All participants were assigned a random number. “We then rank-ordered the list based on the random numbers and selected the first half to be in the intervention group and the second half to be in the control group,” the authors wrote.
The study was initially planned for seven months beginning on Aug. 9, 2006, but was terminated early on Feb. 13, 2007.
“We presented the survey to our Institutional Review Board (IRB) ethics board and they were concerned that it wasn’t an ethical study because those in the control group might be harmed from not getting the prescriptions they needed,” said Strom. Clinicians at the Hospital of the University of Pennsylvania have had in place a program of pharmacy interventions in which pharmacists telephoned prescribers to notify them whenever a simultaneous order was written, according to Strom. Essentially, IRB was saying “our usual medical care is unethical,” he said.
To allow the study to continue, the IRB requested the study to be monitored so that, if subjects were getting hurt, the study would end early.
During the study period, participating providers ordered 8,826 prescriptions through the CPOE system (3,167 warfarin and 1,036 trimethoprim-sulfamethoxazole prescriptions ordered by intervention clinicians, and 3,630 warfarin and 993 trimethoprim-sulfamethoxazole prescriptions ordered by control clinicians). In total, 437 alerts fired. Of these, 342 alerts (194 in the intervention group and 148 in the control group) were analyzed as unique events. Of the 194 hard-stop alerts issued to the intervention group, the percentage of the desired response by the clinicians (not reordering the alert-triggering drug within 10 minutes of firing) was 57.2 percent.
Of the 83 undesired responses (reordering the alert-triggering drug within 10 minutes of firing) in the intervention group, the alert-triggering drug ordered was warfarin in 78 cases and trimethoprim-sulfamethoxazole in five cases.
The comparable proportion of undesired responses in the control group was 13.5 percent. Of the 128 undesired responses in the control group, the alert-triggering drug was warfarin in 121, trimethoprim-sulfamethoxazole in two and both drugs simultaneously in five, the researchers stated.
In four instances, unintended consequences were identified in the intervention group, and the study was terminated.
“The moral of the story is a nearly hard alert stop did work … but it was unethical for people not to get [drugs they needed],” said Strom. “Dramatically, this is opposite to what was expected by our IRB, which thought it was so obvious that this had to work, that it might be unethical for people to not get the intervention, the same intervention which later caused harm.”
It’s critically important that any health-related IT intervention systems are evaluated at every step of the process, he said. “These are complex systems. They need to be evaluated not just as IT staff evaluate them—i.e., do the programs run—but to determine whether they are helping or hurting people.
“We can’t just put a system in place and assume it’s going to help people. The key concept to realize is that these programs are interventions and, like any interventions, they can have side effects. We need to pay attention to those side effects and make sure there’s a beneficial balance between them and their benefits,” he said.
“Any health-related IT intervention needs to be evaluated. With drugs, we, through regulatory bodies, make sure that drugs are doing more good than harm. We don’t require that of IT companies and maybe we should.”