The co-chairs of the Certified Technology Comparison Task Force presented their recommendations for a national health IT comparison tool during the Jan. 20 Health IT Joint Committees meeting.

There is “a need for ongoing comparison tools for providers—not just those making their first purchase but those considering modular needs to meet reporting requirements,” said Anita Somplasky, director of the Physician Quality Measures Development and of both of Pennsylvania’s regional extension centers.

She said the group was well aware of existing tools that are very well respected but “most tools lack an empirical source for comparison for quality reporting, objective usability information, comparative product costs and information about products’ ability to integrate with other health IT.” And, she noted that many such tools have costs that are prohibitive to smaller practices.

Somplasky and her co-chair, Cris Ross, CIO of Mayo Clinic, discussed the task force’s findings including the challenges involved and reasons to pursue an ideal comparison tool.

For example, comparative, objective data may encourage competition and drive innovation. Purchasing health IT is complicated and comparison tools may simplify this process. Tools that provide objective comparison and evaluation information scoped by provider/practice characteristics help providers make the right decision for their particular needs.

Ross discussed several ways the federal government could expand its role in this area, such as data reporting. “Not all information is needed by all consumers. Different consumers have different needs.”

There has been much discussion around usability but it’s a tricky issue, he said. Therefore, the federal role might include things like formal evaluations based on objective data. Ross said he realized that is a loaded sentence. "Clearly, it’s something we would aspire to. There is safety surveillance data that could be made public. We could look at error rates, for example.” The government also could take a role providing information on base costs but full cost is a tricky issue to represent accurately and fairly, he added.

There also is a role for the private sector similar to the peer-to-peer reviews and crowdsourcing of subjective reviews seen in other industries.

Ross went over a list of things the Office of the National Coordinator for Health IT should and should not do regarding a comparison tool:

ONC should:

1. Advance data sources like Certified Health IT Product List (CHPL) as an information resource for private sector tools.
2. Contract with one or more tool vendors to ensure tools are accessible to and meet the needs of specialty and small practice providers.
3. Communicate about comparison tool availability to healthcare providers.
4. Make recommendations for private sector consideration.

“ONC could have a powerful role in continuing to push the need for usability, good information and advocate for informed consumers getting information from highly effective private sources. The office has a role to let people know what’s out there.”

ONC should not:

1. Develop and maintain a comparison tool or expand the CHPL to serve as a comparison tool.
2. Endorse one or more tool vendors.

“CHPL is an important data source but not a resource for private sector tools,” he said. “CHPL in and of itself may be very powerful for some people to go to directly but we wouldn’t suggest it should be a proxy or replacement for the kind of comparison, or rating tools that exist in the private sector.”

Somplasky shared the ideal tool attributes as compiled by the task force, including the following:

• Allow for filters that narrow choices for targeted audiences and are permitted across multiple categories simultaneously.
• Accessible to all levels of technical ability. It should be geared towards small and rural practices and should provide cost transparency.
• Given the modularity of certified health IT, tools should be available that allow for comparison of products for a variety of topics.
• Include both objective and subjective information on product usability. “So many factors are subjective so we really wanted to make sure objective criteria are available.”
• For robust comparison, tools should include information from vendors, independent third parties and peer reviews. Government should make available more objective data on health IT products that can be utilized by comparison tool developers.

Objective data about non-certified health IT also should be available, according to the task force, for comparison as appropriate. Comparison tools should be flexible to help providers select health IT that meets the evolving needs of healthcare delivery system reform.

In his last meeting after serving for 10 years, HIT Standards Committee Co-Chair John Halamka, MD, CIO of Beth Israel Deaconess Medical Center said subjective factors are hard to measure so there is a need for a “Yelp-like function.”

“The CHPL tool ought to be extended as far as possible to the extent that it provides value,” Ross added. It should serve as an information broker where possible.

Members of the committees raised questions about possible restrictions on the intellectual property on health IT. Halamka said that the CIOs in the Boston area have discussed whether their IT contracts include any gag orders prohibiting the sharing of experiences, screenshots and other similar information, and “not a single CIO was aware of such a gag order so it’s not necessarily an impediment to a lot of this sharing.”

Kathleen Blake, MD, MPH, vice president for performance improvement at the American Medical Association, said the clinical registry area “is one that is crying out for comparable data.” She also questioned the validity of online reviews. “We’re finding that people are being paid to write reviews.” She called for the addition in the official recommendations of a comment about having a way to establish that someone is qualified to provide a review based on actual use of a product. “I’m worried that we will be inundated with thousands of reviews that come from God knows where.”

The group voted to forward the recommendations on to the ONC after agreeing to amend the first recommendation to add a feasibility analysis conducted by ONC.