What hospitals are top performers? Depends on which ratings system you ask

Patients want to visit a highly rated hospital. Providers want to maintain a high level of quality. Payers are looking to incentivize high-value care. National hospital ratings systems would seem to offer some guidance, but what happens when the same hospital is rated as both a “high” and “low” performer depending on which ratings system you use?

It’s a question that comes to mind after reviewing a comparison of four national ratings systems published in the March issue of Health Affairs, which found little agreement between systems in terms of identifying standout hospitals.

John Matthew Austin, MS, PhD, assistant professor at the Armstrong Institute for Patient Safety and Quality at Johns Hopkins Medicine in Baltimore, and colleagues put the magnifying glass on four hospital ratings systems: U.S. News’ Best Hospitals, HealthGrades’ America’s 100 Best Hospitals, The Leapfrog Group’s Hospital Safety Score and Consumer Reports’ Health Safety Score. All ratings used in the analysis came from between July 2012 and July 2013.

While each ratings system had different terminologies and ratings scales, Austin and colleagues designated hospitals as “high” and “low” performers based on their relative score within each rating system.

Results showed that no hospital was rated as a high performer by all four national rating systems, and only 10 percent of the 844 hospitals rated as high by one system were also rated as a higher performer by any of the other rating systems. A total of three hospitals were rated as a high performer by at least three of the four systems.

Of the hospitals rated as a high performer on multiple ratings systems, Leapfrog and HealthGrades had the most frequent agreement (55 percent), while Consumer Reports and U.S. News had no agreement between their lists of top performers, according to Austin and colleagues.

Likewise, no hospital was rated as a low performer on all three of the rating systems that rated low performance.

In most cases where the ratings systems disagreed on a hospital’s rating, it was a matter of one system classifying a hospital as a top performer while another felt it was middle of the pack. However, the study found 27 cases of extreme cross-rating disagreement in which a hospital was rated as a top performer in one system and a low performer in another.

Austin and colleagues explained that the lack of agreement was primarily because of the methods used by the ratings systems. Leapfrog and Consumer Reports, for instance, focused on hospital safety, while U.S. News focused on difficult patients. HealthGrades looked at general hospital quality over time. Leapfrog’s ratings statistically overrepresented public hospitals among the low-performers. Consumer Reports overrepresented for-profit hospitals and those with larger percentages of Medicaid beneficiaries in its low performer ratings.

“While the lack of agreement among these rating systems is largely explained by their different foci and measures, these differences are likely unclear to most stakeholders,” wrote Austin and colleagues. “The complexity and opacity of the ratings is likely to cause confusion instead of driving patients and purchasers to higher-quality, safer care.”

The authors also pushed for “maximum transparency” from the ratings systems regarding their methodologies and suggested greater use of empirically-driven weighting schemes as opposed to judgment-based methods.

Evan Godt
Evan Godt, Writer

Evan joined TriMed in 2011, writing primarily for Health Imaging. Prior to diving into medical journalism, Evan worked for the Nine Network of Public Media in St. Louis. He also has worked in public relations and education. Evan studied journalism at the University of Missouri, with an emphasis on broadcast media.

Trimed Popup
Trimed Popup