Study: Physician-rating websites' validity is suspect
Online physician rating sites are gaining popularity, yet patients who post on online physician rating sites are less likely to comment on perceived low-quality physicians, and more apt to exaggerate their opinions as compared to offline populations, according to a paper presented June 20 at the Institute for Operations Research and the Management Sciences (INFORMS) Healthcare 2011 conference in Montreal.
"Patients need high-quality information about the most consequential service that they consume: healthcare," wrote Ritu Agarwal, editor-in-chief of the Information Systems Research and professor of information systems and director of the Center for Health Information and Decision Systems at the University of Maryland's Robert H. Smith School of Business in College Park, and colleagues. "They want and need to be able to make informed choices."
Agarwal and colleagues investigated potential biases among internet users rating general practitioners on websites such as Angie's List, healthgrades.com, RateMds.com, Vitals.com and Yelp.com. Agarwal and colleagues looked specifically at the way patients determine which physicians to rate and the intensity of opinions they express.
Citing a 2010 Pew Internet and Life Project survey, the authors wrote, "Online physician ratings are gaining popularity among patients. The public's demand for this information is striking: 59 percent of U.S. adults have looked online for health information, and among them, 16 percent have consulted rankings or reviews online of doctors or other providers. Angie's List also found that 76 percent of its users welcome physician rating information."
Researchers asked if patients using the online sites review the full range of doctors (including those viewed positively, neutrally and negatively); if they were biased toward giving negative reviews to doctors; or if they were biased toward giving positive reviews. On the selection of opinions to express, the authors investigated if respondents tended toward hyperbole and largely reported their most extreme experiences with doctors.
The authors found that physicians with low ratings in offline surveys are less likely to be rated online. In addition, although the authors found a strong correlation between the online ratings and offline population opinion, the association is strongest in the lowest quartile of opinions. These results suggest that online ratings are more informative when identifying low-quality physicians, but not as effective in discerning high-quality physicians from those at a medium level.
The authors also concluded that patients are most likely to provide ratings for their most flagrant or negative experiences with physicians.
Data for the study were taken from four datasets: RateMDs.com; an offline patient survey conducted by the consumer advocacy group Consumers' Checkbook; the U.S. Census 2007 Economic Census; and state medical board websites.
"Patients need high-quality information about the most consequential service that they consume: healthcare," wrote Ritu Agarwal, editor-in-chief of the Information Systems Research and professor of information systems and director of the Center for Health Information and Decision Systems at the University of Maryland's Robert H. Smith School of Business in College Park, and colleagues. "They want and need to be able to make informed choices."
Agarwal and colleagues investigated potential biases among internet users rating general practitioners on websites such as Angie's List, healthgrades.com, RateMds.com, Vitals.com and Yelp.com. Agarwal and colleagues looked specifically at the way patients determine which physicians to rate and the intensity of opinions they express.
Citing a 2010 Pew Internet and Life Project survey, the authors wrote, "Online physician ratings are gaining popularity among patients. The public's demand for this information is striking: 59 percent of U.S. adults have looked online for health information, and among them, 16 percent have consulted rankings or reviews online of doctors or other providers. Angie's List also found that 76 percent of its users welcome physician rating information."
Researchers asked if patients using the online sites review the full range of doctors (including those viewed positively, neutrally and negatively); if they were biased toward giving negative reviews to doctors; or if they were biased toward giving positive reviews. On the selection of opinions to express, the authors investigated if respondents tended toward hyperbole and largely reported their most extreme experiences with doctors.
The authors found that physicians with low ratings in offline surveys are less likely to be rated online. In addition, although the authors found a strong correlation between the online ratings and offline population opinion, the association is strongest in the lowest quartile of opinions. These results suggest that online ratings are more informative when identifying low-quality physicians, but not as effective in discerning high-quality physicians from those at a medium level.
The authors also concluded that patients are most likely to provide ratings for their most flagrant or negative experiences with physicians.
Data for the study were taken from four datasets: RateMDs.com; an offline patient survey conducted by the consumer advocacy group Consumers' Checkbook; the U.S. Census 2007 Economic Census; and state medical board websites.