AHRQ provides guidelines for evaluating medical homes
 |
In a brief authored by David Meyers, MD, director of the Center for Primary Care, Prevention and Clinical Partnerships at the AHRQ, and colleagues, the AHRQ provided six “key points” for evaluators to consider when studying PCMHs.
Focus evaluations on quality, cost and experience
The brief recommended that PCMH evaluators consider a model’s ability to deliver quality care, affordable care and pleasurable experiences.
The brief suggested that evaluators accomplish this by measuring patient safety and patient outcomes; total costs and “measures of utilization that drive cost,” such as hospitalizations and emergency department visits; and experiences of not just patients, but also of families, providers and healthcare support staff.
Include comparison practices
“Evaluations with comparisons are more valuable than those without,” Meyers wrote. “Gathering data from comparison practices makes it possible for evaluations to demonstrate that changes in outcomes are the result of intervention.”
For the purpose of comparison, the AHRQ ranked randomized controlled studies as “excellent,” matched comparison studies as “very good” and pre-post evaluation as “poor.”
Recognize that the PCMH is a practice-level intervention and account for clustering
The brief reminded readers that, in the evaluation of PCMHs, the practice is the unit of intervention and not the patient, and that evaluations have to account for “clustering,” which “occurs when outcomes for patients within a practice (that is, a cluster) are more similar to each other than to outcomes for patients in other practices, because of systematic differences between the practices.”
The brief warned that ignoring clustering can easily lead to inaccurate conclusions that a particular intervention works and that skilled statisticians should be hired to account for clustering.
Include as many PCMH practices as possible
The brief declared that the number of practices included in a study, and not the number of patients, strongly influence the statistical power of the study.
According to the brief, evaluations of fewer than 20 intervention practices “typically will lack the statistical power to be able to detect plausible effect sizes for many key outcome measures,” that evaluations of 20 to 100 intervention practices “may have the statistical power to detect plausible effects on cost and service use outcomes among patients with multiple chronic conditions” and that evaluations of more than 100 intervention practices “are likely to be able to demonstrate effects on cost and service use outcomes across all patients.”
Be strategic in identifying the right samples of patients to answer each evaluation question
While the brief recommended that evaluators measure costs across the entire intervention, it also suggested that evaluators examine the data associated with subsections of patients because “the ability to detect changes in cost and utilization outcomes among people with chronic diseases is much greater than it is among the general patient population.”
Rethink the number of patients from whom data are collected to answer key evaluation questions
The brief suggested that evaluators limit sample sizes of patients in studies of practice-level interventions because larger sample sizes cost more and don’t necessarily increase the validity of a study.
“Depending on the degree of clustering, a study should be able to detect plausible effects with only 20 to 100 patients per practice,” Meyers wrote. “In practice-level interventions, gathering data from larger numbers of patients per practice only slightly improves the minimum effect that can be detected.”
The AHRQ brief is available for download in its entirety here.