Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Medicare’s Hospital Compare Performance Measures and Mortality Rates—Reply

Medicare’s Hospital Compare Performance Measures and Mortality Rates—Reply Letters Section Editor: Robert M. Golub, MD, Senior Editor. In Reply: Dr Fierer raises an important point: 3 of the performance measures used by the Hospital Quality Alliance that we studied are not based on evidence from randomized controlled trials. It may, however, be reasonable to implement some performance measures based on observational data because some interventions are not feasible to study in a randomized controlled trial. Limiting measures to those with randomized controlled trial evidence would limit the clinical areas in which performance measurement is possible, increasing the difficulty of providing meaningful information about quality through performance measurement. Indeed, numerous established performance measures are based on results from observational studies and expert consensus.1 However, when the potential benefit from a measured intervention is uncertain or small, there is increased risk that the inaccuracy of performance measurement will outweigh the benefits. We agree that there must be vigilance in balancing the potential unintended consequences of performance measurement, such as the overuse of broad-spectrum antibiotics, with the potential benefits of measures such as early administration of antibiotic therapy for pneumonia, a benefit that has not been demonstrated in randomized controlled trials. Dr Shekelle raises another interesting point—that one should expect the differences in mortality across hospitals to be small given the small difference in performance across hospitals. Randomized controlled trial evidence shows that many of the measured activities have important effects on individual mortality rates. Thus, we agree that all eligible patients should receive these interventions. However, our study suggests that these performance measures do not differentiate well between low-mortality and high-mortality hospitals. Our findings are consistent with Bradley et al,2 who showed that adherence with these performance measures accounts for a small amount of the variation in mortality across hospitals. Our study found that 1-year mortality rates at low-mortality hospitals were 0.13 lower than at high-mortality hospitals (0.27 vs 0.40). Yet the difference in mortality rates across hospitals ranked by performance ranged from 0.002 to 0.012, or less than one tenth of the total mortality difference across hospitals. Williams et al3 found that improvements in hospital performance over time did not translate into improvements in mortality rates. While these performance measures are tightly linked to patient outcomes on an individual level, their failure to explain differences in mortality rates across hospitals suggests they are not tightly linked to hospital-level outcomes. In our view, their utility as hospital measures of quality is questionable. Differences in mortality rates across hospitals are likely the result of a complex set of factors, including regional characteristics, hospital care processes, and organizational and environmental variables, as well as other factors that are unknown or currently unmeasured. Efforts should be directed toward identifying and developing quality measures that reflect these differences in mortality. In the meantime, combining quality measures based on process with those based on outcomes may take us farther toward the goal of identifying meaningful differences in quality across hospitals. Back to top Article Information Financial Disclosures: None reported. References 1. Wenger NS, Shekelle PG. Assessing care of vulnerable elders: ACOVE project overview. Ann Intern Med. 2001;135:642-64611601946Google ScholarCrossref 2. Bradley EH, Herrin J, Elbel B. et al. Hospital quality for acute myocardial infarction: correlation among process measures and relationship with short-term mortality. JAMA. 2006;296:72-7816820549Google ScholarCrossref 3. Williams SC, Schmaltz SP, Morton DJ, Koss RG, Loeb JM. Quality of care in US hospitals as reflected by standardized measures, 2002-2004. N Engl J Med. 2005;353:255-26416034011Google ScholarCrossref http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png JAMA American Medical Association

Medicare’s Hospital Compare Performance Measures and Mortality Rates—Reply

Werner, Rachel M.; Bradlow, Eric T.
JAMA , Volume 297 (13) – Apr 4, 2007
Download PDF
2 pages

Loading next page...
 
/lp/american-medical-association/medicare-s-hospital-compare-performance-measures-and-mortality-rates-3xPGNOqcv9

References (3)

Publisher
American Medical Association
Copyright
Copyright © 2007 American Medical Association. All Rights Reserved.
ISSN
0098-7484
eISSN
1538-3598
DOI
10.1001/jama.297.13.1431
Publisher site
See Article on Publisher Site

Abstract

Letters Section Editor: Robert M. Golub, MD, Senior Editor. In Reply: Dr Fierer raises an important point: 3 of the performance measures used by the Hospital Quality Alliance that we studied are not based on evidence from randomized controlled trials. It may, however, be reasonable to implement some performance measures based on observational data because some interventions are not feasible to study in a randomized controlled trial. Limiting measures to those with randomized controlled trial evidence would limit the clinical areas in which performance measurement is possible, increasing the difficulty of providing meaningful information about quality through performance measurement. Indeed, numerous established performance measures are based on results from observational studies and expert consensus.1 However, when the potential benefit from a measured intervention is uncertain or small, there is increased risk that the inaccuracy of performance measurement will outweigh the benefits. We agree that there must be vigilance in balancing the potential unintended consequences of performance measurement, such as the overuse of broad-spectrum antibiotics, with the potential benefits of measures such as early administration of antibiotic therapy for pneumonia, a benefit that has not been demonstrated in randomized controlled trials. Dr Shekelle raises another interesting point—that one should expect the differences in mortality across hospitals to be small given the small difference in performance across hospitals. Randomized controlled trial evidence shows that many of the measured activities have important effects on individual mortality rates. Thus, we agree that all eligible patients should receive these interventions. However, our study suggests that these performance measures do not differentiate well between low-mortality and high-mortality hospitals. Our findings are consistent with Bradley et al,2 who showed that adherence with these performance measures accounts for a small amount of the variation in mortality across hospitals. Our study found that 1-year mortality rates at low-mortality hospitals were 0.13 lower than at high-mortality hospitals (0.27 vs 0.40). Yet the difference in mortality rates across hospitals ranked by performance ranged from 0.002 to 0.012, or less than one tenth of the total mortality difference across hospitals. Williams et al3 found that improvements in hospital performance over time did not translate into improvements in mortality rates. While these performance measures are tightly linked to patient outcomes on an individual level, their failure to explain differences in mortality rates across hospitals suggests they are not tightly linked to hospital-level outcomes. In our view, their utility as hospital measures of quality is questionable. Differences in mortality rates across hospitals are likely the result of a complex set of factors, including regional characteristics, hospital care processes, and organizational and environmental variables, as well as other factors that are unknown or currently unmeasured. Efforts should be directed toward identifying and developing quality measures that reflect these differences in mortality. In the meantime, combining quality measures based on process with those based on outcomes may take us farther toward the goal of identifying meaningful differences in quality across hospitals. Back to top Article Information Financial Disclosures: None reported. References 1. Wenger NS, Shekelle PG. Assessing care of vulnerable elders: ACOVE project overview. Ann Intern Med. 2001;135:642-64611601946Google ScholarCrossref 2. Bradley EH, Herrin J, Elbel B. et al. Hospital quality for acute myocardial infarction: correlation among process measures and relationship with short-term mortality. JAMA. 2006;296:72-7816820549Google ScholarCrossref 3. Williams SC, Schmaltz SP, Morton DJ, Koss RG, Loeb JM. Quality of care in US hospitals as reflected by standardized measures, 2002-2004. N Engl J Med. 2005;353:255-26416034011Google ScholarCrossref

Journal

JAMA – American Medical Association

Published: Apr 4, 2007

There are no references for this article.