Pay For Performance: Infection Rates Inaccurate In Determining Report Card Grades
ICU-acquired infection rates are not an indication of patients' mortality risk, according to researchers at the University of Pennsylvania, undermining a central tenet of many pay-for-performance initiatives.
Public reporting of quality data is increasingly common in health care. These report cards are designed to improve the quality of care by helping patients choose the best hospitals. Yet, they only work if they successfully identify high performers, and may be misleading if they steer patients toward poor performers.
To examine whether or not publicly-reported infection rates actually identify the best hospitals, Kate Courtright, M.D., resident physician at the University of Pennsylvania, and colleagues looked at patients in Pennsylvania hospitals especially at risk for two types of infections: pneumonia and blood stream.
They calculated hospital death rates accounting for differences in illness severity across 158 hospitals, which included nearly 19,000 admissions involving mechanical ventilation and over 16,000 ICU admissions involving central venous catheterization, and compared them to ICU-acquired infection rates. They then used rank correlation and linear regression to determine the relationship between infections and death.
“We found that ICU-acquired infection rates, as reported on a state website, did not correlate with death rates for at-risk patients,” said Dr. Courtright, lead author of the report. “In fact, hospitals with lower rates of ICU-acquired infection did not also have lower death rates for at-risk patients.” For example, the 43 hospitals that reported no cases of ICU-acquired pneumonia had an average death rate of 35.7 percent for patients receiving mechanical ventilation; hospitals with higher infection rates had an average death rate of 34.6 percent. These numbers were not statistically different.
Despite their limitations, Dr. Courtright noted, ICU-acquired infections rates are likely to continue to be a part of hospital report cards. However, “both policy-makers and the public should recognize that these rates, at least as reported by hospitals, provide limited information about the quality of the hospital, and may misidentify high and low performers,” she said. “More comprehensive report cards that report both complications like ICU-acquired infections and overall survival rates are needed to help patients make correct decisions.”
In the meantime, more care is needed to ensure that hospital report cards don't do more harm than good. This is especially important because under upcoming health care reform, infection rates are also to be used for hospital reimbursement. Such pay-for-performance strategies could actually penalize good hospitals with low mortality rates.
“Report cards only work if they successfully identify the best hospitals,” concluded Dr. Courtright. “We were surprised to find that many hospitals with good report cards from an infection standpoint are not that good from a more important standpoint — patient survival. Additionally, many hospitals with high infection rates actually had very good survival rates. Using these report cards to choose a hospital may be misleading and potentially harmful.”