Measurement & Quality Initiatives
Commonwealth Study Demonstrates "Public Reporting Matters"
|WCHQ Measure||First Year of Measurement||Significant Improvement (through 2008)||Number of Years to Improve||Percentage Improvement Since First Year|
|HbA1c Control (less than 7.0%)||2003-2004||Yes||4||8.9|
|Kidney Function Monitored||2003-2004||Yes||2||17.3|
|LDL Control (less than 100 mg/dL)||2003-2004||Yes||2||14.9|
|Blood Pressure Control (less than 130/80 mmHg)||2006-2007||No||--||2.0|
|Coronary Artery Disease|
|LDL Control (less than 100 mg/dL)||2007||No||--||1.2|
|Blood Pressure Control (less than 140/90 mmHg)||2004-2005||Yes||2||9.1|
|Screening for Pneumococcal Vaccinations||2007||No||--||4.3|
|Breast Cancer Screening||2004-2005||Yes||4||4.0|
|Cervical Cancer Screening||2003-2005||No||--||4.3|
|Colorectal Cancer Screening||2005||Yes||3||6.7|
The study also evaluated the rate of improvement. Analysis of each member group demonstrated the following findings:
- No correlation between group size and rate of improvement
- Variable correlation with rate of improvement and the group decision to focus on that measure
- Strong correlation between the initial rank of a group compared to its peers and the subsequent rate of improvement (Figure 2)
Relationship between rate of improvement and initial comparative ranking for Glycohemoglobin testing
In general, programs that were initially ranked quite low compared to their counterparts improved at a greater rate. This last finding, coupled with the demonstrated improvement in all but one of the measures, shows that "public reporting matters."
How did participating healthcare providers react to the public reporting of WCHQ measures?
The member survey demonstrated that it was common for WCHQ member organizations to focus on WCHQ measures during the study period. Every group reported formally giving priority for improvement to at least one WCHQ measure, in response to WCHQ reporting. Nine groups indicated their priorities were always or nearly always in response to WCHQ reporting, while seven showed a mix of responses, with five of those only occasionally choosing their priorities in response to WCHQ reporting.
While there are some missing data for the detailed questions on quality improvement implementation timing, the study obtained relatively complete information on whether WCHQ member clinics have implemented the activities asked about. Overall, there was a significant amount of activity in implementing systems and procedures to improve care quality and outcomes at the clinic level. The most common initiatives implemented by WCHQ members at care sites were adopting guidelines (85% - 87%) and patient reminders (76% - 82%). One-on-one diabetes education (81%) and providing diabetes data to primary providers (81%) were also very common.
Are there differences in quality improvement when WCHQ participation is compared to non-participation?
The Dartmouth Institute completed a comparative analysis of WCHQ member organizations versus the remainder of Wisconsin, Iowa/South Dakota and the rest of the United States, focusing on performance and rates of change for the diabetes related measures and mammography.
The study demonstrated that WCHQ member organizations outperformed the comparator groups, including the remainder of Wisconsin, nearby states of Iowa and South Dakota, and the rest of the United States in measures of glycohemoglobin testing, lipid testing in diabetics (Figure 3) and breast cancer screening (Figure 4) — all of which are publicly reported through WCHQ. The statistical data supporting the graphical displays can be found in Figure 5.
In each of these measures, there was a trend toward the rate of improvement during the study years being higher for WCHQ members, but this did not reach statistical significance. In contrast, Iowa and South Dakota patient populations were more likely to have received a diabetes related eye examination, which is not among WCHQ's publicly reported measures.
Patients of WCHQ members also tend to be somewhat more affluent and less likely to be on Medicaid than the comparison groups. This may create a bias in favor of better performance among WCHQ members. However, the performance of Iowa/South Dakota on the diabetic eye examination demonstrates that more is at play than just demographics.
Performance on diabetes related measures for 2004-2007, using 20% Medicare sample (Adjusted for differences in age, gender, race and income)
Performance on mammography screening for 2004-2007, using 20% Medicare sample (Adjusted for differences in age, gender, race and income)
Statistical comparison between WCHQ and comparator groups
|Measures||WCHQ||Non WCHQ, WI||IA/SD||Rest of the US|
|Hgb A1c testing|
|Odds ratio - annual change WCHQ vs.||--||1.06*||1.06*||1.05*|
|Odds ratio - annual change WCHQ vs.||--||1.01||0.99||0.99|
|Odds Ratio - annual change WCHQ vs.||--||1.05||1.05*||1.07**|
|All 3 tests|
|Odds ratio - annual change WCHQ vs.||--||1.02||0.99||1.01|
|Odds ratio - annual change WCHQ vs.||--||1.03||1.02||1.04|
* P less than 0.05; Note that due to multiple comparisons a p value less than 0.017 required for significance.
** P less than 0.017 (multiple comparison adjusted significance value)
The three components of this study provide compelling evidence that public reporting of ambulatory measures led to sustained improved performance among WCHQ member organizations. Clearly this was not a randomized controlled study and, therefore, there are several potential weaknesses. The decision to join WCHQ is voluntary and, as such, the members are highly motivated. The decision by groups to focus on WCHQ measures for improvement and the apparently more rapid improvement among the lower performing practice groups strongly suggest that public reporting of comparative performance is a true driver for improvement. The findings are a nice example of the old management adage of "you manage what you measure."
This study clearly indicates the value of membership in an organization such as WCHQ, in public reporting of healthcare outcomes and working collaboratively to improve care.
Save/Print this page (PDF)