+ Site Statistics
+ Search Articles
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ PDF Full Text
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Translate
+ Recently Requested

Concordance and robustness of quality indicator sets for hospitals: an analysis of routine data

Concordance and robustness of quality indicator sets for hospitals: an analysis of routine data

Bmc Health Services Research 11: 106

Hospitals are increasingly being evaluated with respect to the quality of provided care. In this setting, several indicator sets compete with one another for the assessment of effectiveness and safety. However, there have been few comparative investigations covering different sets. The objective of this study was to answer three questions: How concordant are different indicator sets on a hospital level? What is the effect of applying different reference values? How stable are the positions of a hospital ranking? Routine data were made available to three companies offering the Patient Safety Indicators, an indicator set from the HELIOS Hospital Group, and measurements based on Disease Staging™. Ten hospitals from North Rhine-Westphalia, comprising a total of 151,960 inpatients in 2006, volunteered to participate in this study. The companies provided standard quality reports for the ten hospitals. Composite measures were defined for strengths and weaknesses. In addition to the different indicator sets, different reference values for one set allowed the construction of several comparison groups. Concordance and robustness were analyzed using the non-parametric correlation coefficient and Kendall's W. Indicator sets differing only in the reference values of the indicators showed significant correlations in most of the pairs with respect to weaknesses (maximum r = 0.927, CI 0.714-0.983, p < 0.001). There were also significant correlations between different sets (maximum r = 0.829, CI 0.417-0.958, p = 0.003) having different indicators or when different methods for performance assessment were applied. The results were weaker measuring hospital strengths (maximum r = 0.669, CI 0.068-0.914, p = 0.034). In a hospital ranking, only two hospitals belonged consistently either to the superior or to the inferior half of the group. Even altering reference values or the supplier for the same indicator set changed the rank for nine out of ten hospitals. Our results reveal an unsettling lack of concordance in estimates of hospital performance when different quality indicator sets are used. These findings underline the lack of consensus regarding optimal validated measures for judging hospital quality. The indicator sets shared a common definition of quality, independent of their focus on patient safety, mortality, or length of stay. However, for most of the hospitals, changing the indicator set or the reference value resulted in a shift from the superior to the inferior half of the group or vice versa. Thus, while taken together the indicator sets offer the hospitals complementary pictures of their quality, on an individual basis they do not establish a reliable ranking.

(PDF emailed within 0-6 h: $19.90)

Accession: 036094640

Download citation: RISBibTeXText

PMID: 21592374

DOI: 10.1186/1472-6963-11-106

Related references

30-Day Survival Probabilities as a Quality Indicator for Norwegian Hospitals: Data Management and Analysis. Plos One 10(9): E0136547, 2016

Measuring stroke care and quality in routine data sets. JAMA Neurology 70(1): 130-131, 2013

Using routine data to evaluate quality of care in British hospitals. Medical Care 35(10 Suppl): Os102-Os111, 1997

Quality assurance in German hospitals - federal quality of care monitoring vs. evaluation of routine clinical data. A head-to-head comparison on the example of pressure ulcers. Gesundheitswesen ) 73(12): 803-809, 2012

Concordance and predictive value of two adverse drug event data sets. Bmc Medical Informatics and Decision Making 14: 74, 2015

Improving Concordance of MRI and PET/CT Interpretations With Retrospectively Coregistered MRI and PET/CT Data Sets. Current Problems in Diagnostic Radiology 44(3): 232-236, 2016

Robustness of Hospital Benchmarking with the Hospital Standardized Mortality Ratio (HSMR): An Analysis of Secondary Data from 37 German Hospitals. Gesundheitswesen ) 78(10): 637-644, 2015

Concordance among chromosomal genic and morphologic data sets among selected peromyscine rodents. American Zoologist 24(3): 80A, 1984

Analysis of multicrystal pump-probe data sets. II. Scaling of ratio data sets. Acta Crystallographica. Section A, Foundations and Advances 72(Pt 2): 250-260, 2016

The concordance of terpenoid, ISSR and RAPD markers, and ITS sequence data sets among genotypes: an example from Juniperus. Biochemical Systematics and Ecology 31(4): 375-387, 2003

Incidence of coercive measures as an indicator of quality in psychiatric hospitals. Problems of data recording and processing, preliminary results of a benchmarking study. Psychiatrische Praxis 34(1): 26-33, 2006

Disentangling quality and safety indicator data: a longitudinal, comparative study of hand hygiene compliance and accreditation outcomes in 96 Australian hospitals. Bmj Open 4(9): E005284, 2015

International prevalence of adverse drug events in hospitals: an analysis of routine data from England, Germany, and the USA. Bmc Health Services Research 14(): 125-125, 2014

Congruence or concordance in archaeobotany: Assessing micro- and macro-botanical data sets from Icelandic middens. Journal of Archaeological Science 26(7): 833-844, July, 1999

Computational analysis of unassigned high-quality MS/MS spectra in proteomic data sets. Proteomics 10(14): 2712-2718, 2010