+ Site Statistics
References:
54,258,434
Abstracts:
29,560,870
PMIDs:
28,072,757
+ Search Articles
+ PDF Full Text Service
How our service works
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ Translate
+ Recently Requested

Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research



Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research



Journal of Evaluation in Clinical Practice 18(1): 12-18



The Cochrane Collaboration is strongly encouraging the use of a newly developed tool, the Cochrane Collaboration Risk of Bias Tool (CCRBT), for all review groups. However, the psychometric properties of this tool to date have yet to be described. Thus, the objective of this study was to add information about psychometric properties of the CCRBT including inter-rater reliability and concurrent validity, in comparison with the Effective Public Health Practice Project Quality Assessment Tool (EPHPP). Both tools were used to assess the methodological quality of 20 randomized controlled trials included in our systematic review of the effectiveness of knowledge translation interventions to improve the management of cancer pain. Each study assessment was completed independently by two reviewers using each tool. We analysed the inter-rater reliability of each tool's individual domains, as well as final grade assigned to each study. The EPHPP had fair inter-rater agreement for individual domains and excellent agreement for the final grade. In contrast, the CCRBT had slight inter-rater agreement for individual domains and fair inter-rater agreement for final grade. Of interest, no agreement between the two tools was evident in their final grade assigned to each study. Although both tools were developed to assess 'quality of the evidence', they appear to measure different constructs. Both tools performed quite differently when evaluating the risk of bias or methodological quality of studies in knowledge translation interventions for cancer pain. The newly introduced CCRBT assigned these studies a higher risk of bias. Its psychometric properties need to be more thoroughly validated, in a range of research fields, to understand fully how to interpret results from its application.

Please choose payment method:






(PDF emailed within 0-6 h: $19.90)

Accession: 051666165

Download citation: RISBibTeXText

PMID: 20698919

DOI: 10.1111/j.1365-2753.2010.01516.x


Related references

Methodological quality and implications for practice of systematic Cochrane reviews in pediatric oral health: a critical assessment. Bmc Oral Health 14: 35, 2014

Evaluation of methodological quality of systematic reviews and meta-analyses: AMSTAR (A Measurement Tool for the Assessment of Multiple Systematic Reviews). Rofo 184(10): 937-940, 2013

Most systematic reviews of high methodological quality on psoriasis interventions are classified as high risk of bias using ROBIS tool. Journal of Clinical Epidemiology 92: 79-88, 2017

Evaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials: overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews. Systematic Reviews 5: 80, 2017

Methodological quality of systematic reviews in subfertility: a comparison of Cochrane and non-Cochrane systematic reviews in assisted reproductive technologies. Human Reproduction 27(12): 3460-3466, 2013

Practicalities of using a modified version of the Cochrane Collaboration risk of bias tool for randomised and non-randomised study designs applied in a health technology assessment setting. Research Synthesis Methods 5(3): 200-211, 2016

The evolution of assessing bias in Cochrane systematic reviews of interventions: celebrating methodological contributions of the Cochrane Collaboration. Systematic Reviews 2: 79, 2014

Risk of Bias in Systematic Reviews of Non-Randomized Studies of Adverse Cardiovascular Effects of Thiazolidinediones and Cyclooxygenase-2 Inhibitors: Application of a New Cochrane Risk of Bias Tool. Plos Medicine 13(4): E1001987, 2016

Development of a quality assessment tool for systematic reviews of observational studies (QATSO) of HIV prevalence in men having sex with men and associated risk behaviours. Emerging Themes in Epidemiology 5(): 23-23, 2008

Development of the anatomical quality assessment (AQUA) tool for the quality assessment of anatomical studies included in meta-analyses and systematic reviews. Clinical Anatomy 30(1): 6-13, 2016

Bias risk assessment: (3) Revised Cochrane bias risk assessment tool for individual randomized, cross-over trials (RoB2.0). Zhonghua Liu Xing Bing Xue Za Zhi 38(10): 1436-1440, 2018

Evaluating the methodologic quality of systematic reviews and meta-analyses. AMSTAR (A Measurement Tool for the Assessment of Multiple Systematic Reviews. Rofo 185(10): 937-940, 2014

Disagreements in risk of bias assessment for randomised controlled trials included in more than one Cochrane systematic reviews: a research on research study using cross-sectional design. BMJ Open 9(4): E028382, 2019

Risk on bias assessment: (6) A Revised Tool for the Quality Assessment on Diagnostic Accuracy Studies (QUADAS-2). Zhonghua Liu Xing Bing Xue Za Zhi 39(4): 524-531, 2018

Overall confidence in the results of systematic reviews on exercise therapy for chronic low back pain: a cross-sectional analysis using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) 2 tool. Brazilian Journal of Physical Therapy 2019, 2019