+ Site Statistics
+ Search Articles
+ PDF Full Text Service
How our service works
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ Translate
+ Recently Requested

Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: a database survey

Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: a database survey

BMJ 344: D7762

To examine the potential for publication bias, data availability bias, and reviewer selection bias in recently published meta-analyses that use individual participant data and to investigate whether authors of such meta-analyses seemed aware of these issues. In a database of 383 meta-analyses of individual participant data that were published between 1991 and March 2009, we surveyed the 31 most recent meta-analyses of randomised trials that examined whether an intervention was effective. Identification of relevant articles and data extraction was undertaken by one author and checked by another. Only nine (29%) of the 31 meta-analyses included individual participant data from "grey literature" (such as unpublished studies) in their primary meta-analysis, and the potential for publication bias was discussed or investigated in just 10 (32%). Sixteen (52%) of the 31 meta-analyses did not obtain all the individual participant data requested, yet five of these (31%) did not mention this as a potential limitation, and only six (38%) examined how trials without individual participant data might affect the conclusions. In nine (29%) of the meta-analyses reviewer selection bias was a potential issue, as the identification of relevant trials was either not stated or based on a more selective, non-systematic approach. Investigation of four meta-analyses containing data from ≥10 trials revealed one with an asymmetric funnel plot consistent with publication bias, and the inclusion of studies without individual participant data revealed additional heterogeneity between trials. Publication, availability, and selection biases are a potential concern for meta-analyses of individual participant data, but many reviewers neglect to examine or discuss them. These issues warn against uncritically viewing any meta-analysis that uses individual participant data as the most reliable. Reviewers should seek individual participant data from all studies identified by a systematic review; include, where possible, aggregate data from any studies lacking individual participant data to consider their potential impact; and investigate funnel plot asymmetry in line with recent guidelines.

Please choose payment method:

(PDF emailed within 0-6 h: $19.90)

Accession: 051664857

Download citation: RISBibTeXText

PMID: 22214758

DOI: 10.1136/bmj.d7762

Related references

Is there bias in the publication of individual patient data meta-analyses?. International Journal of Technology Assessment in Health Care 16(2): 657-667, 2000

Assessment of funnel plot asymmetry and publication bias in reproductive health meta-analyses: an analytic survey. Reproductive Health 4: 3, 2007

Cognitive Bias Modification for Behavior Change in Alcohol and Smoking Addiction: Bayesian Meta-Analysis of Individual Participant Data. Neuropsychology Review 2019, 2019

Publication bias in meta-analyses from the Cochrane Database of Systematic Reviews. Statistics in Medicine 34(20): 2781-2793, 2016

Understanding selection bias, time-lags and measurement bias in secondary data sources: Putting the Encyclopedia of Associations database in broader context. Social Science Research 42(6): 1750-1764, 2013

Individual participant data meta-analyses compared with meta-analyses based on aggregate data. Cochrane Database of Systematic Reviews 9: Mr000007, 2016

Correction of Selection Bias in Survey Data: Is the Statistical Cure Worse Than the Bias?. American Journal of Public Health 107(4): 503-505, 2017

Alcohol intake and gastric cancer: Meta-analyses of published data versus individual participant data pooled analyses (StoP Project). Cancer Epidemiology 54: 125-132, 2018

Tobacco smoking and gastric cancer: meta-analyses of published data versus pooled analyses of individual participant data (StoP Project). European Journal of Cancer Prevention 27(3): 197-204, 2018

Review of publication bias in studies on publication bias: mandatory publication of data may help. BMJ 331(7517): 638-638, 2005

Publication Bias in Meta-Analyses: Prevention, Assessment and Adjustments. Biometrics 62(2): 626-627, 2006

Assessment of Publication Bias in Meta-Analyses of Cardiovascular Diseases. Journal of Epidemiology and Community Health (1979-) 59(10): 864-869, 2005

Empirical assessment of effect of publication bias on meta-analyses. BMJ 320(7249): 1574-1577, 2000

Assessment of publication bias in meta-analyses of cardiovascular diseases. Journal of Epidemiology and Community Health 59(10): 864-869, 2005

The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey. Cmaj 176(8): 1091-1096, 2007