+ Site Statistics
+ Search Articles
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ PDF Full Text
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Translate
+ Recently Requested

Examining publication bias-a simulation-based evaluation of statistical tests on publication bias

Examining publication bias-a simulation-based evaluation of statistical tests on publication bias

Peerj 5: E4115

Publication bias is a form of scientific misconduct. It threatens the validity of research results and the credibility of science. Although several tests on publication bias exist, no in-depth evaluations are available that examine which test performs best for different research settings. Four tests on publication bias, Egger's test (FAT), p-uniform, the test of excess significance (TES), as well as the caliper test, were evaluated in a Monte Carlo simulation. Two different types of publication bias and its degree (0%, 50%, 100%) were simulated. The type of publication bias was defined either as file-drawer, meaning the repeated analysis of new datasets, or p-hacking, meaning the inclusion of covariates in order to obtain a significant result. In addition, the underlying effect (β = 0, 0.5, 1, 1.5), effect heterogeneity, the number of observations in the simulated primary studies (N = 100, 500), and the number of observations for the publication bias tests (K = 100, 1,000) were varied. All tests evaluated were able to identify publication bias both in the file-drawer and p-hacking condition. The false positive rates were, with the exception of the 15%- and 20%-caliper test, unbiased. The FAT had the largest statistical power in the file-drawer conditions, whereas under p-hacking the TES was, except under effect heterogeneity, slightly better. The CTs were, however, inferior to the other tests under effect homogeneity and had a decent statistical power only in conditions with 1,000 primary studies. The FAT is recommended as a test for publication bias in standard meta-analyses with no or only small effect heterogeneity. If two-sided publication bias is suspected as well as under p-hacking the TES is the first alternative to the FAT. The 5%-caliper test is recommended under conditions of effect heterogeneity and a large number of primary studies, which may be found if publication bias is examined in a discipline-wide setting when primary studies cover different research problems.

(PDF emailed within 1 workday: $29.90)

Accession: 063750689

Download citation: RISBibTeXText

PMID: 29204324

Related references

Systematic evaluation and comparison of statistical tests for publication bias. Journal of Epidemiology 15(6): 235-243, 2005

Review of publication bias in studies on publication bias: mandatory publication of data may help. Bmj 331(7517): 638-638, 2005

Review of publication bias in studies on publication bias: studies on publication bias are probably susceptible to the bias they study. Bmj 331(7517): 637-638, 2005

The specter of publication bias: adjustment for publication bias in the evidence on cardiac death associated with passive smoking in nonsmoking women. International Journal of Cardiology 149(3): 388-389, 2011

The relationship between study findings and publication outcome in anesthesia research: a retrospective observational study examining publication bias. Canadian Journal of Anaesthesia 63(6): 682-690, 2017

Systematic review of publication bias in studies on publication bias. Bmj 331(7514): 433-434, 2005

Publication and related bias in meta-analysis: Power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology 53(11): 1119-1129, 2000

Inflation of type I error rate in two statistical tests for the detection of publication bias in meta-analyses with binary outcomes. Statistics in Medicine 21(17): 2465-2477, 2002

Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study. Bmc Medical Research Methodology 9: 2, 2009

Publication bias: evidence of delayed publication in a cohort study of clinical research projects. Bmj 315(7109): 640-645, 1997

Associations with publication and assessing publication bias in dementia diagnostic test accuracy studies. International Journal of Geriatric Psychiatry 30(12): 1250-1256, 2016

Time to full publication of studies of anticancer drugs for breast cancer, and the potential for publication bias. International Journal of Technology Assessment in Health Care 26(1): 110-116, 2010

Non-publication and publication bias in reproductive medicine: a cohort analysis. Human Reproduction 32(8): 1658-1666, 2018

The publication process itself was the major cause of publication bias in genetic epidemiology. Journal of Clinical Epidemiology 59(12): 1312-1318, 2006

Full publication of clinical trials presented at a national maternal-fetal medicine meeting: is there a publication bias?. American Journal of Perinatology 26(9): 679-682, 2010