+ Site Statistics
+ Search Articles
+ PDF Full Text Service
How our service works
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ Translate
+ Recently Requested

Small sample mediation testing: misplaced confidence in bootstrapped confidence intervals

Small sample mediation testing: misplaced confidence in bootstrapped confidence intervals

Journal of Applied Psychology 100(1): 194-202

Bootstrapping is an analytical tool commonly used in psychology to test the statistical significance of the indirect effect in mediation models. Bootstrapping proponents have particularly advocated for its use for samples of 20-80 cases. This advocacy has been heeded, especially in the Journal of Applied Psychology, as researchers are increasingly utilizing bootstrapping to test mediation with samples in this range. We discuss reasons to be concerned with this escalation, and in a simulation study focused specifically on this range of sample sizes, we demonstrate not only that bootstrapping has insufficient statistical power to provide a rigorous hypothesis test in most conditions but also that bootstrapping has a tendency to exhibit an inflated Type I error rate. We then extend our simulations to investigate an alternative empirical resampling method as well as a Bayesian approach and demonstrate that they exhibit comparable statistical power to bootstrapping in small samples without the associated inflated Type I error. Implications for researchers testing mediation hypotheses in small samples are presented. For researchers wishing to use these methods in their own research, we have provided R syntax in the online supplemental materials.

Please choose payment method:

(PDF emailed within 0-6 h: $19.90)

Accession: 055818863

Download citation: RISBibTeXText

PMID: 24731180

DOI: 10.1037/a0036635

Related references

Reporting Frequency and Sample Size: Effects on Prediction, Confidence Levels, and Confidence Intervals. Journal of Behavioral Finance 8(3): 154-160, 2007

The empirical coverage of confidence intervals: point estimates and confidence intervals for confidence levels. Biometrical Journal. Biometrische Zeitschrift 54(4): 537-551, 2013

eComment. Everything changes even statistics: it is time to use bootstrapped confidence intervals?. Interactive Cardiovascular and Thoracic Surgery 19(1): 69-69, 2015

Testing with confidence: the use (and misuse) of confidence intervals in biomedical research. Journal of Science and Medicine in Sport 7(2): 135-137, 2004

Bootstrapped confidence intervals for the Cox model using a linear relative risk form. Statistics in Medicine 8(8): 927-935, 1989

The DBOOT program for calculating genetic distance and identity and their bootstrapped confidence intervals. Genetika Moskva 32(7): 1017-1020, 1996

Sample sizes for constructing confidence intervals and testing hypotheses. Statistics in Medicine 11(2): 269, 1992

Robust regression and small sample confidence intervals. Journal of Statistical Planning and Inference 57(1): 39-48, 1997

Simultaneous small-sample multivariate Bernoulli confidence intervals. Biometrics 41(4): 1001-1013, 1985

On small-sample confidence intervals for parameters in discrete distributions. Biometrics 57(3): 963-971, 2001

Unconditional small-sample confidence intervals for the odds ratio. Biostatistics 3(3): 379-386, 2003

The Effects of Nonnormal Distributions on Confidence Intervals Around the Standardized Mean Difference: Bootstrap and Parametric Confidence Intervals. Educational and Psychological Measurement 65(1): 51-69, 2005

Can't miss: conquer any number task by making important statistics simple. Part 4. Confidence intervals with t distributions, standard error, and confidence intervals for proportions. Journal for Healthcare Quality 26(4): 26-32, 2004

Confidence Intervals Make a Difference: Effects of Showing Confidence Intervals on Inferential Reasoning. Educational and Psychological Measurement 72(6): 1039-1052, 2012

Identifying patterns in the diet of mackerel icefish Champsocephalus gunnari at South Georgia using bootstrapped confidence intervals of a dietary index. Polar Biology 32(4): 569-581, 2009