eCite Digital Repository

Standardized mean differences cause funnel plot distortion in publication bias assessments


Zwetsloot, PP and Van Der Naald, M and Sena, ES and Howells, DW and IntHout, J and De Groot, JAH and Chamuleau, SAJ and MacLeod, MR and Wever, KE, Standardized mean differences cause funnel plot distortion in publication bias assessments, eLife, 6 Article e24260. ISSN 2050-084X (2017) [Refereed Article]

PDF (Published version)

Copyright Statement

Copyright 2017 Bradshaw et al. This article is distributed under the terms of the Creative Commons Attribution 4.0 International (CC BY 4.0) License, ( which permits unrestricted use and redistribution provided that the original author and source are credited.

DOI: doi:10.7554/eLife.24260


Meta-analyses are increasingly used for synthesis of evidence from biomedical research, and often include an assessment of publication bias based on visual or analytical detection of asymmetry in funnel plots. We studied the influence of different normalisation approaches, sample size and intervention effects on funnel plot asymmetry, using empirical datasets and illustrative simulations. We found that funnel plots of the Standardized Mean Difference (SMD) plotted against the standard error (SE) are susceptible to distortion, leading to overestimation of the existence and extent of publication bias. Distortion was more severe when the primary studies had a small sample size and when an intervention effect was present. We show that using the Normalised Mean Difference measure as effect size (when possible), or plotting the SMD against a sample size-based precision estimate, are more reliable alternatives. We conclude that funnel plots using the SMD in combination with the SE are unsuitable for publication bias assessments and can lead to false-positive results.

Item Details

Item Type:Refereed Article
Keywords:metaanalysis, funnel plot distortion
Research Division:Biological Sciences
Research Group:Other biological sciences
Research Field:Other biological sciences not elsewhere classified
Objective Division:Health
Objective Group:Clinical health
Objective Field:Diagnosis of human diseases and conditions
UTAS Author:Howells, DW (Professor David Howells)
ID Code:152230
Year Published:2017
Web of Science® Times Cited:85
Deposited By:Medicine
Deposited On:2022-08-15
Last Modified:2022-09-19
Downloads:1 View Download Statistics

Repository Staff Only: item control page