• Megan L Head, Luke Holman, Rob Lanfear, Andrew T Kahn, and Michael D Jennions.
• Division of Evolution, Ecology and Genetics, Research School of Biology, Australian National University, Acton, Canberra, Australia.
• Plos Biol. 2015 Mar 1; 13 (3): e1002106e1002106.

AbstractA focus on novel, confirmatory, and statistically significant results leads to substantial bias in the scientific literature. One type of bias, known as "p-hacking," occurs when researchers collect or select data or statistical analyses until nonsignificant results become significant. Here, we use text-mining to demonstrate that p-hacking is widespread throughout science. We then illustrate how one can test for p-hacking when performing a meta-analysis and show that, while p-hacking is probably common, its effect seems to be weak relative to the real effect sizes being measured. This result suggests that p-hacking probably does not drastically alter scientific consensuses drawn from meta-analyses.

This article appears in the collection: Drowning in the Sea of Evidence.

7 keywords...

hide…