The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline

Erik Wetter; Michelangelo Vianello; Eric Jan M. Wagenmakers; Jesse J. Chandler; Felix Cheung; Fiery A. Cushman; Jesse R. Graham; Matt Motyl; Jay J. Van Bavel; Martin Schweinsberg; Nikhil Madan; S. Amy Sommer; Jennifer Jordan; Tierney Warren; Eli Awtrey; Luke Lei Zhu; Daniel Diermeier; Justin E. Heinze; Malavika Srinivasan; David Tannenbaum; Eliza Bivolaru; Jason Dana; Clintin P. Davis-Stober; Christilene du Plessis; Quentin F. Gronau; Andrew C. Hafenbrack; Eko Yi Liao; Alexander Ly; Maarten Marsman; Toshio Murase; Israr Qureshi; Michael Schaerer; Nico Thornley; Christina M. Tworek; Tabitha Anderson; Christopher W. Bauman; Wendy L. Bedwell; Victoria Brescoll; Andrew Canavan; Erik Cheries; Sapna Cheryan; Andrei Cimpian; Mark A. Clark; Diana Cordon; Peter H. Ditto; Thomas Donahue; Sarah E. Frick; Monica Gamez-Djokic; Lynn Wong; Rebecca Hofstein Grady; Jun Gu; Adam Hahn; Brittany E. Hanson; Nicole J. Hartwich; Kristie Hein; Yoel Inbar; Lilly Jiang; Tehlyr Kellogg; Deanna M. Kennedy; Nicole Legate; Timo P. Luoma; Heidi Maibuecher; Peter Meindl; Jennifer Miles; Alexandra Mislin; Daniel C. Molden; George Newman; Hoai Huong Ngo; Harvey Packham; Philip S. Ramsay; Jennifer L. Ray; Aaron M. Sackett; Anne-Laure Sellier; Tatiana Sokolova; Walter Sowden; Daniel Storage; Xiaomin Sun; Anthony N. Washburn; Cong Wei; Carlos T. Wilson; Sophie-Charlotte Darroux; Eric Luis Uhlmann

doi:10.1016/j.jesp.2015.10.001

Back

The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline

Journal article

Open access

Peer reviewed

The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline

Erik Wetter, Michelangelo Vianello, Eric Jan M. Wagenmakers, Jesse J. Chandler, Felix Cheung, Fiery A. Cushman, Jesse R. Graham, Matt Motyl, Jay J. Van Bavel, Martin Schweinsberg, …

Journal of Experimental Social Psychology, Vol.66, pp.55-67

2016

DOI: https://doi.org/10.1016/j.jesp.2015.10.001

Abstract

Crowdsourcing science

Meta-science

Methodology

Replication

Reproducibility

Research transparency

This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which findings are replicated in qualified independent laboratories before (rather than after) they are published. Our goal is to establish a non-adversarial replication process with highly informative final results. To illustrate the Pre-Publication Independent Replication (PPIR) approach, 25 research groups conducted replications of all ten moral judgment effects which the last author and his collaborators had “in the pipeline” as of August 2014. Six findings replicated according to all replication criteria, one finding replicated but with a significantly smaller effect size than the original, one finding replicated consistently in the original culture but not outside of it, and two findings failed to find support. In total, 40% of the original findings failed at least one major replication criterion. Potential ways to implement and incentivize pre-publication independent replication on a large scale are discussed.

Files and links (2)

pdf

SSE_2016_Article_Wetter_Erik_the pipeline532.21 kBDownload View

Published (Version of record)CC BY-NC-ND V4.0, Open Access

url

https://doi.org/10.1016/j.jesp.2015.10.001View

Published (Version of record)CC BY-NC-ND V4.0, Open

Metrics

1 Record Views

66 Times Cited - Web of Science

Details

Title: The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline
Creators - without role: Erik Wetter - Stockholm School of Economics, Department of Management and Organization
Michelangelo Vianello - University of Padova
Eric Jan M. Wagenmakers - University of Amsterdam (NL)
Jesse J. Chandler - University of Michigan (US)
Felix Cheung - Michigan State University (US)
Fiery A. Cushman - Harvard University (US)
Jesse R. Graham - Salem State University
Matt Motyl - University of Illinois (US)
Jay J. Van Bavel - New York University
Martin Schweinsberg - INSEAD (FR)
Nikhil Madan - INSEAD (FR)
S. Amy Sommer - HEC Paris (FR)
Jennifer Jordan - University of Groningen (NL)
Tierney Warren - INSEAD (FR)
Eli Awtrey - University of Washington in St. Louis (US)
Luke Lei Zhu - University of Manitoba
Daniel Diermeier - University of Chicago (US)
Justin E. Heinze - University of Michigan (US)
Malavika Srinivasan - Harvard University (US)
David Tannenbaum - University of Chicago (US)
Eliza Bivolaru - INSEAD (FR)
Jason Dana - Yale University (US)
Clintin P. Davis-Stober - University of Missouri System
Christilene du Plessis - Rotterdam School of Management (NL)
Quentin F. Gronau - University of Amsterdam (NL)
Andrew C. Hafenbrack - Católica Lisbon School of Business and Economics
Eko Yi Liao - Macau University of Science and Technology
Alexander Ly - University of Amsterdam (NL)
Maarten Marsman - University of Amsterdam (NL)
Toshio Murase - Roosevelt University
Israr Qureshi - IE Business School
Michael Schaerer - INSEAD (FR)
Nico Thornley - INSEAD (FR)
Christina M. Tworek - University of Illinois (US)
Tabitha Anderson - Illinois Institute of Technology
Christopher W. Bauman - University of California (US)
Wendy L. Bedwell - University of South Florida
Victoria Brescoll - Yale University (US)
Andrew Canavan - Illinois Institute of Technology
Erik Cheries - University of Massachusetts
Sapna Cheryan - Washington University (US)
Andrei Cimpian - University of Illinois (US)
Mark A. Clark - American University Washington
Diana Cordon - Illinois Institute of Technology
Peter H. Ditto - University of California (US)
Thomas Donahue - Illinois Institute of Technology
Sarah E. Frick - University of South Florida
Monica Gamez-Djokic - Northwestern University (US)
Lynn Wong - INSEAD (FR)
Rebecca Hofstein Grady - University of California (US)
Jun Gu - Monash University (AU)
Adam Hahn - University of Cologne
Brittany E. Hanson - University of Illinois (US)
Nicole J. Hartwich - University of Cologne
Kristie Hein - Illinois Institute of Technology
Yoel Inbar - University of Toronto (CA)
Lilly Jiang - University of Washington in Seattle
Tehlyr Kellogg - Illinois Institute of Technology
Deanna M. Kennedy - Washington University (US)
Nicole Legate - Illinois Institute of Technology
Timo P. Luoma - University of Cologne
Heidi Maibuecher - Illinois Institute of Technology
Peter Meindl - University of Southern California (US)
Jennifer Miles - University of California (US)
Alexandra Mislin - American University Washington
Daniel C. Molden - Northwestern University (US)
George Newman - Yale University (US)
Hoai Huong Ngo - University Paris Nanterre
Harvey Packham - University of Hong Kong (CN)
Philip S. Ramsay - University of South Florida
Jennifer L. Ray - New York University
Aaron M. Sackett - University of St. Thomas
Anne-Laure Sellier - HEC Paris (FR)
Tatiana Sokolova - University of Michigan (US)
Walter Sowden - University of Michigan (US)
Daniel Storage - University of Illinois (US)
Xiaomin Sun - Beijing Normal University (CN)
Anthony N. Washburn - University of Illinois (US)
Cong Wei - Beijing Normal University (CN)
Carlos T. Wilson - Illinois Institute of Technology
Sophie-Charlotte Darroux - INSEAD (FR)
Eric Luis Uhlmann - INSEAD (FR)
Publication Details: Journal of Experimental Social Psychology, Vol.66, pp.55-67
Publisher: Elsevier
Number of pages: 13
Identifiers: 991001480491506056
Academic Unit: Department of Management and Organization
Language: English
Resource Type: Journal article

The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline

Abstract

Files and links (2)

Metrics

Details

Stockholm School of Economics Social media