دورية أكاديمية
أعرض في EDS

Challenges for assessing replicability in preclinical cancer biology.

التفاصيل البيبلوغرافية
العنوان: Challenges for assessing replicability in preclinical cancer biology.
المؤلفون: Errington TM; Center for Open Science, Charlottesville, United States., Denis A; Center for Open Science, Charlottesville, United States., Perfito N; Science Exchange, Palo Alto, United States., Iorns E; Science Exchange, Palo Alto, United States., Nosek BA; Center for Open Science, Charlottesville, United States.; University of Virginia, Charlottesville, United States.
المصدر: ELife [Elife] 2021 Dec 07; Vol. 10. Date of Electronic Publication: 2021 Dec 07.
نوع المنشور: Journal Article; Research Support, Non-U.S. Gov't
اللغة: English
بيانات الدورية: Publisher: eLife Sciences Publications, Ltd Country of Publication: England NLM ID: 101579614 Publication Model: Electronic Cited Medium: Internet ISSN: 2050-084X (Electronic) Linking ISSN: 2050084X NLM ISO Abbreviation: Elife Subsets: MEDLINE
أسماء مطبوعة: Original Publication: Cambridge, UK : eLife Sciences Publications, Ltd., 2012-
مواضيع طبية MeSH: Neoplasms* , Reproducibility of Results*, Biomedical Research/*methods, Animals ; Humans ; Research Design
مستخلص: We conducted the Reproducibility Project: Cancer Biology to investigate the replicability of preclinical research in cancer biology. The initial aim of the project was to repeat 193 experiments from 53 high-impact papers, using an approach in which the experimental protocols and plans for data analysis had to be peer reviewed and accepted for publication before experimental work could begin. However, the various barriers and challenges we encountered while designing and conducting the experiments meant that we were only able to repeat 50 experiments from 23 papers. Here we report these barriers and challenges. First, many original papers failed to report key descriptive and inferential statistics: the data needed to compute effect sizes and conduct power analyses was publicly accessible for just 4 of 193 experiments. Moreover, despite contacting the authors of the original papers, we were unable to obtain these data for 68% of the experiments. Second, none of the 193 experiments were described in sufficient detail in the original paper to enable us to design protocols to repeat the experiments, so we had to seek clarifications from the original authors. While authors were extremely or very helpful for 41% of experiments, they were minimally helpful for 9% of experiments, and not at all helpful (or did not respond to us) for 32% of experiments. Third, once experimental work started, 67% of the peer-reviewed protocols required modifications to complete the research and just 41% of those modifications could be implemented. Cumulatively, these three factors limited the number of experiments that could be repeated. This experience draws attention to a basic and fundamental concern about replication - it is hard to assess whether reported findings are credible.
Competing Interests: TE, AD, BN Employed by the nonprofit Center for Open Science that has a mission to increase openness, integrity, and reproducibility of research, NP, EI Employed by and hold shares in Science Exchange Inc
(© 2021, Errington et al.)
التعليقات: Comment in: Nature. 2021 Dec;600(7889):368-369. (PMID: 34887584)
References: Nature. 2011 Oct 02;478(7370):529-33. (PMID: 21964340)
Cell. 2011 Oct 14;147(2):344-57. (PMID: 22000013)
Sci Rep. 2013 Nov 06;3:3146. (PMID: 24192909)
JAMA. 1990 Mar 9;263(10):1420-3. (PMID: 2406475)
PLoS One. 2017 Sep 13;12(9):e0183591. (PMID: 28902887)
Nature. 2012 Aug 23;488(7412):522-6. (PMID: 22854781)
Nat Med. 2011 Feb;17(2):211-5. (PMID: 21240262)
Cell Metab. 2016 Sep 13;24(3):348-360. (PMID: 27626191)
Science. 2015 Aug 28;349(6251):aac4716. (PMID: 26315443)
Cell. 2011 Sep 16;146(6):904-17. (PMID: 21889194)
Science. 2010 May 21;328(5981):1031-5. (PMID: 20378772)
Nature. 2012 Mar 28;483(7391):531-3. (PMID: 22460880)
Elife. 2016 Jul 07;5:. (PMID: 27387362)
Perspect Psychol Sci. 2012 Nov;7(6):562-71. (PMID: 26168113)
Circ Res. 2015 Jan 2;116(1):116-26. (PMID: 25552691)
Elife. 2015 Sep 03;4:. (PMID: 26335297)
Elife. 2017 Jun 27;6:. (PMID: 28653617)
Lancet. 2014 Jan 18;383(9913):267-76. (PMID: 24411647)
Elife. 2017 Jan 19;6:. (PMID: 28100397)
R Soc Open Sci. 2016 Sep 21;3(9):160384. (PMID: 27703703)
Scientometrics. 2016;108:315-328. (PMID: 27390387)
Nature. 2012 Jul 26;487(7408):500-4. (PMID: 22763439)
PLoS Biol. 2011 Dec;9(12):e1001195. (PMID: 22162946)
Elife. 2015 Jul 31;4:e06434. (PMID: 26231042)
Elife. 2019 Jun 19;8:. (PMID: 31215867)
Science. 2016 Mar 4;351(6277):1024-6. (PMID: 26941302)
Nature. 2011 Nov 20;481(7381):380-4. (PMID: 22101433)
Nature. 2010 Jun 24;465(7301):1033-8. (PMID: 20577206)
PLoS Med. 2005 Aug;2(8):e124. (PMID: 16060722)
Elife. 2016 Mar 11;5:e10860. (PMID: 26971564)
J Empir Res Hum Res Ethics. 2007 Dec;2(4):3-14. (PMID: 19385804)
Elife. 2014 Dec 10;3:. (PMID: 25490933)
Cell. 2010 Apr 2;141(1):69-80. (PMID: 20371346)
Cell. 2011 Jul 8;146(1):148-63. (PMID: 21729786)
Nature. 2011 Nov 09;479(7374):547-51. (PMID: 22080947)
Elife. 2021 Dec 07;10:. (PMID: 34874009)
PLoS Biol. 2021 Mar 1;19(3):e3001107. (PMID: 33647013)
PLoS One. 2009 Nov 30;4(11):e7824. (PMID: 19956596)
Genome Res. 2012 Feb;22(2):299-306. (PMID: 22009989)
Lancet. 2014 Jan 11;383(9912):156-65. (PMID: 24411644)
Elife. 2015 Jan 26;4:. (PMID: 25621566)
Nature. 2010 Jan 21;463(7279):318-25. (PMID: 20032975)
Elife. 2020 Oct 19;9:. (PMID: 33073769)
Cancer Cell. 2010 Mar 16;17(3):225-34. (PMID: 20171147)
Nature. 2011 Oct 05;478(7368):197-203. (PMID: 21976023)
Elife. 2017 Jan 19;6:. (PMID: 28100400)
Nature. 2010 Aug 12;466(7308):869-73. (PMID: 20668451)
Nature. 2012 Oct 11;490(7419):187-91. (PMID: 23060188)
Elife. 2016 Feb 16;5:. (PMID: 26882073)
Nature. 2012 Jan 26;483(7387):100-3. (PMID: 22281684)
Elife. 2018 Jan 09;7:. (PMID: 29313488)
Nature. 2010 Dec 16;468(7326):973-7. (PMID: 21107323)
Prev Sci. 2011 Jun;12(2):103-17. (PMID: 21541692)
Nat Hum Behav. 2021 Aug;5(8):990-997. (PMID: 34168323)
Nature. 2012 May 09;485(7399):502-6. (PMID: 22622578)
Sci Data. 2016 Mar 15;3:160018. (PMID: 26978244)
PLoS One. 2013 Jun 21;8(6):e67111. (PMID: 23805293)
Nat Hum Behav. 2018 Sep;2(9):637-644. (PMID: 31346273)
Nature. 2014 Mar 27;507(7493):423-5. (PMID: 24678540)
Perspect Psychol Sci. 2012 Nov;7(6):632-8. (PMID: 26168122)
Nature. 2011 Jul 06;475(7354):106-9. (PMID: 21734707)
PLoS Biol. 2019 May 1;17(5):e3000246. (PMID: 31042704)
Proc Natl Acad Sci U S A. 2018 Mar 13;115(11):2600-2606. (PMID: 29531091)
PLoS Negl Trop Dis. 2015 Nov 20;9(11):e0004194. (PMID: 26587586)
Psychol Sci. 2012 May 1;23(5):524-32. (PMID: 22508865)
Nat Cell Biol. 2010 May;12(5):468-76. (PMID: 20418870)
Elife. 2019 Dec 17;8:. (PMID: 31845647)
Elife. 2016 Feb 26;5:. (PMID: 26943899)
Nature. 2010 Dec 9;468(7325):839-43. (PMID: 21057493)
Nature. 2011 Jul 13;475(7355):231-4. (PMID: 21753854)
Nature. 2010 Apr 15;464(7291):1071-6. (PMID: 20393566)
Nature. 2011 Sep 07;477(7363):179-84. (PMID: 21901007)
Circ Res. 2017 Jun 9;120(12):1916-1926. (PMID: 28373349)
Perspect Psychol Sci. 2012 Nov;7(6):537-42. (PMID: 26168110)
Lancet. 2014 Jan 11;383(9912):166-75. (PMID: 24411645)
Cell. 2012 Sep 28;151(1):56-67. (PMID: 23021215)
Cell. 2012 May 11;149(4):780-94. (PMID: 22579283)
Nature. 2010 Dec 9;468(7325):824-8. (PMID: 21102434)
Elife. 2016 Feb 19;5:. (PMID: 26894955)
Nature. 2011 Aug 03;478(7370):524-8. (PMID: 21814200)
Cancer Cell. 2010 Dec 14;18(6):553-67. (PMID: 21130701)
Elife. 2015 Feb 25;4:. (PMID: 25714925)
Elife. 2018 Jan 09;7:. (PMID: 29313490)
Elife. 2015 Jun 18;4:e06938. (PMID: 26086719)
Nature. 2020 Jul;583(7817):518-520. (PMID: 32694846)
Cell. 2016 Aug 25;166(5):1059-1060. (PMID: 27565332)
Infect Immun. 2012 Mar;80(3):891-6. (PMID: 22184414)
Proc Natl Acad Sci U S A. 2011 May 10;108(19):7950-5. (PMID: 21498687)
Elife. 2018 Oct 08;7:. (PMID: 30295289)
BMJ Open Sci. 2020 Aug 27;4(1):e100051. (PMID: 35047690)
Elife. 2015 Jan 26;4:. (PMID: 25621564)
Elife. 2016 Jan 29;5:e07383. (PMID: 26826285)
Nature. 2011 Dec 14;481(7380):190-4. (PMID: 22170610)
Elife. 2016 Feb 17;5:. (PMID: 26885666)
Elife. 2020 Apr 21;9:. (PMID: 32314732)
Psychol Bull. 2009 Jul;135(4):555-88. (PMID: 19586162)
Psychol Sci. 2011 Nov;22(11):1359-66. (PMID: 22006061)
Pers Soc Psychol Rev. 1998;2(3):196-217. (PMID: 15647155)
Nature. 2010 Mar 18;464(7287):427-30. (PMID: 20179705)
Nature. 2010 Dec 16;468(7326):968-72. (PMID: 21107320)
Science. 2012 Aug 10;337(6095):730-5. (PMID: 22855427)
Sci Eng Ethics. 2015 Feb;21(1):127-37. (PMID: 24668038)
Elife. 2016 Mar 01;5:. (PMID: 26943900)
Bull Med Libr Assoc. 1999 Oct;87(4):437-43. (PMID: 10550028)
Cell. 2011 Oct 14;147(2):370-81. (PMID: 22000015)
Elife. 2016 Feb 23;5:. (PMID: 26905833)
PLoS One. 2010 Apr 07;5(4):e10068. (PMID: 20383332)
Elife. 2017 Jun 27;6:. (PMID: 28653623)
Cell. 2010 Jan 22;140(2):209-21. (PMID: 20141835)
Elife. 2017 Jan 19;6:. (PMID: 28100392)
Nature. 2011 Jun 08;475(7355):222-5. (PMID: 21654748)
Cancer Cell. 2011 Jan 18;19(1):17-30. (PMID: 21251613)
Trends Cogn Sci. 2019 Oct;23(10):815-818. (PMID: 31421987)
Nat Rev Drug Discov. 2011 Aug 31;10(9):712. (PMID: 21892149)
PLoS Biol. 2014 Jan;12(1):e1001756. (PMID: 24409096)
Elife. 2021 Dec 07;10:. (PMID: 34874005)
Elife. 2017 Jan 19;6:. (PMID: 28100395)
Elife. 2015 Sep 01;4:. (PMID: 26327698)
Nature. 2015 Jun 11;522(7555):151-3. (PMID: 26062496)
Elife. 2015 May 05;4:e06847. (PMID: 25939392)
Nature. 2010 Mar 18;464(7287):431-5. (PMID: 20130576)
Elife. 2018 Mar 13;7:. (PMID: 29533182)
Nature. 2012 Aug 23;488(7412):527-30. (PMID: 22854777)
Elife. 2014 Dec 10;3:. (PMID: 25490934)
Elife. 2016 Mar 21;5:. (PMID: 26999821)
Elife. 2015 Aug 19;4:. (PMID: 26287525)
Elife. 2015 Jul 16;4:e04796. (PMID: 26179155)
Nat Med. 2012 Jun;18(6):883-91. (PMID: 22635005)
Elife. 2017 Jan 19;6:. (PMID: 28100394)
Elife. 2019 Mar 12;8:. (PMID: 30860027)
Lancet. 2014 Jan 11;383(9912):101-4. (PMID: 24411643)
Sci Transl Med. 2011 Aug 17;3(96):96ra77. (PMID: 21849665)
Nature. 2012 Jul 26;487(7408):505-9. (PMID: 22763448)
Elife. 2018 Dec 11;7:. (PMID: 30526855)
Nature. 2011 Dec 07;481(7379):85-9. (PMID: 22158103)
Nature. 2012 Mar 28;483(7391):570-5. (PMID: 22460902)
Nature. 2020 Mar;579(7798):190-192. (PMID: 32157231)
Elife. 2015 Jul 31;4:e07420. (PMID: 26231040)
Nature. 2012 Jan 18;482(7383):53-8. (PMID: 22258507)
Elife. 2014 Dec 10;3:. (PMID: 25490932)
Nature. 2011 Aug 18;476(7360):346-50. (PMID: 21760589)
PLoS One. 2015 Aug 05;10(8):e0132382. (PMID: 26244868)
Elife. 2015 Jun 25;4:e07072. (PMID: 26111384)
Perspect Psychol Sci. 2012 Nov;7(6):615-31. (PMID: 26168121)
Proc Natl Acad Sci U S A. 2012 Apr 24;109(17):6662-7. (PMID: 22451913)
Elife. 2016 Feb 11;5:. (PMID: 26882501)
Nat Genet. 2010 Feb;42(2):181-5. (PMID: 20081860)
Nature. 2019 Sep;573(7773):187-189. (PMID: 31506624)
Elife. 2014 Dec 10;3:. (PMID: 25490935)
Elife. 2015 Jan 26;4:. (PMID: 25621565)
Elife. 2015 May 22;4:. (PMID: 27879198)
Science. 2012 Oct 5;338(6103):120-3. (PMID: 22903521)
Elife. 2015 Mar 17;4:. (PMID: 25781343)
Science. 2015 Jun 26;348(6242):1422-5. (PMID: 26113702)
Proc Natl Acad Sci U S A. 2021 Apr 27;118(17):. (PMID: 33893240)
PLoS Biol. 2020 Mar 27;18(3):e3000691. (PMID: 32218571)
Nature. 2012 Feb 15;483(7390):474-8. (PMID: 22343901)
Psychol Bull. 1990 Nov;108(3):480-98. (PMID: 2270237)
Exp Neurol. 2012 Feb;233(2):597-605. (PMID: 22078756)
BMC Med. 2008 Jun 16;6:13. (PMID: 18558004)
Elife. 2016 Jun 23;5:. (PMID: 27336789)
Science. 2016 Mar 25;351(6280):1433-6. (PMID: 26940865)
فهرسة مساهمة: Keywords: Reproducibility Project: Cancer Biology; cancer biology; human; mouse; open data; open science; preregistration; replication; reproducibility
تواريخ الأحداث: Date Created: 20211207 Date Completed: 20211215 Latest Revision: 20230615
رمز التحديث: 20230615
مُعرف محوري في PubMed: PMC8651289
DOI: 10.7554/eLife.67995
PMID: 34874008
قاعدة البيانات: MEDLINE
الوصف
تدمد:2050-084X
DOI:10.7554/eLife.67995