An illusion of predictability in scientific results: Even experts confuse inferential uncertainty and outcome variability.

التفاصيل البيبلوغرافية
العنوان:	An illusion of predictability in scientific results: Even experts confuse inferential uncertainty and outcome variability.
المؤلفون:	Zhang S; Department of Applied Mathematics, University of Colorado, Boulder, CO 80309., Heck PR; Office of Research, Consumer Financial Protection Bureau, Washington, DC 20552., Meyer MN; Department of Bioethics & Decision Sciences, Geisinger Health System, Danville, PA 17822., Chabris CF; Department of Bioethics & Decision Sciences, Geisinger Health System, Danville, PA 17822., Goldstein DG; Microsoft Research, New York, NY 10012., Hofman JM; Microsoft Research, New York, NY 10012.
المصدر:	Proceedings of the National Academy of Sciences of the United States of America [Proc Natl Acad Sci U S A] 2023 Aug 15; Vol. 120 (33), pp. e2302491120. Date of Electronic Publication: 2023 Aug 09.
نوع المنشور:	Journal Article
اللغة:	English
بيانات الدورية:	Publisher: National Academy of Sciences Country of Publication: United States NLM ID: 7505876 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1091-6490 (Electronic) Linking ISSN: 00278424 NLM ISO Abbreviation: Proc Natl Acad Sci U S A Subsets: PubMed not MEDLINE; MEDLINE
أسماء مطبوعة:	Original Publication: Washington, DC : National Academy of Sciences
مستخلص:	Traditionally, scientists have placed more emphasis on communicating inferential uncertainty (i.e., the precision of statistical estimates) compared to outcome variability (i.e., the predictability of individual outcomes). Here, we show that this can lead to sizable misperceptions about the implications of scientific results. Specifically, we present three preregistered, randomized experiments where participants saw the same scientific findings visualized as showing only inferential uncertainty, only outcome variability, or both and answered questions about the size and importance of findings they were shown. Our results, composed of responses from medical professionals, professional data scientists, and tenure-track faculty, show that the prevalent form of visualizing only inferential uncertainty can lead to significant overestimates of treatment effects, even among highly trained experts. In contrast, we find that depicting both inferential uncertainty and outcome variability leads to more accurate perceptions of results while appearing to leave other subjective impressions of the results unchanged, on average.
References:	PLoS Biol. 2015 Apr 22;13(4):e1002128. (PMID: 25901488) IEEE Trans Vis Comput Graph. 2014 Dec;20(12):2142-51. (PMID: 26356928) Br J Anaesth. 2003 Apr;90(4):514-6. (PMID: 12644429) Br Med J (Clin Res Ed). 1986 Mar 15;292(6522):746-50. (PMID: 3082422) Psychol Methods. 2005 Dec;10(4):389-96. (PMID: 16392994) Nat Methods. 2019 Jul;16(7):565-566. (PMID: 31217592) PLoS One. 2015 Nov 16;10(11):e0142444. (PMID: 26571487) J Pers Soc Psychol. 2000 Apr;78(4):772-90. (PMID: 10794380) Cortex. 2013 Mar;49(3):609-10. (PMID: 23347556) Wellcome Open Res. 2019 Apr 1;4:63. (PMID: 31069261) Nature. 2021 Jul;595(7866):181-188. (PMID: 34194044) Psychon Bull Rev. 2014 Oct;21(5):1157-64. (PMID: 24420726) Nat Methods. 2013 Oct;10(10):921-2. (PMID: 24161969) Nat Methods. 2014 Feb;11(2):119-20. (PMID: 24645192) Am Psychol. 2005 Feb-Mar;60(2):170-80. (PMID: 15740449) Psychol Methods. 2008 Mar;13(1):19-30. (PMID: 18331151) Psychon Bull Rev. 2012 Aug;19(4):601-7. (PMID: 22648655) J Cell Biol. 2007 Apr 9;177(1):7-11. (PMID: 17420288) IEEE Trans Vis Comput Graph. 2021 Feb;27(2):272-282. (PMID: 33048681)
معلومات مُعتمدة:	DGE 2040434 National Science Foundation (NSF)
فهرسة مساهمة:	Keywords: experiments; science communication; statistics; uncertainty; visualization
تواريخ الأحداث:	Date Created: 20230809 Latest Revision: 20230823
رمز التحديث:	20231215
مُعرف محوري في PubMed:	PMC10438372
DOI:	10.1073/pnas.2302491120
PMID:	37556500
قاعدة البيانات:	MEDLINE

الوصف
تدمد:	1091-6490
DOI:	10.1073/pnas.2302491120