دورية أكاديمية
أعرض في EDS

Quantifying Interrater Agreement and Reliability Between Thoracic Pathologists: Paradoxical Behavior of Cohen's Kappa in the Presence of a High Prevalence of the Histopathologic Feature in Lung Cancer.

التفاصيل البيبلوغرافية
العنوان: Quantifying Interrater Agreement and Reliability Between Thoracic Pathologists: Paradoxical Behavior of Cohen's Kappa in the Presence of a High Prevalence of the Histopathologic Feature in Lung Cancer.
المؤلفون: Tan KS; Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, New York., Yeh YC; Department of Pathology and Laboratory Medicine, Taipei Veterans General Hospital, Taipei, Taiwan., Adusumilli PS; Thoracic Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, New York., Travis WD; Department of Pathology and Laboratory Medicine, Memorial Sloan Kettering Cancer Center, New York, New York.
المصدر: JTO clinical and research reports [JTO Clin Res Rep] 2023 Dec 16; Vol. 5 (1), pp. 100618. Date of Electronic Publication: 2023 Dec 16 (Print Publication: 2024).
نوع المنشور: Journal Article
اللغة: English
بيانات الدورية: Publisher: Elsevier Inc Country of Publication: United States NLM ID: 101769967 Publication Model: eCollection Cited Medium: Internet ISSN: 2666-3643 (Electronic) Linking ISSN: 26663643 NLM ISO Abbreviation: JTO Clin Res Rep Subsets: PubMed not MEDLINE
أسماء مطبوعة: Publication: [New York] : Elsevier Inc., [2020]-
مستخلص: Introduction: Cohen's kappa is often used to quantify the agreement between two pathologists. Nevertheless, a high prevalence of the feature of interest can lead to seemingly paradoxical results, such as low Cohen's kappa values despite high "observed agreement." Here, we investigate Cohen's kappa using data from histologic subtyping assessment of lung adenocarcinomas and introduce alternative measures that can overcome this "kappa paradox."
Methods: A total of 50 frozen sections from stage I lung adenocarcinomas less than or equal to 3 cm in size were independently reviewed by two pathologists to determine the absence or presence of five histologic patterns (lepidic, papillary, acinar, micropapillary, solid). For each pattern, observed agreement (proportion of cases with concordant "absent" or "present" ratings) and Cohen's kappa were calculated, along with Gwet's AC1.
Results: The prevalence of any amount of the histologic patterns ranged from 42% (solid) to 97% (acinar). On the basis of Cohen's kappa, there was substantial agreement for four of the five patterns (lepidic, 0.65; papillary, 0.67; micropapillary, 0.64; solid, 0.61). Acinar had the lowest Cohen's kappa (0.43, moderate agreement), despite having the highest observed agreement (88%). In contrast, Gwet's AC1 values were close to or higher than Cohen's kappa across patterns (lepidic, 0.64; papillary, 0.69; micropapillary, 0.71; solid, 0.73; acinar, 0.85). The proportion of positive versus negative agreement was 93% versus 50% for acinar.
Conclusions: Given the dependence of Cohen's kappa on feature prevalence, interrater agreement studies should include complementary indices such as Gwet's AC1 and proportions of specific agreement, especially in settings with a high prevalence of the feature of interest.
(© 2024 The Authors.)
References: BMC Med Res Methodol. 2013 Jul 29;13:97. (PMID: 23890315)
J Clin Epidemiol. 2005 Jul;58(7):655-61. (PMID: 15939215)
Br J Math Stat Psychol. 2008 May;61(Pt 1):29-48. (PMID: 18482474)
J Clin Epidemiol. 2006 Oct;59(10):1033-9. (PMID: 16980142)
J Clin Epidemiol. 1993 May;46(5):423-9. (PMID: 8501467)
Psychol Bull. 1968 Oct;70(4):213-20. (PMID: 19673146)
Biometrics. 1977 Mar;33(1):159-74. (PMID: 843571)
Proc Am Thorac Soc. 2011 Sep;8(5):381-5. (PMID: 21926387)
J Clin Epidemiol. 1990;43(6):543-9. (PMID: 2348207)
J Clin Epidemiol. 1990;43(6):551-8. (PMID: 2189948)
J Chronic Dis. 1987;40(2):171-8. (PMID: 3818871)
BMC Med Res Methodol. 2013 Apr 29;13:61. (PMID: 23627889)
MethodsX. 2023 May 10;10:102212. (PMID: 37234937)
Histopathology. 2015 Jun;66(7):922-38. (PMID: 24889415)
J Clin Epidemiol. 1988;41(10):959-68. (PMID: 3193139)
J Clin Epidemiol. 2000 May;53(5):499-503. (PMID: 10812322)
Biometrics. 1990 Jun;46(2):293-302. (PMID: 2364122)
Int J Nurs Stud. 2011 Jun;48(6):661-71. (PMID: 21514934)
J Clin Epidemiol. 2011 Jun;64(6):701-2; author reply 702. (PMID: 21411278)
معلومات مُعتمدة: P30 CA008748 United States CA NCI NIH HHS; R01 CA235667 United States CA NCI NIH HHS; R01 CA236615 United States CA NCI NIH HHS
فهرسة مساهمة: Keywords: Diagnostic accuracy; Interobserver coefficient; Performance metrics; Predominant histologic subtypes; Reproducibility; Sensitivity and specificity
تواريخ الأحداث: Date Created: 20240129 Latest Revision: 20240201
رمز التحديث: 20240201
مُعرف محوري في PubMed: PMC10820331
DOI: 10.1016/j.jtocrr.2023.100618
PMID: 38283651
قاعدة البيانات: MEDLINE
الوصف
تدمد:2666-3643
DOI:10.1016/j.jtocrr.2023.100618