دورية أكاديمية

PneumoLLM: Harnessing the power of large language model for pneumoconiosis diagnosis.

التفاصيل البيبلوغرافية
العنوان: PneumoLLM: Harnessing the power of large language model for pneumoconiosis diagnosis.
المؤلفون: Song M; Institute of Basic Medical Sciences Chinese Academy of Medical Sciences, School of Basic Medicine Peking Union Medical College, Beijing, 100005, China; State Key Laboratory of Respiratory Health and Multimorbidity, Beijing, 100005, China., Wang J; School of Automation, Northwestern Polytechnical University, Shaanxi, Xi'an 710072, China., Yu Z; Jinneng Holding Coal Industry Group Co. Ltd Occupational Disease Precaution Clinic, Shanxi, 037001, China., Wang J; School of Medicine, Tsinghua University, Beijing, 100084, China., Yang L; School of Electronics and Control Engineering, Chang'an University, Shaanxi, Xi'an 710064, China., Lu Y; School of Automation, Northwestern Polytechnical University, Shaanxi, Xi'an 710072, China., Li B; Center of Respiratory Medicine, China-Japan Friendship Hospital, National Center for Respiratory Medicine, Institute of Respiratory Medicine, Chinese Academy of Medical Sciences, National Clinical Research Center for Respiratory Diseases, Beijing, 100020, China., Wang X; Department of Respiratory, the Second Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang, 150086, China; Internal Medicine, Harbin Medical University, Harbin, Heilongjiang, 150081, China., Wang X; School of Automation, Northwestern Polytechnical University, Shaanxi, Xi'an 710072, China., Huang Q; School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi'an 710072, China., Li Z; Translational Research Center, Shanghai YangZhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), Shanghai 201619, China; School of Mechanical Engineering, Tongji University, Shanghai 201804, China., Kanellakis NI; Laboratory of Pleural and Lung Cancer Translational Research, CAMS Oxford Institute, Nuffield Department of Medicine, University of Oxford, Oxford, UK; Oxford Centre for Respiratory Medicine, Churchill Hospital, Oxford University Hospitals NHS Foundation Trust, Oxford, UK; National Institute for Health Research Oxford Biomedical Research Centre, University of Oxford, Oxford, UK., Liu J; Institute of Basic Medical Sciences Chinese Academy of Medical Sciences, School of Basic Medicine Peking Union Medical College, Beijing, 100005, China; Plastic Surgery Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100144, China; State Key Laboratory of Common Mechanism Research for Major Diseases, Beijing, 100005, China. Electronic address: ljf@pumc.edu.cn., Wang J; Institute of Basic Medical Sciences Chinese Academy of Medical Sciences, School of Basic Medicine Peking Union Medical College, Beijing, 100005, China; State Key Laboratory of Respiratory Health and Multimorbidity, Beijing, 100005, China. Electronic address: wangjing@ibms.pumc.edu.cn., Wang B; School of Automation, Northwestern Polytechnical University, Shaanxi, Xi'an 710072, China. Electronic address: wbl921129@gmail.com., Yang J; Institute of Basic Medical Sciences Chinese Academy of Medical Sciences, School of Basic Medicine Peking Union Medical College, Beijing, 100005, China; Plastic Surgery Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100144, China; State Key Laboratory of Common Mechanism Research for Major Diseases, Beijing, 100005, China.
المصدر: Medical image analysis [Med Image Anal] 2024 Jun 20; Vol. 97, pp. 103248. Date of Electronic Publication: 2024 Jun 20.
Publication Model: Ahead of Print
نوع المنشور: Journal Article
اللغة: English
بيانات الدورية: Publisher: Elsevier Country of Publication: Netherlands NLM ID: 9713490 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1361-8423 (Electronic) Linking ISSN: 13618415 NLM ISO Abbreviation: Med Image Anal Subsets: MEDLINE
أسماء مطبوعة: Publication: Amsterdam : Elsevier
Original Publication: London : Oxford University Press, [1996-
مستخلص: The conventional pretraining-and-finetuning paradigm, while effective for common diseases with ample data, faces challenges in diagnosing data-scarce occupational diseases like pneumoconiosis. Recently, large language models (LLMs) have exhibits unprecedented ability when conducting multiple tasks in dialogue, bringing opportunities to diagnosis. A common strategy might involve using adapter layers for vision-language alignment and diagnosis in a dialogic manner. Yet, this approach often requires optimization of extensive learnable parameters in the text branch and the dialogue head, potentially diminishing the LLMs' efficacy, especially with limited training data. In our work, we innovate by eliminating the text branch and substituting the dialogue head with a classification head. This approach presents a more effective method for harnessing LLMs in diagnosis with fewer learnable parameters. Furthermore, to balance the retention of detailed image information with progression towards accurate diagnosis, we introduce the contextual multi-token engine. This engine is specialized in adaptively generating diagnostic tokens. Additionally, we propose the information emitter module, which unidirectionally emits information from image tokens to diagnosis tokens. Comprehensive experiments validate the superiority of our methods.
Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
(Copyright © 2024 Elsevier B.V. All rights reserved.)
فهرسة مساهمة: Keywords: Foundational model; Large language model; Medical image diagnosis
تواريخ الأحداث: Date Created: 20240628 Latest Revision: 20240628
رمز التحديث: 20240629
DOI: 10.1016/j.media.2024.103248
PMID: 38941859
قاعدة البيانات: MEDLINE
الوصف
تدمد:1361-8423
DOI:10.1016/j.media.2024.103248