التفاصيل البيبلوغرافية
| العنوان: |
IQAGPT: computed tomography image quality assessment with vision-language and ChatGPT models. |
| المؤلفون: |
Chen, Zhihao, Hu, Bin, Niu, Chuang, Chen, Tao, Li, Yuxin, Shan, Hongming, Wang, Ge |
| المصدر: |
Visual Computing for Industry, Biomedicine & Art; 8/5/2024, Vol. 7 Issue 1, p1-17, 17p |
| مصطلحات موضوعية: |
LANGUAGE models, CHATGPT, GENERATIVE pre-trained transformers, COMPUTED tomography, COMPUTER-assisted image analysis (Medicine) |
| مستخلص: |
Large language models (LLMs), such as ChatGPT, have demonstrated impressive capabilities in various tasks and attracted increasing interest as a natural language interface across many domains. Recently, large vision-language models (VLMs) that learn rich vision–language correlation from image–text pairs, like BLIP-2 and GPT-4, have been intensively investigated. However, despite these developments, the application of LLMs and VLMs in image quality assessment (IQA), particularly in medical imaging, remains unexplored. This is valuable for objective performance evaluation and potential supplement or even replacement of radiologists' opinions. To this end, this study introduces IQAGPT, an innovative computed tomography (CT) IQA system that integrates image-quality captioning VLM with ChatGPT to generate quality scores and textual reports. First, a CT-IQA dataset comprising 1,000 CT slices with diverse quality levels is professionally annotated and compiled for training and evaluation. To better leverage the capabilities of LLMs, the annotated quality scores are converted into semantically rich text descriptions using a prompt template. Second, the image-quality captioning VLM is fine-tuned on the CT-IQA dataset to generate quality descriptions. The captioning model fuses image and text features through cross-modal attention. Third, based on the quality descriptions, users verbally request ChatGPT to rate image-quality scores or produce radiological quality reports. Results demonstrate the feasibility of assessing image quality using LLMs. The proposed IQAGPT outperformed GPT-4 and CLIP-IQA, as well as multitask classification and regression models that solely rely on images. [ABSTRACT FROM AUTHOR] |
|
Copyright of Visual Computing for Industry, Biomedicine & Art is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) |
| قاعدة البيانات: |
Complementary Index |
Find this article in full text from Springer Verlag. 
Full Text Finder