تقرير
Semantic segmentation of surgical hyperspectral images under geometric domain shifts
| العنوان: | Semantic segmentation of surgical hyperspectral images under geometric domain shifts |
|---|---|
| المؤلفون: | Sellner, Jan, Seidlitz, Silvia, Studier-Fischer, Alexander, Motta, Alessandro, Özdemir, Berkin, Müller-Stich, Beat Peter, Nickel, Felix, Maier-Hein, Lena |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Electrical Engineering and Systems Science - Image and Video Processing, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning, I.2.10, I.4.6, J.3 |
| الوصف: | Robust semantic segmentation of intraoperative image data could pave the way for automatic surgical scene understanding and autonomous robotic surgery. Geometric domain shifts, however, although common in real-world open surgeries due to variations in surgical procedures or situs occlusions, remain a topic largely unaddressed in the field. To address this gap in the literature, we (1) present the first analysis of state-of-the-art (SOA) semantic segmentation networks in the presence of geometric out-of-distribution (OOD) data, and (2) address generalizability with a dedicated augmentation technique termed "Organ Transplantation" that we adapted from the general computer vision community. According to a comprehensive validation on six different OOD data sets comprising 600 RGB and hyperspectral imaging (HSI) cubes from 33 pigs semantically annotated with 19 classes, we demonstrate a large performance drop of SOA organ segmentation networks applied to geometric OOD data. Surprisingly, this holds true not only for conventional RGB data (drop of Dice similarity coefficient (DSC) by 46 %) but also for HSI data (drop by 45 %), despite the latter's rich information content per pixel. Using our augmentation scheme improves on the SOA DSC by up to 67 % (RGB) and 90 % (HSI) and renders performance on par with in-distribution performance on real OOD test data. The simplicity and effectiveness of our augmentation scheme makes it a valuable network-independent tool for addressing geometric domain shifts in semantic scene segmentation of intraoperative data. Our code and pre-trained models are available at https://github.com/IMSY-DKFZ/htc. Comment: The first two authors (Jan Sellner and Silvia Seidlitz) contributed equally to this paper |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2303.10972 |
| رقم الأكسشن: | edsarx.2303.10972 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |