تقرير
Give Me Your Attention: Dot-Product Attention Considered Harmful for Adversarial Patch Robustness
العنوان: | Give Me Your Attention: Dot-Product Attention Considered Harmful for Adversarial Patch Robustness |
---|---|
المؤلفون: | Lovisotto, Giulio, Finnie, Nicole, Munoz, Mauricio, Mummadi, Chaithanya Kumar, Metzen, Jan Hendrik |
سنة النشر: | 2022 |
المجموعة: | Computer Science |
مصطلحات موضوعية: | Computer Science - Computer Vision and Pattern Recognition, 68T07, I.4 |
الوصف: | Neural architectures based on attention such as vision transformers are revolutionizing image recognition. Their main benefit is that attention allows reasoning about all parts of a scene jointly. In this paper, we show how the global reasoning of (scaled) dot-product attention can be the source of a major vulnerability when confronted with adversarial patch attacks. We provide a theoretical understanding of this vulnerability and relate it to an adversary's ability to misdirect the attention of all queries to a single key token under the control of the adversarial patch. We propose novel adversarial objectives for crafting adversarial patches which target this vulnerability explicitly. We show the effectiveness of the proposed patch attacks on popular image classification (ViTs and DeiTs) and object detection models (DETR). We find that adversarial patches occupying 0.5% of the input can lead to robust accuracies as low as 0% for ViT on ImageNet, and reduce the mAP of DETR on MS COCO to less than 3%. Comment: to be published in IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022, CVPR22 |
نوع الوثيقة: | Working Paper |
URL الوصول: | http://arxiv.org/abs/2203.13639 |
رقم الأكسشن: | edsarx.2203.13639 |
قاعدة البيانات: | arXiv |
الوصف غير متاح. |