تقرير
SpikeZIP-TF: Conversion is All You Need for Transformer-based SNN
| العنوان: | SpikeZIP-TF: Conversion is All You Need for Transformer-based SNN |
|---|---|
| المؤلفون: | You, Kang, Xu, Zekai, Nie, Chen, Deng, Zhijie, Guo, Qinghai, Wang, Xiang, He, Zhezhi |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Neural and Evolutionary Computing, Computer Science - Artificial Intelligence |
| الوصف: | Spiking neural network (SNN) has attracted great attention due to its characteristic of high efficiency and accuracy. Currently, the ANN-to-SNN conversion methods can obtain ANN on-par accuracy SNN with ultra-low latency (8 time-steps) in CNN structure on computer vision (CV) tasks. However, as Transformer-based networks have achieved prevailing precision on both CV and natural language processing (NLP), the Transformer-based SNNs are still encounting the lower accuracy w.r.t the ANN counterparts. In this work, we introduce a novel ANN-to-SNN conversion method called SpikeZIP-TF, where ANN and SNN are exactly equivalent, thus incurring no accuracy degradation. SpikeZIP-TF achieves 83.82% accuracy on CV dataset (ImageNet) and 93.79% accuracy on NLP dataset (SST-2), which are higher than SOTA Transformer-based SNNs. The code is available in GitHub: https://github.com/Intelligent-Computing-Research-Group/SpikeZIP_transformer Comment: * These authors contributed equally to this work |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2406.03470 |
| رقم الأكسشن: | edsarx.2406.03470 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |