دورية أكاديمية
Differentiable channel pruning guided via attention mechanism: a novel neural network pruning approach
| العنوان: | Differentiable channel pruning guided via attention mechanism: a novel neural network pruning approach |
|---|---|
| المؤلفون: | Hanjing Cheng, Zidong Wang, Lifeng Ma, Zhihui Wei, Fawaz E. Alsaadi, Xiaohui Liu |
| المصدر: | Complex & Intelligent Systems, Vol 9, Iss 5, Pp 5611-5624 (2023) |
| بيانات النشر: | Springer, 2023. |
| سنة النشر: | 2023 |
| المجموعة: | LCC:Electronic computers. Computer science LCC:Information technology |
| مصطلحات موضوعية: | Artificial Intelligence, Network pruning, Neural Architecture Search, Gumbel-softmax sampling, Attention mechanism, Electronic computers. Computer science, QA75.5-76.95, Information technology, T58.5-58.64 |
| الوصف: | Abstract Neural network pruning offers great prospects for facilitating the deployment of deep neural networks on computational resource limited devices. Neural architecture search (NAS) provides an efficient way to automatically seek appropriate neural architecture design for compressed model. It is observed that, for existing NAS-based pruning methods, there is usually a lack of layer information when searching the optimal neural architecture. In this paper, we propose a new NAS approach, namely, differentiable channel pruning method guided via attention mechanism (DCP-A), where the adopted attention mechanism is able to provide layer information to guide the optimization of the pruning policy. The training process is differentiable with Gumbel-softmax sampling, while parameters are optimized under a two-stage training procedure. The neural network block with the shortcut is dedicatedly designed, which is of help to prune the network not only on its width but also on its depth. Extensive experiments are performed to verify the applicability and superiority of the proposed method. Detailed analysis with visualization of the pruned model architecture shows that our proposed DCP-A learns explainable pruning policies. |
| نوع الوثيقة: | article |
| وصف الملف: | electronic resource |
| اللغة: | English |
| تدمد: | 2199-4536 2198-6053 |
| Relation: | https://doaj.org/toc/2199-4536; https://doaj.org/toc/2198-6053 |
| DOI: | 10.1007/s40747-023-01022-6 |
| URL الوصول: | https://doaj.org/article/0e1a561cccc342078b6a510601df9ce1 |
| رقم الأكسشن: | edsdoj.0e1a561cccc342078b6a510601df9ce1 |
| قاعدة البيانات: | Directory of Open Access Journals |
Find this article in full text from Springer Verlag. 
Full Text Finder | تدمد: | 21994536 21986053 |
|---|---|
| DOI: | 10.1007/s40747-023-01022-6 |