دورية أكاديمية
أعرض في EDS

Unsupervised Adaptive Weight Pruning for Energy-Efficient Neuromorphic Systems.

التفاصيل البيبلوغرافية
العنوان: Unsupervised Adaptive Weight Pruning for Energy-Efficient Neuromorphic Systems.
المؤلفون: Guo W; Sensors Lab, Advanced Membranes & Porous Materials Center, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia.; Communication and Computing Systems Lab, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia., Fouda ME; Department of Electrical Engineering and Computer Science, University of California, Irvine, Irvine, CA, United States., Yantir HE; Sensors Lab, Advanced Membranes & Porous Materials Center, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia.; Communication and Computing Systems Lab, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia., Eltawil AM; Communication and Computing Systems Lab, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia.; Department of Electrical Engineering and Computer Science, University of California, Irvine, Irvine, CA, United States., Salama KN; Sensors Lab, Advanced Membranes & Porous Materials Center, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia.
المصدر: Frontiers in neuroscience [Front Neurosci] 2020 Nov 12; Vol. 14, pp. 598876. Date of Electronic Publication: 2020 Nov 12 (Print Publication: 2020).
نوع المنشور: Journal Article
اللغة: English
بيانات الدورية: Publisher: Frontiers Research Foundation Country of Publication: Switzerland NLM ID: 101478481 Publication Model: eCollection Cited Medium: Print ISSN: 1662-4548 (Print) Linking ISSN: 1662453X NLM ISO Abbreviation: Front Neurosci Subsets: PubMed not MEDLINE
أسماء مطبوعة: Original Publication: Lausanne : Frontiers Research Foundation
مستخلص: To tackle real-world challenges, deep and complex neural networks are generally used with a massive number of parameters, which require large memory size, extensive computational operations, and high energy consumption in neuromorphic hardware systems. In this work, we propose an unsupervised online adaptive weight pruning method that dynamically removes non-critical weights from a spiking neural network (SNN) to reduce network complexity and improve energy efficiency. The adaptive pruning method explores neural dynamics and firing activity of SNNs and adapts the pruning threshold over time and neurons during training. The proposed adaptation scheme allows the network to effectively identify critical weights associated with each neuron by changing the pruning threshold dynamically over time and neurons. It balances the connection strength of neurons with the previous layer with adaptive thresholds and prevents weak neurons from failure after pruning. We also evaluated improvement in the energy efficiency of SNNs with our method by computing synaptic operations (SOPs). Simulation results and detailed analyses have revealed that applying adaptation in the pruning threshold can significantly improve network performance and reduce the number of SOPs. The pruned SNN with 800 excitatory neurons can achieve a 30% reduction in SOPs during training and a 55% reduction during inference, with only 0.44% accuracy loss on MNIST dataset. Compared with a previously reported online soft pruning method, the proposed adaptive pruning method shows 3.33% higher classification accuracy and 67% more reduction in SOPs. The effectiveness of our method was confirmed on different datasets and for different network sizes. Our evaluation showed that the implementation overhead of the adaptive method regarding speed, area, and energy is negligible in the network. Therefore, this work offers a promising solution for effective network compression and building highly energy-efficient neuromorphic systems in real-time applications.
(Copyright © 2020 Guo, Fouda, Yantir, Eltawil and Salama.)
References: Front Neurosci. 2020 Nov 12;14:598876. (PMID: 33281549)
Front Neurosci. 2019 Apr 26;13:405. (PMID: 31080402)
Front Comput Neurosci. 2015 Aug 03;9:99. (PMID: 26941637)
J Neurosci. 2006 Sep 20;26(38):9673-82. (PMID: 16988038)
Biol Cybern. 2006 Jul;95(1):1-19. (PMID: 16622699)
Front Neurosci. 2018 Dec 03;12:891. (PMID: 30559644)
Biosystems. 2005 Jan-Mar;79(1-3):11-20. (PMID: 15649585)
Science. 2014 Aug 8;345(6197):668-73. (PMID: 25104385)
IEEE Trans Pattern Anal Mach Intell. 2020 Mar;42(3):568-579. (PMID: 30561340)
PLoS Comput Biol. 2014 Apr 10;10(4):e1003560. (PMID: 24722397)
Front Neurosci. 2013 Oct 08;7:178. (PMID: 24115919)
فهرسة مساهمة: Keywords: STDP; neuromorphic computing; pattern recognition; pruning; spiking neural networks; unsupervised learning
تواريخ الأحداث: Date Created: 20201207 Latest Revision: 20240330
رمز التحديث: 20240330
مُعرف محوري في PubMed: PMC7689062
DOI: 10.3389/fnins.2020.598876
PMID: 33281549
قاعدة البيانات: MEDLINE
الوصف
تدمد:1662-4548
DOI:10.3389/fnins.2020.598876