تقرير
Deep Learning without Weight Symmetry
| العنوان: | Deep Learning without Weight Symmetry |
|---|---|
| المؤلفون: | Ji-An, Li, Benna, Marcus K. |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science Quantitative Biology |
| مصطلحات موضوعية: | Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Quantitative Biology - Neurons and Cognition |
| الوصف: | Backpropagation (BP), a foundational algorithm for training artificial neural networks, predominates in contemporary deep learning. Although highly successful, it is often considered biologically implausible. A significant limitation arises from the need for precise symmetry between connections in the backward and forward pathways to backpropagate gradient signals accurately, which is not observed in biological brains. Researchers have proposed several algorithms to alleviate this symmetry constraint, such as feedback alignment and direct feedback alignment. However, their divergence from backpropagation dynamics presents challenges, particularly in deeper networks and convolutional layers. Here we introduce the Product Feedback Alignment (PFA) algorithm. Our findings demonstrate that PFA closely approximates BP and achieves comparable performance in deep convolutional networks while avoiding explicit weight symmetry. Our results offer a novel solution to the longstanding weight symmetry problem, leading to more biologically plausible learning in deep convolutional networks compared to earlier methods. |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2405.20594 |
| رقم الأكسشن: | edsarx.2405.20594 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |