دورية أكاديمية
Towards biologically plausible model-based reinforcement learning in recurrent spiking networks by dreaming new experiences.
| العنوان: | Towards biologically plausible model-based reinforcement learning in recurrent spiking networks by dreaming new experiences. |
|---|---|
| المؤلفون: | Capone C; INFN, Sezione di Roma, Rome, RM, 00185, Italy. cristiano0capone@gmail.com.; Natl. Center for Radiation Protection and Computational Physics, Istituto Superiore di Sanitá, Rome, 00161, Italy. cristiano0capone@gmail.com., Paolucci PS; INFN, Sezione di Roma, Rome, RM, 00185, Italy. |
| المصدر: | Scientific reports [Sci Rep] 2024 Jun 25; Vol. 14 (1), pp. 14656. Date of Electronic Publication: 2024 Jun 25. |
| نوع المنشور: | Journal Article |
| اللغة: | English |
| بيانات الدورية: | Publisher: Nature Publishing Group Country of Publication: England NLM ID: 101563288 Publication Model: Electronic Cited Medium: Internet ISSN: 2045-2322 (Electronic) Linking ISSN: 20452322 NLM ISO Abbreviation: Sci Rep Subsets: MEDLINE |
| أسماء مطبوعة: | Original Publication: London : Nature Publishing Group, copyright 2011- |
| مواضيع طبية MeSH: | Models, Neurological* , Reinforcement, Psychology* , Neural Networks, Computer*, Humans ; Action Potentials/physiology ; Neurons/physiology ; Algorithms ; Learning/physiology ; Nerve Net/physiology ; Animals ; Computer Simulation |
| مستخلص: | Humans and animals can learn new skills after practicing for a few hours, while current reinforcement learning algorithms require a large amount of data to achieve good performances. Recent model-based approaches show promising results by reducing the number of necessary interactions with the environment to learn a desirable policy. However, these methods require biological implausible ingredients, such as the detailed storage of older experiences, and long periods of offline learning. The optimal way to learn and exploit world-models is still an open question. Taking inspiration from biology, we suggest that dreaming might be an efficient expedient to use an inner model. We propose a two-module (agent and model) spiking neural network in which "dreaming" (living new experiences in a model-based simulated environment) significantly boosts learning. Importantly, our model does not require the detailed storage of experiences, and learns online the world-model and the policy. Moreover, we stress that our network is composed of spiking neurons, further increasing the biological plausibility and implementability in neuromorphic hardware. (© 2024. The Author(s).) |
| References: | Front Neurorobot. 2023 Jan 20;16:1075647. (PMID: 36742191) Neural Comput. 2007 Jun;19(6):1468-502. (PMID: 17444757) PLoS Comput Biol. 2013 Apr;9(4):e1003024. (PMID: 23592970) PLoS Comput Biol. 2021 Jun 28;17(6):e1009045. (PMID: 34181642) Neuron. 2018 Mar 21;97(6):1244-1252.e5. (PMID: 29503184) Sci Rep. 2019 Jun 20;9(1):8990. (PMID: 31222151) PLoS Comput Biol. 2018 Jul 9;14(7):e1006322. (PMID: 29985966) Int J Neural Syst. 2020 Jun;30(6):2050027. (PMID: 32466691) PLoS Comput Biol. 2022 Mar 24;18(3):e1009753. (PMID: 35324886) Nature. 2020 Dec;588(7839):604-609. (PMID: 33361790) Psychol Bull. 2010 Jan;136(1):65-86. (PMID: 20063926) J Neurosci. 2016 Apr 13;36(15):4231-47. (PMID: 27076422) Philos Trans A Math Phys Eng Sci. 2023 Jul 24;381(2251):20220050. (PMID: 37271169) PLoS Comput Biol. 2022 Jun 21;18(6):e1010221. (PMID: 35727852) Neural Netw. 2019 Dec;120:108-115. (PMID: 31500931) Nat Commun. 2020 Jul 17;11(1):3625. (PMID: 32681001) |
| تواريخ الأحداث: | Date Created: 20240625 Date Completed: 20240626 Latest Revision: 20240905 |
| رمز التحديث: | 20240905 |
| مُعرف محوري في PubMed: | PMC11199658 |
| DOI: | 10.1038/s41598-024-65631-y |
| PMID: | 38918553 |
| قاعدة البيانات: | MEDLINE |
Find this article in full text from Springer Verlag. 
Full Text Finder | تدمد: | 2045-2322 |
|---|---|
| DOI: | 10.1038/s41598-024-65631-y |