دورية أكاديمية
أعرض في EDS

Reinforcement Motor Learning After Cerebellar Damage Is Related to State Estimation.

التفاصيل البيبلوغرافية
العنوان: Reinforcement Motor Learning After Cerebellar Damage Is Related to State Estimation.
المؤلفون: White CM; Moss Rehabilitation Research Institute, Medical Arts Building, Suite 100, 50 Township Line Rd, Elkins Park, PA, USA., Snow EC; Moss Rehabilitation Research Institute, Medical Arts Building, Suite 100, 50 Township Line Rd, Elkins Park, PA, USA., Therrien AS; Moss Rehabilitation Research Institute, Medical Arts Building, Suite 100, 50 Township Line Rd, Elkins Park, PA, USA. Amanda.Therrien@jefferson.edu.; Department of Rehabilitation Medicine, Thomas Jefferson University, Philadelphia, PA, USA. Amanda.Therrien@jefferson.edu.
المصدر: Cerebellum (London, England) [Cerebellum] 2024 Jun; Vol. 23 (3), pp. 1061-1073. Date of Electronic Publication: 2023 Oct 12.
نوع المنشور: Journal Article
اللغة: English
بيانات الدورية: Publisher: Springer Country of Publication: United States NLM ID: 101089443 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1473-4230 (Electronic) Linking ISSN: 14734222 NLM ISO Abbreviation: Cerebellum Subsets: MEDLINE
أسماء مطبوعة: Publication: <2006->: New York : Springer
Original Publication: London : Martin Dunitz, c2002-
مواضيع طبية MeSH: Reinforcement, Psychology*, Humans ; Male ; Female ; Middle Aged ; Aged ; Adult ; Cerebellum/physiopathology ; Cerebellum/physiology ; Learning/physiology ; Psychomotor Performance/physiology ; Movement/physiology ; Cerebellar Diseases/physiopathology
مستخلص: Recent work showed that individuals with cerebellar degeneration could leverage intact reinforcement learning (RL) to alter their movement. However, there was marked inter-individual variability in learning, and the factors underlying it were unclear. Cerebellum-dependent sensory prediction may contribute to RL in motor contexts by enhancing body state estimates, which are necessary to solve the credit-assignment problem. The objective of this study was to test the relationship between the predictive component of state estimation and RL in individuals with cerebellar degeneration. Individuals with cerebellar degeneration and neurotypical control participants completed two tasks: an RL task that required them to alter the angle of reaching movements and a state estimation task that tested the somatosensory perception of active and passive movement. The state estimation task permitted the calculation of the active benefit shown by each participant, which is thought to reflect the cerebellum-dependent predictive component of state estimation. We found that the cerebellar and control groups showed similar magnitudes of learning with reinforcement and active benefit on average, but there was substantial variability across individuals. Using multiple regression, we assessed potential predictors of RL. Our analysis included active benefit, somatosensory acuity, clinical ataxia severity, movement variability, movement speed, and age. We found a significant relationship in which greater active benefit predicted better learning with reinforcement in the cerebellar, but not the control group. No other variables showed significant relationships with learning. Overall, our results support the hypothesis that the integrity of sensory prediction is a strong predictor of RL after cerebellar damage.
(© 2023. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.)
References: Glickstein M, Strata P, Voogd J. Cerebellum: history. Neuroscience. 2009;162:549–59. (PMID: 1927242610.1016/j.neuroscience.2009.02.054)
Wolpert DM, Miall RC, Kawato M. Internal models in the cerebellum. Trends Cogn Sci. 1998;2:338–47. (PMID: 2122723010.1016/S1364-6613(98)01221-2)
Wolpert DM, Ghahramani Z. Computational principles of movement neuroscience. Nat Neurosci. 2000;3:1212–7. (PMID: 1112784010.1038/81497)
Krakauer JW, Pine ZM, Ghilardi MF, Ghez C. Learning of visuomotor transformations for vectorial planning of reaching trajectories. J Neurosci. 2000;20:8916–24. (PMID: 11102502677309410.1523/JNEUROSCI.20-23-08916.2000)
Shadmehr R, Mussa-Ivaldi FA. Adaptive representation of dynamics during learning of a motor task. J Neurosci. 1994;14:3208–24. (PMID: 8182467657749210.1523/JNEUROSCI.14-05-03208.1994)
Bastian AJ, Martin TA, Keating JG, Thach WT. Cerebellar ataxia: abnormal control of interaction torques across multiple joints. J Neurophysiol. 1996;76:492–509. (PMID: 883623910.1152/jn.1996.76.1.492)
Day BL, Thompson PD, Harding AE, Marsden CD. Influence of vision on upper limb reaching movements in patients with cerebellar ataxia. Brain. 1998;121(Pt 2):357–72. (PMID: 954951110.1093/brain/121.2.357)
Bhanpuri NH, Okamura AM, Bastian AJ. Predicting and correcting ataxia using a model of cerebellar function. Brain. 2014;137:1931–44. (PMID: 24812203406502110.1093/brain/awu115)
Weiner MJ, Hallett M, Funkenstein HH. Adaptation to lateral displacement of vision in patients with lesions of the central nervous system. Neurology. 1983;33:766–72. (PMID: 668252010.1212/WNL.33.6.766)
Martin TA, Keating JG, Goodkin HP, Bastian AJ, Thach WT. Throwing while looking through prisms: I. Focal olivocerebellar lesions impair adaptation. Brain. 1996;119:1183–98. (PMID: 881328210.1093/brain/119.4.1183)
Maschke M, Gomez CM, Ebner TJ, Konczak J. Hereditary cerebellar ataxia progressively impairs force adaptation during goal-directed arm movements. J Neurophysiol. 2004;91:230–8. (PMID: 1367940310.1152/jn.00557.2003)
Morton SM, Bastian AJ. Prism adaptation during walking generalizes to reaching and requires the cerebellum. J Neurophysiol. 2004;92:2497–509. (PMID: 1519008810.1152/jn.00129.2004)
Smith MA, Shadmehr R. Intact ability to learn internal models of arm dynamics in Huntington’s disease but not cerebellar degeneration. J Neurophysiol. 2005;93:2809–21. (PMID: 1562509410.1152/jn.00943.2004)
Tseng Y, Diedrichsen J, Krakauer JW, Shadmehr R, Bastian AJ. Sensory prediction errors drive cerebellum-dependent adaptation of reaching. J Neurophysiol. 2007;98:54–62. (PMID: 1750750410.1152/jn.00266.2007)
Izawa J, Criscimagna-Hemminger SE, Shadmehr R. Cerebellar contributions to reach adaptation and learning sensory consequences of action. J Neurosci. 2012;32:4230–9. (PMID: 22442085332658410.1523/JNEUROSCI.6353-11.2012)
Schlerf JE, Xu J, Klemfuss NM, Griffiths TL, Ivry RB. Individuals with cerebellar degeneration show similar adaptation deficits with large and small visuomotor errors. J Neurophysiol. 2013;109:1164–73. (PMID: 2319745010.1152/jn.00654.2011)
Ilg W, Bastian AJ, Boesch S, Burciu RG, Celnik P, Claaßen J, et al. Consensus paper: management of degenerative cerebellar disorders. Cerebellum. 2014;13:248–68. (PMID: 24222635434412610.1007/s12311-013-0531-6)
Morton SM, Bastian AJ. Can rehabilitation help ataxia? Neurology. 2009;73:1818–9. (PMID: 1986463510.1212/WNL.0b013e3181c33b21)
Ilg W, Timmann D. Gait ataxia—specific cerebellar influences and their rehabilitation. Mov Disord. 2013;28:1566–75. (PMID: 2413284510.1002/mds.25558)
Haith AM, Krakauer JW. Model-based and model-free mechanisms of human motor learning. Adv Exp Med Biol. 2013;782:1–21. (PMID: 23296478357016510.1007/978-1-4614-5465-6_1)
Krakauer JW, Hadjiosif AM, Xu J, Wong AL, Haith AM. Motor learning. Compr Physiol. 2019;9:613–63. (PMID: 3087358310.1002/cphy.c170043)
Therrien AS, Wolpert DM, Bastian AJ. Effective reinforcement learning following cerebellar damage requires a balance between exploration and motor noise. Brain. 2016;139:101–14. (PMID: 2662636810.1093/brain/awv329)
McDougle SD, Boggess MJ, Crossley MJ, Parvin D, Ivry RB, Taylor JA. Credit assignment in movement-dependent reinforcement learning. PNAS. 2016;113:6797–802. (PMID: 27247404491417910.1073/pnas.1523669113)
Uehara S, Mawase F, Celnik P. Learning similar actions by reinforcement or sensory-prediction errors rely on distinct physiological mechanisms. Cereb Cortex. 2018;28:3478–90. (PMID: 2896882710.1093/cercor/bhx214)
Niv Y. Reinforcement learning in the brain. J Math Psychol. 2009;53:139–54. (PMID: 10.1016/j.jmp.2008.12.005)
Sutton RS, Barto AG. Reinforcement learning: an introduction. Cambridge, Massachusetts: The MIT Press; 1998.
Therrien AS, Statton MA, Bastian AJ. Reinforcement signaling can be used to reduce elements of cerebellar reaching ataxia. Cerebellum. 2021;20:62–73. (PMID: 32880848792797710.1007/s12311-020-01183-x)
Taylor JA, Ivry RB. Cerebellar and prefrontal cortex contributions to adaptation, strategies, and reinforcement learning. In: Ramnani N, editor. Prog Brain Res. Elsevier; 2014. p. 217–53.
Stengel RF. Optimal control and estimation. Dover Publications; 1994.
Wolpert DM, Ghahramani Z, Jordan MI. An internal model for sensorimotor integration. Science. 1995;269:1880–2. (PMID: 756993110.1126/science.7569931)
Paillard J, Brouchon M. A proprioceptive contribution to the spatial encoding of position cues for ballistic movements. Brain Res. 1974;71:273–84. (PMID: 446806310.1016/0006-8993(74)90971-8)
Adamovich SV, Berkinblit MB, Fookson O, Poizner H. Pointing in 3D space to remembered targets. I. Kinesthetic versus visual target presentation. J Neurophysiol. 1998;79:2833–46. (PMID: 963609010.1152/jn.1998.79.6.2833)
Fuentes CT, Bastian AJ. Where is your arm? Variations in proprioception across space and tasks. J Neurophysiol. 2010;103:164–71. (PMID: 1986444110.1152/jn.00494.2009)
Bhanpuri NH, Okamura AM, Bastian AJ. Active force perception depends on cerebellar function. J Neurophysiol. 2012;107:1612–20. (PMID: 2219062010.1152/jn.00983.2011)
Bhanpuri NH, Okamura AM, Bastian AJ. Predictive modeling by the cerebellum improves proprioception. J Neurosci. 2013;33:14301–6. (PMID: 24005283376104410.1523/JNEUROSCI.0784-13.2013)
Weeks HM, Therrien AS, Bastian AJ. Proprioceptive localization deficits in people with cerebellar damage. Cerebellum. 2017;16:427–37. (PMID: 27538404560968310.1007/s12311-016-0819-4)
Weeks HM, Therrien AS, Bastian AJ. The cerebellum contributes to proprioception during motion. J Neurophysiol. 2017;118:693–702. (PMID: 28404825553943710.1152/jn.00417.2016)
Zimmet AM, Cao D, Bastian AJ, Cowan NJ. Cerebellar patients have intact feedback control that can be leveraged to improve reaching. Elife. 2020;9:e53246. (PMID: 33025903757773510.7554/eLife.53246)
Holland P, Codol O, Galea JM. Contribution of explicit processes to reinforcement-based motor learning. J Neurophysiol. 2018;119:2241–55. (PMID: 29537918603211510.1152/jn.00901.2017)
Codol O, Holland PJ, Galea JM. The relationship between reinforcement and explicit control during visuomotor adaptation. Sci Rep. 2018;8:9121. (PMID: 29904096600252410.1038/s41598-018-27378-1)
Butcher PA, Ivry RB, Kuo S-H, Rydz D, Krakauer JW, Taylor JA. The cerebellum does more than sensory prediction error-based learning in sensorimotor adaptation tasks. J Neurophysiol. 2017;118:1622–36. (PMID: 28637818559611910.1152/jn.00451.2017)
Tsay JS, Schuck L, Ivry RB. Cerebellar degeneration impairs strategy discovery but not strategy recall. Cerebellum. 2022; Online ahead of print; https://doi.org/10.1007/s12311-022-01500-6 .
Wong AL, Marvel CL, Taylor JA, Krakauer JW. Can patients with cerebellar disease switch learning mechanisms to reduce their adaptation deficits? Brain. 2019;142:662–73. (PMID: 30689760639165110.1093/brain/awy334)
Brand M, Schiebener J. Interactions of age and cognitive functions in predicting decision making under risky conditions over the life span. J Clin Exp Neuropsychol. 2013;35:9–23. (PMID: 2313686910.1080/13803395.2012.740000)
Eppinger B, Nystrom LE, Cohen JD. Reduced sensitivity to immediate reward during decision-making in older than younger adults. PLoS One. 2012;7:e36953. (PMID: 22655032335999610.1371/journal.pone.0036953)
Weller JA, King ML, Figner B, Denburg NL. Information use in risky decision making: do age differences depend on affective context? Psychol Aging. 2019;34:1005–20. (PMID: 31580088747349310.1037/pag0000397)
Trouillas P, Takayanagi T, Hallett M, Currier RD, Subramony SH, Wessel K, et al. International Cooperative Ataxia Rating Scale for pharmacological assessment of the cerebellar syndrome. J Neurol Sci. 1997;145:205–11. (PMID: 909405010.1016/S0022-510X(96)00231-6)
Gibo TL, Criscimagna-Hemminger SE, Okamura AM, Bastian AJ. Cerebellar motor learning: are environment dynamics more important than error size? J Neurophysiol. 2013;110:322–33. (PMID: 23596337372706910.1152/jn.00745.2012)
Wichmann FA, Hill NJ. The psychometric function: I. Fitting, sampling, and goodness of fit. Percept Psychophys. 2001;63:1293–313. (PMID: 1180045810.3758/BF03194544)
Mawase F, Uehara S, Bastian AJ, Celnik P. Motor learning enhances use-dependent plasticity. J Neurosci. 2017;37:2673–85. (PMID: 28143961535432110.1523/JNEUROSCI.3303-16.2017)
van Mastrigt NM, Tsay JS, Wang T, Avraham G, Abram SJ, van der Kooij K, et al. Implicit reward-based motor learning. Exp Brain Res. 2023;.
Forano M, Franklin DW. Reward actively engages both implicit and explicit components in dual force field adaptation. bioRxiv. 2023:2023.08.09.552587.
Vassiliadis P, Derosiere G, Duque J. Beyond motor noise: considering other causes of impaired reinforcement learning in cerebellar patients. eNeuro. 2019;6 ENEURO.0458-18.2019.
Hull C. Prediction signals in the cerebellum: beyond supervised motor learning. eLife. 2020;9:e54073. (PMID: 32223891710537610.7554/eLife.54073)
Yoshida J, Oñate M, Khatami L, Vera J, Nadim F, Khodakhah K. Cerebellar contributions to the basal ganglia influence motor coordination, reward processing, and movement vigor. J Neurosci. 2022;42:8406–15. (PMID: 36351826966592110.1523/JNEUROSCI.1535-22.2022)
Sendhilnathan N, Ipata A, Goldberg ME. Mid-lateral cerebellar complex spikes encode multiple independent reward-related signals during reinforcement learning. Nat Commun. 2021;12:6475. (PMID: 34753927857862110.1038/s41467-021-26338-0)
Larry N, Yarkoni M, Lixenberg A, Joshua M. Cerebellar climbing fibers encode expected reward size. Elife. 2019;8:e46870. (PMID: 31661073684464410.7554/eLife.46870)
Thoma P, Bellebaum C, Koch B, Schwarz M, Daum I. The cerebellum is involved in reward-based reversal learning. Cerebellum. 2008;7:433–43. (PMID: 1859233110.1007/s12311-008-0046-8)
Cardoso CD, Branco LD, Cotrena C, Kristensen CH, Schneider Bakos DD, Fonseca RP. The impact of frontal and cerebellar lesions on decision making: evidence from the Iowa Gambling Task. Front Neurosci. 2014;8:61. (PMID: 24782697398659210.3389/fnins.2014.00061)
Nicholas J, Amlang C, Lin C-YR, Montaser-Kouhsari L, Desai N, Pan M-K, et al. The role of the cerebellum in learning to predict reward: evidence from cerebellar ataxia. bioRxiv. 2022:2022.11.04.515251.
Lixenberg A, Yarkoni M, Botschko Y, Joshua M. Encoding of eye movements explains reward-related activity in cerebellar simple spikes. J Neurophysiol. 2020;123:786–99. (PMID: 31940216705263110.1152/jn.00363.2019)
Powell DJ, Marder E, Nusbaum MP. Perturbation-specific responses by two neural circuits generating similar activity patterns. Curr Biol. 2021;31:4831–4838.e4. (PMID: 34506730857840710.1016/j.cub.2021.08.042)
Izawa J, Shadmehr R. Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput Biol. 2011;7:e1002012. (PMID: 21423711305331310.1371/journal.pcbi.1002012)
Cashaback JGA, McGregor HR, Mohatarem A, Gribble PL. Dissociating error-based and reinforcement-based loss functions during sensorimotor learning. PLoS Comput Biol. 2017;13:e1005623. (PMID: 28753634555001110.1371/journal.pcbi.1005623)
فهرسة مساهمة: Keywords: Cerebellum; Degeneration; Internal model; Reaching; Somatosensation
تواريخ الأحداث: Date Created: 20231012 Date Completed: 20240518 Latest Revision: 20240518
رمز التحديث: 20240518
DOI: 10.1007/s12311-023-01615-4
PMID: 37828231
قاعدة البيانات: MEDLINE
الوصف
تدمد:1473-4230
DOI:10.1007/s12311-023-01615-4