دورية أكاديمية
Hippocampal spatio-predictive cognitive maps adaptively guide reward generalization.
| العنوان: | Hippocampal spatio-predictive cognitive maps adaptively guide reward generalization. |
|---|---|
| المؤلفون: | Garvert MM; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany. mona.garvert@gmail.com.; Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Berlin, Germany. mona.garvert@gmail.com.; Max Planck Institute for Biological Cybernetics, Tübingen, Germany. mona.garvert@gmail.com., Saanum T; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin, Germany., Schulz E; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin, Germany., Schuck NW; Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Berlin, Germany.; Max Planck Institute for Biological Cybernetics, Tübingen, Germany.; Institute of Psychology, Universität Hamburg, Hamburg, Germany., Doeller CF; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany. doeller@cbs.mpg.de.; Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Jebsen Centre for Alzheimer's Disease NTNU, Trondheim, Norway. doeller@cbs.mpg.de.; Wilhelm Wundt Institute of Psychology, Leipzig University, Leipzig, Germany. doeller@cbs.mpg.de. |
| المصدر: | Nature neuroscience [Nat Neurosci] 2023 Apr; Vol. 26 (4), pp. 615-626. Date of Electronic Publication: 2023 Apr 03. |
| نوع المنشور: | Journal Article; Research Support, Non-U.S. Gov't |
| اللغة: | English |
| بيانات الدورية: | Publisher: Nature Publishing Group Country of Publication: United States NLM ID: 9809671 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1546-1726 (Electronic) Linking ISSN: 10976256 NLM ISO Abbreviation: Nat Neurosci Subsets: MEDLINE |
| أسماء مطبوعة: | Publication: <2002->: New York, NY : Nature Publishing Group Original Publication: New York, NY : Nature America Inc., c1998- |
| مواضيع طبية MeSH: | Reward* , Brain*, Humans ; Hippocampus ; Generalization, Psychological ; Cognition |
| مستخلص: | The brain forms cognitive maps of relational knowledge-an organizing principle thought to underlie our ability to generalize and make inferences. However, how can a relevant map be selected in situations where a stimulus is embedded in multiple relational structures? Here, we find that both spatial and predictive cognitive maps influence generalization in a choice task, where spatial location determines reward magnitude. Mirroring behavior, the hippocampus not only builds a map of spatial relationships but also encodes the experienced transition structure. As the task progresses, participants' choices become more influenced by spatial relationships, reflected in a strengthening of the spatial map and a weakening of the predictive map. This change is driven by orbitofrontal cortex, which represents the degree to which an outcome is consistent with the spatial rather than the predictive map and updates hippocampal representations accordingly. Taken together, this demonstrates how hippocampal cognitive maps are used and updated flexibly for inference. (© 2023. The Author(s).) |
| References: | Shepard, R. N. Toward a universal law of generalization for psychological science. Science 237, 1317–1323 (1987). (PMID: 362924310.1126/science.3629243) Gershman, S. J. & Daw, N. D. Reinforcement learning and episodic memory in humans and animals: an integrative framework. Annu. Rev. Psychol. 68, 101–128 (2017). (PMID: 2761894410.1146/annurev-psych-122414-033625) Guttman, N. & Kalish, H. I. Discriminability and stimulus generalization. J. Exp. Psychol. 51, 79 (1956). (PMID: 1328644410.1037/h0046219) Kahnt, T. & Tobler, P. N. Dopamine regulates stimulus generalization in the human hippocampus. eLife 5, e12678 (2016). (PMID: 26830462475574710.7554/eLife.12678) Wu, C. M., Schulz, E., Garvert, M. M., Meder, B. & Schuck, N. W. Similarities and differences in spatial and non-spatial cognitive maps. PLoS Comput. Biol. 16, e1008149 (2020). (PMID: 32903264748087510.1371/journal.pcbi.1008149) Barron, H. C. et al. Neuronal computation underlying inferential reasoning in humans and mice. Cell 183, 228–243 (2020). (PMID: 32946810711614810.1016/j.cell.2020.08.035) Brogden, W. J. Sensory pre-conditioning. J. Exp. Psychol. 25, 323 (1939). (PMID: 10.1037/h0058944) Baram, A. B., Muller, T. H., Nili, H., Garvert, M. M. & Behrens, T. E. J. Entorhinal and ventromedial prefrontal cortices abstract and generalize the structure of reinforcement learning problems. Neuron 109, 713–723 (2021). (PMID: 33357385788949610.1016/j.neuron.2020.11.024) Wimmer, G. E., Daw, N. D. & Shohamy, D. Generalization of value in reinforcement learning by humans. Eur. J. Neurosci. 35, 1092–1104 (2012). (PMID: 22487039340461810.1111/j.1460-9568.2012.08017.x) Morgan, L. K., MacEvoy, S. P., Aguirre, G. K. & Epstein, R. A. Distances between real-world locations are represented in the human hippocampus. J. Neurosci. 31, 1238–1245 (2011). (PMID: 21273408307427610.1523/JNEUROSCI.4667-10.2011) O’Keefe, J. & Nadel, L. The Hippocampus as a Cognitive Map (Clarendon Press, 1978). Tolman, E. C. Cognitive maps in rats and men. Psychol. Rev. 55, 189 (1948). (PMID: 1887087610.1037/h0061626) Constantinescu, A. O., O’Reilly, J. X. & Behrens, T. E. Organizing conceptual knowledge in humans with a gridlike code. Science 352, 1464–1468 (2016). (PMID: 27313047524897210.1126/science.aaf0941) Aronov, D., Nevers, R. & Tank, D. W. Mapping of a non-spatial dimension by the hippocampal-entorhinal circuit. Nature 543, 719–722 (2017). (PMID: 28358077549251410.1038/nature21692) Nau, M., Navarro Schröder, T., Bellmund, J. L. S. & Doeller, C. F. Hexadirectional coding of visual space in human entorhinal cortex. Nat. Neurosci. 21, 188–190 (2018). (PMID: 2931174610.1038/s41593-017-0050-8) Theves, S., Fernández, G. & Doeller, C. F. The hippocampus maps concept space, not feature space. J. Neurosci. 40, 7318–7325 (2020). (PMID: 32826311753491410.1523/JNEUROSCI.0494-20.2020) Theves, S., Fernandez, G. & Doeller, C. F. The hippocampus encodes distances in multidimensional feature space. Curr. Biol. 29, 1226–1231.e3 (2019). (PMID: 3090560210.1016/j.cub.2019.02.035) Deuker, L., Bellmund, J., Schröder, T. N. & Doeller, C. An event map of memory space in the hippocampus. eLife 5, e16534 (2016). (PMID: 27710766505380710.7554/eLife.16534) Bellmund, J. L. S., Polti, I. & Doeller, C. F. Sequence memory in the hippocampal-entorhinal region. J. Cogn. Neurosci. 32, 2056–2070 (2020). (PMID: 3253037810.1162/jocn_a_01592) Eichenbaum, H. Time cells in the hippocampus: a new dimension for mapping memories. Nat. Rev. Neurosci. 15, 732–744 (2014). (PMID: 25269553434809010.1038/nrn3827) Schapiro, A. C., Turk-Browne, N. B., Norman, K. A. & Botvinick, M. M. Statistical learning of temporal community structure in the hippocampus. Hippocampus 26, 3–8 (2016). (PMID: 2633266610.1002/hipo.22523) Garvert, M. M., Dolan, R. J. & Behrens, T. E. A map of abstract relational knowledge in the human hippocampal–entorhinal cortex. eLife 6, e17086 (2017). (PMID: 28448253540785510.7554/eLife.17086) Schuck, N. W., Cai, M. B., Wilson, R. C. & Niv, Y. Human orbitofrontal cortex represents a cognitive map of state space. Neuron 91, 1402–1412 (2016). (PMID: 27657452504487310.1016/j.neuron.2016.08.019) Schapiro, A. C., Rogers, T. T., Cordova, N. I., Turk-Browne, N. B. & Botvinick, M. M. Neural representations of events arise from temporal community structure. Nat. Neurosci. 16, 486–492 (2013). (PMID: 23416451374982310.1038/nn.3331) Schapiro, A. C., Kustner, L. V. & Turk-Browne, N. B. Shaping of object representations in the human medial temporal lobe based on temporal regularities. Curr. Biol. 22, 1622–1627 (2021). (PMID: 10.1016/j.cub.2012.06.056) Nieh, E. H. et al. Geometry of abstract learned knowledge in the hippocampus. Nature 595, 80–84 (2021). (PMID: 34135512954997910.1038/s41586-021-03652-7) Zheng, X. Y. et al. Parallel cognitive maps for short-term statistical and long-term semantic relationships in the hippocampal formation. Preprint at bioRxiv https://doi.org/10.1101/2022.08.29.505742 (2022). Shahar, N. et al. Credit assignment to state-independent task representations and its relationship with model-based decision making. Proc. Natl Acad. Sci. USA 116, 15871–15876 (2019). (PMID: 31320592668993410.1073/pnas.1821647116) Niv, Y. Learning task-state representations. Nat. Neurosci. 22, 1544–1553 (2019). (PMID: 31551597724131010.1038/s41593-019-0470-8) Wikenheiser, A. M. & Schoenbaum, G. Over the river, through the woods: cognitive maps in the hippocampus and orbitofrontal cortex. Nat. Rev. Neurosci. 17, 513–523 (2016). (PMID: 27256552554125810.1038/nrn.2016.56) Schuck, N. W. & Niv, Y. Sequential replay of nonspatial task states in the human hippocampus. Science 364, eaaw5181 (2019). (PMID: 31249030724131110.1126/science.aaw5181) Wittkuhn, L., Chien, S., Hall-McMaster, S. & Schuck, N. W. Replay in minds and machines. Neurosci. Biobehav. Rev. 129, 367–388 (2021). (PMID: 3437107810.1016/j.neubiorev.2021.08.002) Stachenfeld, K. L., Botvinick, M. M. & Gershman, S. J. The hippocampus as a predictive map. Nat. Neurosci. 20, 1643 (2017). (PMID: 2896791010.1038/nn.4650) Saanum, T., Schulz, E. & Speekenbrink, M. Compositional generalization in multi-armed bandits. Preprint at https://psyarxiv.com/v6mzb/ (2021). Schulz, E., Tenenbaum, J. B., Duvenaud, D., Speekenbrink, M. & Gershman, S. J. Compositional inductive biases in function learning. Cogn. Psychol. 99, 44–79 (2017). (PMID: 2915418710.1016/j.cogpsych.2017.11.002) Gershman, S. J. Uncertainty and exploration. Decision 6, 277–286 (2019). (PMID: 3376812210.1037/dec0000101) Rigoux, L., Stephan, K. E., Friston, K. J. & Daunizeau, J. Bayesian model selection for group studies-revisited. Neuroimage 84, 971–985 (2014). (PMID: 2401830310.1016/j.neuroimage.2013.08.065) Barron, H. C., Garvert, M. M. & Behrens, T. E. Repetition suppression: a means to index neural representations using bold? Philos. Trans. R. Soc. Lond. B Biol. Sci. 371, 20150355 (2016). (PMID: 27574308500385610.1098/rstb.2015.0355) Grill-Spector, K. Selectivity of adaptation in single units: implications for fmri experiments. Neuron 49, 170–171 (2006). (PMID: 1642369010.1016/j.neuron.2006.01.004) Bellmund, J. L. S., Deuker, L., Montijn, N. D. & Doeller, C. F. Mnemonic construction and representation of temporal structure in the hippocampal formation. Nat. Commun. 13, 3395 (2022). (PMID: 35739096922611710.1038/s41467-022-30984-3) Wager, T. D., Davidson, M. L., Hughes, B. L., Lindquist, M. A. & Ochsner, K. N. Prefrontal-subcortical pathways mediating successful emotion regulation. Neuron 59, 1037–1050 (2008). (PMID: 18817740274232010.1016/j.neuron.2008.09.006) Atlas, L. Y., Bolger, N., Lindquist, M. A. & Wager, T. D. Brain mediators of predictive cue effects on perceived pain. J. Neurosci. 30, 12964–12977 (2010). (PMID: 20881115296655810.1523/JNEUROSCI.0057-10.2010) Banerjee, A. et al. Value-guided remapping of sensory cortex by lateral orbitofrontal cortex. Nature 585, 245–250 (2020). (PMID: 3288414610.1038/s41586-020-2704-z) & Takahashi, Y. K. Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci. 14, 1590–1597 (2011). (PMID: 22037501322571810.1038/nn.2957) Schoenbaum, G., Roesch, M. R., Stalnaker, T. A. & Takahashi, Y. K. A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat. Rev. Neurosci. 10, 885–892 (2009). (PMID: 19904278283529910.1038/nrn2753) Howard, L. R. et al. The hippocampus and entorhinal cortex encode the path and Euclidean distances to goals during navigation. Curr. Biol. 24, 1331–1340 (2014). (PMID: 24909328406293810.1016/j.cub.2014.05.001) Chadwick, M. J., Jolly, A. E., Amos, D. P., Hassabis, D. & Spiers, H. J. A goal direction signal in the human entorhinal/subicular region. Curr. Biol. 25, 87–92 (2015). (PMID: 25532898429114410.1016/j.cub.2014.11.001) Schuck, N. W., Wilson, R. & Niv, Y. in Goal-Directed Decision Making (eds Morris, R. et al.) Ch. 12 (Academic Press, 2018). Doeller, C. F., King, J. A. & Burgess, N. Parallel striatal and hippocampal systems for landmarks and boundaries in spatial memory. Proc. Natl Acad. Sci. USA 105, 5915–5920 (2008). (PMID: 18408152231133710.1073/pnas.0801489105) Gallagher, M., McMahan, R. W. & Schoenbaum, G. Orbitofrontal cortex and representation of incentive value in associative learning. J. Neurosci. 19, 6610–6614 (1999). (PMID: 10414988678279110.1523/JNEUROSCI.19-15-06610.1999) Wikenheiser, A. M., Marrero-Garcia, Y. & Schoenbaum, G. Suppression of ventral hippocampal output impairs integrated orbitofrontal encoding of task structure. Neuron 95, 1197–1207.e3 (2017). (PMID: 28823726563755310.1016/j.neuron.2017.08.003) Boorman, E. D., Rajendran, V. G., O’Reilly, J. X. & Behrens, T. E. Two anatomically and computationally distinct learning signals predict changes to stimulus-outcome associations in hippocampus. Neuron 89, 1343–1354 (2016). (PMID: 26948895481944910.1016/j.neuron.2016.02.014) Zhou, J. et al. Evolving schema representations in orbitofrontal ensembles during learning. Nature 590, 606–611 (2021). (PMID: 3336181910.1038/s41586-020-03061-2) Henson, R., Shallice, T. & Dolan, R. Neuroimaging evidence for dissociable forms of repetition priming. Science 287, 1269–1272 (2000). (PMID: 1067883410.1126/science.287.5456.1269) Müller, N. G., Strumpf, H., Scholz, M., Baier, B. & Melloni, L. Repetition suppression versus enhancement—it’s quantity that matters. Cereb. Cortex 23, 315–322 (2012). (PMID: 2231404710.1093/cercor/bhs009) Segaert, K., Weber, K., de Lange, F. P., Petersson, K. M. & Hagoort, P. The suppression of repetition enhancement: a review of fMRI studies. Neuropsychologia 51, 59–66 (2013). (PMID: 2315934410.1016/j.neuropsychologia.2012.11.006) Wissig, S. C. & Kohn, A. The influence of surround suppression on adaptation effects in primary visual cortex. J. Neurophysiol. 107, 3370–3384 (2012). (PMID: 22423001337841110.1152/jn.00739.2011) Turk-Browne, N., Yi, D.-J., Leber, A. & Chun, M. Visual quality determines the direction of neural repetition effects. Cereb. Cortex 17, 425–433 (2006). (PMID: 1656529410.1093/cercor/bhj159) Schlichting, M. L., Mumford, J. A. & Preston, A. R. Learning-related representational changes reveal dissociable integration and separation signatures in the hippocampus and prefrontal cortex. Nat. Commun. 6, 8151 (2015). (PMID: 2630319810.1038/ncomms9151) Favila, S. E., Chanales, A. J. H. & Kuhl, B. A. Experience-dependent hippocampal pattern differentiation prevents interference during subsequent learning. Nat. Commun. 7, 11066 (2016). (PMID: 27925613482083710.1038/ncomms11066) Russek, E. M., Momennejad, I., Botvinick, M. M., Gershman, S. J. & Daw, N. D. Predictive representations can link model-based reinforcement learning to model-free mechanisms. PLoS Comput. Biol. 13, e1005768 (2017). (PMID: 28945743562894010.1371/journal.pcbi.1005768) Kondor, R. & Lafferty, J. D. (2002) Diffusion Kernels on Graphs and Other Discrete Structures. Proceedings of the International Conference on Machine Learning, 315–322. Schulz, E., Franklin, N. T. & Gershman, S. J. Finding structure in multi-armed bandits. Cogn. Psychol. 119, 101261 (2020). (PMID: 3205913310.1016/j.cogpsych.2019.101261) Bates, D., Mächler, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48 (2015). (PMID: 10.18637/jss.v067.i01) Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J. & Friston, K. J. Bayesian model selection for group studies. Neuroimage 46, 1004–1017 (2009). (PMID: 1930693210.1016/j.neuroimage.2009.03.025) Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P. & Dolan, R. J. Model-based influences on humans’ choices and striatal prediction errors. Neuron 69, 1204–1215 (2011). (PMID: 21435563307792610.1016/j.neuron.2011.02.027) Feinberg, D. et al. Multiplexed echo planar imaging for sub-second whole brain fMRI and fast diffusion imaging. PLoS ONE 5, e15710 (2010). (PMID: 21187930300495510.1371/journal.pone.0015710) Moeller, S. et al. Multiband multislice ge-epi at 7 tesla, with 16-fold acceleration using partial parallel imaging with application to high spatial and temporal whole-brain fmri. Magn. Reson. Med. 63, 1144–1153 (2010). (PMID: 20432285290624410.1002/mrm.22361) Esteban, O. et al. fMRIPrep: a robust preprocessing pipeline for functional MRI. Nat. Methods 16, 111–116 (2018). (PMID: 30532080631939310.1038/s41592-018-0235-4) Gorgolewski, K. et al. Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in python. Front. Neuroinform. 5, 13 (2011). (PMID: 21897815315996410.3389/fninf.2011.00013) Gorgolewski, K. et al. (2011). Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in Python. Front. Neuroimform. 5, 13. Tustison, N. J. et al. N4itk: improved n3 bias correction. IEEE Trans. Med. Imaging 29, 1310–1320 (2010). (PMID: 20378467307185510.1109/TMI.2010.2046908) Avants, B., Epstein, C., Grossman, M. & Gee, J. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12, 26–41 (2008). (PMID: 1765999810.1016/j.media.2007.06.004) Zhang, Y., Brady, M. & Smith, S. Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Trans. Med. Imaging 20, 45–57 (2001). (PMID: 1129369110.1109/42.906424) Reuter, M., Rosas, H. D. & Fischl, B. Highly accurate inverse consistent registration: a robust approach. NeuroImage 53, 1181–1196 (2010). (PMID: 2063728910.1016/j.neuroimage.2010.07.020) Dale, A. M., Fischl, B. & Sereno, M. I. Cortical surface-based analysis: I. segmentation and surface reconstruction. NeuroImage 9, 179–194 (1999). (PMID: 993126810.1006/nimg.1998.0395) Klein, A. et al. Mindboggling morphometry of human brains. PLoS Comput. Biol. 13, e1005350 (2017). (PMID: 28231282532288510.1371/journal.pcbi.1005350) Evans, A., Janke, A., Collins, D. & Baillet, S. Brain templates and atlases. NeuroImage 62, 911–922 (2012). (PMID: 2224858010.1016/j.neuroimage.2012.01.024) Glasser, M. F. et al. The minimal preprocessing pipelines for the human connectome project. NeuroImage 80, 105–124 (2013). (PMID: 2366897010.1016/j.neuroimage.2013.04.127) Greve, D. N. & Fischl, B. Accurate and robust brain image alignment using boundary-based registration. NeuroImage 48, 63–72 (2009). (PMID: 1957361110.1016/j.neuroimage.2009.06.060) Jenkinson, M., Bannister, P., Brady, M. & Smith, S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. NeuroImage 17, 825–841 (2002). (PMID: 1237715710.1006/nimg.2002.1132) Cox, R. W. & Hyde, J. S. Software tools for analysis and visualization of fmri data. NMR Biomed. 10, 171–178 (1997). (PMID: 943034410.1002/(SICI)1099-1492(199706/08)10:4/5<171::AID-NBM453>3.0.CO;2-L) Power, J. D. et al. Methods to detect, characterize, and remove motion artifact in resting state fmri. NeuroImage 84, 320–341 (2014). (PMID: 2399431410.1016/j.neuroimage.2013.08.048) Behzadi, Y., Restom, K., Liau, J. & Liu, T. T. A component based noise correction method (CompCor) for BOLD and perfusion based fmri. NeuroImage 37, 90–101 (2007). (PMID: 1756012610.1016/j.neuroimage.2007.04.042) Satterthwaite, T. D. et al. An improved framework for confound regression and filtering for control of motion artifact in the preprocessing of resting-state functional connectivity data. NeuroImage 64, 240–256 (2013). (PMID: 2292629210.1016/j.neuroimage.2012.08.052) Lanczos, C. Evaluation of noisy data. J. Soc. Ind. Appl. Math. Ser. B Numer. Anal. 1, 76–85 (1964). (PMID: 10.1137/0701007) Garvert, M. M., Saanum, T., Schulz, E., Schuck, N. W. & Doeller, C. F. Cognitive maps for novel inference. 10.18112/openneuro.ds004360.v1.0.0 (2022). Saanum, T. & Garvert, M. tankred-saanum/cognitive-maps-for-rewards: release test v.01. Zenodo https://doi.org/10.5281/zenodo.7486683 (2022). |
| تواريخ الأحداث: | Date Created: 20230403 Date Completed: 20230407 Latest Revision: 20230418 |
| رمز التحديث: | 20231215 |
| مُعرف محوري في PubMed: | PMC10076220 |
| DOI: | 10.1038/s41593-023-01283-x |
| PMID: | 37012381 |
| قاعدة البيانات: | MEDLINE |
Find this article in full text from Springer Verlag. 
Full Text Finder كن أول من يترك تعليقا!