Lowered inter-stimulus discriminability hurts incremental contributions to learning.

التفاصيل البيبلوغرافية
العنوان:	Lowered inter-stimulus discriminability hurts incremental contributions to learning.
المؤلفون:	Yoo AH; Department of Psychology, University of California, Berkeley, USA.; Helen Wills Neuroscience Institute, University of California, Berkeley, USA., Keglovits H; Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, USA., Collins AGE; Department of Psychology, University of California, Berkeley, USA. annecollins@berkeley.edu.; Helen Wills Neuroscience Institute, University of California, Berkeley, USA. annecollins@berkeley.edu.
المصدر:	Cognitive, affective & behavioral neuroscience [Cogn Affect Behav Neurosci] 2023 Oct; Vol. 23 (5), pp. 1346-1364. Date of Electronic Publication: 2023 Sep 01.
نوع المنشور:	Journal Article
اللغة:	English
بيانات الدورية:	Publisher: Springer Country of Publication: United States NLM ID: 101083946 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1531-135X (Electronic) Linking ISSN: 15307026 NLM ISO Abbreviation: Cogn Affect Behav Neurosci Subsets: MEDLINE
أسماء مطبوعة:	Publication: 2011- : New York : Springer Original Publication: Austin, TX : Psychonomic Society, c2001-
مواضيع طبية MeSH:	Learning/physiology , Reinforcement, Psychology, Humans ; Animals ; Dogs ; Reward ; Memory
مستخلص:	How does the similarity between stimuli affect our ability to learn appropriate response associations for them? In typical laboratory experiments learning is investigated under somewhat ideal circumstances, where stimuli are easily discriminable. This is not representative of most real-life learning, where overlapping "stimuli" can result in different "rewards" and may be learned simultaneously (e.g., you may learn over repeated interactions that a specific dog is friendly, but that a very similar looking one isn't). With two experiments, we test how humans learn in three stimulus conditions: one "best case" condition in which stimuli have idealized and highly discriminable visual and semantic representations, and two in which stimuli have overlapping representations, making them less discriminable. We find that, unsurprisingly, decreasing stimuli discriminability decreases performance. We develop computational models to test different hypotheses about how reinforcement learning (RL) and working memory (WM) processes are affected by different stimulus conditions. Our results replicate earlier studies demonstrating the importance of both processes to capture behavior. However, our results extend previous studies by demonstrating that RL, and not WM, is affected by stimulus distinctness: people learn slower and have higher across-stimulus value confusion at decision when stimuli are more similar to each other. These results illustrate strong effects of stimulus type on learning and demonstrate the importance of considering parallel contributions of different cognitive processes when studying behavior. (© 2023. The Author(s).)
References:	Akaike, H. (1972). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716–723. (PMID: 10.1109/TAC.1974.1100705) Alexander, G. E., DeLong, M. R., & Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annual Review of Neuroscience, 9, 357–381. https://doi.org/10.1146/annurev.ne.09.030186.002041. Baddeley, A.D., Hitch, G. (1974). Working Memory. In Bower, G. H. (Ed), Psychology of Learning and Motivation Psychology of Learning and Motivation, 8, 47–89. Academic Press. https://doi.org/10.1016/S0079-7421(08)60452-1. Barto, A. G. (1995). Adaptive Critics and the Basal Ganglia. In J. C. Houk & D. Beiser (Eds.), Models of Information Processing in the Basal Ganglia (pp. 215–232). MA MIT Press: Cambridge. Bays, P. M. (2014). Noise in neural populations accounts for errors in working memory. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 34(10), 3632–3645. https://doi.org/10.1523/JNEUROSCI.3204-13.2014. Bornstein, A. M., & Norman, K. A. (2017). Reinstated episodic context guides sampling-based decisions for reward. Nature Neuroscience, 20(7), 997–1003. https://doi.org/10.1038/nn.4573. Brady, T. F., Störmer, V. S., & Alvarez, G. A. (2016). Working memory is not fixed-capacity: More active storage capacity for real-world objects than for simple stimuli. Proceedings of the National Academy of Sciences, 113(27), 7459–7464. https://doi.org/10.1073/pnas.1520027113. Christophel, T. B., Klink, P. C., Spitzer, B., Roelfsema, P. R., & Haynes, J. D. (2017). The distributed nature of working memory. Trends in Cognitive Sciences, 21(2), 111–124. https://doi.org/10.1016/j.tics.2016.12.007. Chun, M. M., Golomb, J. D., & Turk-Browne, N. B. (2011). A taxonomy of external and internal attention. Annual Review of Psychology, 62, 73–101. https://doi.org/10.1146/annurev.psych.093008.100427. Collins, A. G. E. (2018). The Tortoise and the Hare: Interactions between reinforcement learning and working memory. Journal of Cognitive Neuroscience, 30, 10. https://doi.org/10.1162/jocn_a_01238. Collins, A. G. E., Brown, J. K., Gold, J. M., Waltz, J. A., & Frank, M. J. (2014). Working memory contributions to reinforcement learning impairments in schizophrenia. Journal of Neuroscience, 34(41), 13747–13756. https://doi.org/10.1523/JNEUROSCI.0989-14.2014. Collins, A. G. E., Ciullo, B., Frank, M. J., & Badre, D. (2017). Working memory load strengthens reward prediction errors. Journal of Neuroscience, 37(16), 4332–4342. https://doi.org/10.1523/JNEUROSCI.2700-16.2017. Collins, A. G. E., & Frank, M. J. (2012). How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. European Journal of Neuroscience, 35(7), 1024–1035. https://doi.org/10.1111/j.1460-9568.2011.07980.x. Collins, A. G. E., & Frank, M. J. (2014). Opponent actor learning (OpAL): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychological Review, 121(3), 337–366. https://doi.org/10.1037/a0037015. Cowan, N. (2017). The many faces of working memory and short-term storage. Psychological Bulletin & Review, 24(4), 1158–1170. https://doi.org/10.3758/s13423-016-1191-6. Conrad, R. (1964). Acoustic Confusions in Immediates Memory. British Journal of Psychology, 55(1), 75–84. https://doi.org/10.1111/j.2044-8295.1964.tb00899.x. Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69(6), 1204–1215. https://doi.org/10.1016/j.neuron.2011.02.027. Eckstein, M. K., Master, S. L., Xia, L., Dahl, R. E., Wilbrecht, L. & Collins, A. G. (2022). The interpretation of computational model parameters depends on the context. eLife, 11, e75474. https://doi.org/10.7554/eLife.75474. Emrich, S. M., Lockhart, H. A., & Al-Aidroos, N. (2017). Attention mediates the flexible allocation of visual working memory resources. Journal of Experimental Psychology. Human Perception and Performance, 43(7), 1454–1465. https://doi.org/10.1037/xhp0000398. Farashahi, S., Rowe, K., Aslami, Z., Lee, D., & Soltani, A. (2017). Feature-based learning improves adaptability without compromising precision. Nature Communications, 8(1), 1768. https://doi.org/10.1038/s41467-017-01874-w. Farashahi, S., Xu, J., Wu, S. W. & Soltani, A. (2020). Learning arbitrary stimulus-reward associations for naturalistic stimuli involves transition from learning about features to learning about objects. Cognition, 205, 104425. https://doi.org/10.1016/j.cognition.2020.104425. Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007). Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proceedings of the National Academy of Sciences, 104(41), 16311–16316. https://doi.org/10.1073/pnas.0706111104. Frank, M. J., & O’Reilly, R. C. (2006). A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behavioral Neuroscience, 120(3), 497–517. https://doi.org/10.1037/0735-7044.120.3.497. Frank, M. J., Seeberger, L. C., & O’Reilly, R. C. (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science, 306(5703), 1940–1943. https://doi.org/10.1126/science.1102941. Gershman, S. J. (2015). Do learning rates adapt to the distribution of rewards? Psychonomic Bulletin & Review, 22(5), 1320–1327. https://doi.org/10.3758/s13423-014-0790-3. Goodman, N. D., Tenenbaum, J. B., Feldman, J., & Griffiths, T. L. (2008). A rational analysis of rule-based concept learning. Cognitive Science, 32(1), 108–154. https://doi.org/10.1080/03640210701802071. Haber, S. N. (2011). Neural circuits of reward and decision making: Integrative networks across Corticobasal Ganglia Loops. In R. B., Mars, J., Sallet, M. F. S. Rushworth & N. Yeung (Eds), Neural Basis of Motivational and Cognitive Control. MIT Press. Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006). The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 26(32), 8360–8367. https://doi.org/10.1523/JNEUROSCI.1010-06.2006. Harrison, S. A., & Tong, F. (2009). Decoding reveals the contents of visual working memory in early visual areas. Nature, 458(7238), 632–635. https://doi.org/10.1038/nature07832. Hoskin, A. N., Bornstein, A. M., Norman, K. A., & Cohen, J. D. (2019). Refresh my memory: Episodic memory reinstatements intrude on working memory maintenance. Cognitive, Affective, & Behavioral Neuroscience, 19(2), 338–354. https://doi.org/10.3758/s13415-018-00674-z. Hurvich, M. C., & Tsai, L. C. (1987). Regression and time series model selection in small samples. Biometrika, 76, 297–307. Jafarpour, A., Buffalo, E. A., Knight, R. T. & Collins, A. G. E. (2022). Event segmentation reveals working memory forgetting rate. iScience, 25(3), 103902. https://doi.org/10.1016/j.isci.2022.103902. Klyszejko, Z., Rahmati, M., & Curtis, C. E. (2014). Attentional priority determines working memory precision. Vision Research, 105, 70–76. https://doi.org/10.1016/j.visres.2014.09.002. Leong, Y. C., Radulescu, A., Daniel, R., DeWoskin, V., & Niv, Y. (2017). Dynamic interaction between reinforcement learning and attention in multidimensional environments. Neuron, 93(2), 451–463. https://doi.org/10.1016/j.neuron.2016.12.040. Love, B. C., Medin, D. L. & Gureckis, T. M. (2004). SUSTAIN: A network model of category learning. Psychological Review, 111(2), 309–332. https://doi.org/10.1037/0033-295X.111.2.309. Luck, J. S., & Vogel, K. E. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390, 279–281. McClure, S. M., Berns, G. S., & Montague, P. R. (2003). Temporal prediction errors in a passive learning task activate human striatum. Neuron, 38(2), 339–346. https://doi.org/10.1016/s0896-6273(03)00154-5. McDougle, S. D., & Collins, A. G. E. (2020). Modeling the influence of working memory, reinforcement, and action uncertainty on reaction time and choice during instrumental learning. Psychonomic Bulletin & Review, 28(1), 20–39. https://doi.org/10.3758/s13423-020-01774-z. Murdock, B. B., Jr. (1962). The serial position effect of free recall. Journal of Experimental Psychology, 64(5), 482–488. https://doi.org/10.1037/h0045106. Nilsson, H., Rieskamp, J., & Wagenmakers, E. J. (2011). Hierarchical Bayesian parameter estimation for cumulative prospect theory. Journal of Mathematical Psychology, 55(1), 84-93. https://doi.org/10.1016/j.jmp.2010.08.006. Niv, Y., Daniel, R., Geana, A., Gershman, S. J., Leong, Y. C., Radulescu, A., & Wilson, R. C. (2015). Reinforcement learning in multidimensional environments relies on attention mechanisms. Journal of Neuroscience, 35(21), 8145–8157. https://doi.org/10.1523/JNEUROSCI.2978-14.2015. Niv, Y., Edlund, J. A., Dayan, P., & O’Doherty, J. P. (2012). Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain. Journal of Neuroscience, 32(2), 551–562. https://doi.org/10.1523/JNEUROSCI.5498-10.2012. Nosofsky, R. M. (1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115(1), 39–57. https://doi.org/10.1037/0096-3445.115.1.39. Nosofsky, R. M., Palmeri, T. J., & McKinley, S. C. (1994). Rule-plus-exception model of classification learning. Psychological Review, 101(1), 53–79. https://doi.org/10.1037/0033-295x.101.1.53. O’Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron, 38(2), 329–337. https://doi.org/10.1016/s0896-6273(03)00169-7. Oemisch, M., Westendorff, S., Azimi, M., Hassani, S. A., Ardid, S., Tiesinga, P., & Womelsdorf, T. (2019). Feature-specific prediction errors and surprise across macaque fronto-striatal circuits. Nature Communications, 10(1), 176. https://doi.org/10.1038/s41467-018-08184-9. Palminteri, S., Wyart, V., & Koechlin, E. (2017). The importance of falsification in computational cognitive modeling. Trends in Cognitive Sciences, 21(6), 425–433. https://doi.org/10.1016/j.tics.2017.03.011. Petrides, M. (1985). Deficits on conditional associative-learning tasks after frontal- and temporal-lobe lesions in man. Neuropsychologia, 23(5), 601–614. https://doi.org/10.1016/0028-3932(85)90062-4. Radulescu, A., Niv, Y., & Ballard, I. (2019). Holistic reinforcement learning: The role of structure and attention. Trends in Cognitive Sciences, 23(4), 278–292. https://doi.org/10.1016/j.tics.2019.01.010. Radulescu, A., Vong, W. K. & Gureckis, T. M. (2022). Name that state: How language affects human reinforcement learning Name that state: How language affects human reinforcement learning. Proceedings of the Annual Meeting of the Cognitive Science Society, 44. Rich, A. S., & Gureckis, T. M. (2018). The limits of learning: Exploration, generalization, and the development of learning traps. Journal of Experimental Psychology. General, 147(11), 1553–1570. https://doi.org/10.1037/xge0000466. Riggall, A. C., & Postle, B. R. (2012). The relationship between working memory storage and elevated activity as measured with functional magnetic resonance imaging. Journal of Neuroscience, 32(38), 12990–12998. https://doi.org/10.1523/JNEUROSCI.1892-12.2012. Rigoux, L., Stephan, K. E., Friston, K. J., & Daunizeau, J. (2014). Bayesian model selection for group studies - revisited. NeuroImage, 84, 971–985. https://doi.org/10.1016/j.neuroimage.2013.08.065. Rmus, M., & Collins, A. G. E. (2020). What is a Choice in Reinforcement Learning? In Proceedings of the The Annual Meeting of the Cognitive Science Society. Rmus, M., McDougle, S. D., & Collins, A. G. E. (2021). The role of executive function in shaping reinforcement learning. Current Opinion in Behavioral Sciences, 38, 66–73. https://doi.org/10.1016/j.cobeha.2020.10.003. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593–1599. https://doi.org/10.1126/science.275.5306.1593. Shepard, R. N., Hovland, C. I. & Jenkins, H. M. (1961). Learning and memorization of classifications. Psychological Monographs: General and Applied, 75(13), 1–42. https://doi.org/10.1037/h0093825. Shohamy, D., Myers, C. E., Grossman, S., Sage, J., Gluck, M. A., & Poldrack, R. A. (2004). Cortico-striatal contributions to feedback-based learning: converging data from neuroimaging and neuropsychology. Brain, 127(4), 851–859. https://doi.org/10.1093/brain/awh100. Sims, C. R. (2015). The cost of misremembering: Inferring the loss function in visual working memory. Journal of Vision, 15(3), 2. https://doi.org/10.1167/15.3.2. Souza, A. S., Thalmann, M., & Oberauer, K. (2018). The precision of spatial selection into the focus of attention in working memory. Psychonomic Bulletin & Review, 25(6), 2281–2288. https://doi.org/10.3758/s13423-018-1471-4. Starr, A., Srinivasan, M., & Bunge, S. A. (2020). Semantic knowledge influences visual working memory in adults and children. PLOS ONE, 15(11). https://doi.org/10.1371/journal.pone.0241110. Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J., & Friston, K. J. (2009). Bayesian Model Selection for Group Studies. NeuroImage, 46(4), 1004–1017. https://doi.org/10.1016/j.neuroimage.2009.03.025. Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology: General, 18(6), 643–662. https://doi.org/10.1037/h0054651. Sugawara, M., & Katahira, K. (2021). Dissociation between asymmetric value updating and perseverance in human reinforcement learning. Scientific Reports, 11(1), 3574. https://doi.org/10.1038/s41598-020-80593-7. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: an introduction. Cambridge: Mass MIT Press. Valentin, V. V., Dickinson, A., & O’Doherty, J. P. (2007). Determining the neural substrates of goal-directed learning in the human brain. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 27(15), 4019–4026. https://doi.org/10.1523/JNEUROSCI.0564-07.2007. Viejo, G., Khamassi, M., Brovelli, A., & Girard, B. (2015). Modeling choice and reaction time during arbitrary visuomotor learning through the coordination of adaptive working memory and reinforcement learning. Frontiers in Behavioral Neuroscience, 9, 225. https://doi.org/10.3389/fnbeh.2015.00225. Wilken, P., & Ma, W. J. (2004). A detection theory account of change detection. Journal of Vision, 4(12), 1120–1135. https://doi.org/10.1167/4.12.11. Wilson, R. C. & Collins, A. G. (2019). Ten simple rules for the computational modeling of behavioral data. eLife, 8, e49547. https://doi.org/10.7554/eLife.49547. Wilson, R. C., & Niv, Y. X. (2012). Inferring relevance in a changing world. Frontiers in Human Neuroscience, 5, 189. https://doi.org/10.3389/fnhum.2011.00189. Wunderlich, K., Beierholm, U. R., Bossaerts, P., & O’Doherty, J. P. (2011). The human prefrontal cortex mediates integration of potential causes behind observed outcomes. Journal of Neurophysiology, 106(3), 1558–1569. https://doi.org/10.1152/jn.01051.2010. Xia, L., Master, S. L., Eckstein, M. K., Baribault, B., Dahl, R. E., Wilbrecht, L., & Collins, A. G. E. (2021). Modeling changes in probabilistic reinforcement learning during adolescence. PLOS Computational Biology, 17(7). https://doi.org/10.1371/journal.pcbi.1008524. Yoo, A. H., & Collins, A. G. E. (2022). How working memory and reinforcement learning are intertwined: A cognitive, neural, and computational perspective. Journal of Cognitive Neuroscience, 34(4), 551–568. https://doi.org/10.1162/jocn_a_01808. Yoo, A. H., Klyszejko, Z., Curtis, C. E., & Ma, W. J. (2018). Strategic allocation of working memory resource. Scientific Reports, 8, 16162. https://doi.org/10.1038/s41598-018-34282-1.
فهرسة مساهمة:	Keywords: Computational modeling; Reinforcement learning; Working memory
تواريخ الأحداث:	Date Created: 20230901 Date Completed: 20231004 Latest Revision: 20231201
رمز التحديث:	20240628
مُعرف محوري في PubMed:	PMC10545593
DOI:	10.3758/s13415-023-01104-5
PMID:	37656373
قاعدة البيانات:	MEDLINE

Find this article in full text from Springer Verlag.

Full Text Finder

الوصف
تدمد:	1531-135X
DOI:	10.3758/s13415-023-01104-5