دورية أكاديمية
Influence of reinforcement and its omission on trial-by-trial changes of response bias in perceptual decision making.
| العنوان: | Influence of reinforcement and its omission on trial-by-trial changes of response bias in perceptual decision making. |
|---|---|
| المؤلفون: | Stüttgen MC; Institute of Pathophysiology, University Medical Center of the Johannes Gutenberg University Mainz, Germany., Dietl A; Institute of Pathophysiology, University Medical Center of the Johannes Gutenberg University Mainz, Germany., Stoilova Eckert VV; Institute of Pathophysiology, University Medical Center of the Johannes Gutenberg University Mainz, Germany., de la Cuesta-Ferrer L; Institute of Pathophysiology, University Medical Center of the Johannes Gutenberg University Mainz, Germany., Blanke JH; Institute of Pathophysiology, University Medical Center of the Johannes Gutenberg University Mainz, Germany., Koß C; Centre for Cognitive Science, Institute of Psychology, Technical University of Darmstadt, Germany., Jäkel F; Centre for Cognitive Science, Institute of Psychology, Technical University of Darmstadt, Germany. |
| المصدر: | Journal of the experimental analysis of behavior [J Exp Anal Behav] 2024 May; Vol. 121 (3), pp. 294-313. Date of Electronic Publication: 2024 Mar 01. |
| نوع المنشور: | Journal Article; Research Support, N.I.H., Extramural; Research Support, Non-U.S. Gov't |
| اللغة: | English |
| بيانات الدورية: | Publisher: Wiley-Blackwell Country of Publication: United States NLM ID: 0203727 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1938-3711 (Electronic) Linking ISSN: 00225002 NLM ISO Abbreviation: J Exp Anal Behav Subsets: MEDLINE |
| أسماء مطبوعة: | Publication: [Malden, MA] : Wiley-Blackwell Original Publication: Waltham, Mass., Society for the Experimental Analysis of Behavior. |
| مواضيع طبية MeSH: | Reinforcement, Psychology* , Decision Making* , Discrimination Learning*, Animals ; Rats ; Male ; Models, Psychological ; Signal Detection, Psychological ; Conditioning, Operant ; Choice Behavior ; Acoustic Stimulation ; Discrimination, Psychological |
| مستخلص: | Discrimination performance in perceptual choice tasks is known to reflect both sensory discriminability and nonsensory response bias. In the framework of signal detection theory, these aspects of discrimination performance are quantified through separate measures, sensitivity (d') for sensory discriminability and decision criterion (c) for response bias. However, it is unknown how response bias (i.e., criterion) changes at the single-trial level as a consequence of reinforcement history. We subjected rats to a two-stimulus two-response conditional discrimination task with auditory stimuli and induced response bias through unequal reinforcement probabilities for the two responses. We compared three signal-detection-theory-based criterion learning models with respect to their ability to fit experimentally observed fluctuations of response bias on a trial-by-trial level. These models shift the criterion by a fixed step (1) after each reinforced response or (2) after each nonreinforced response or (3) after both. We find that all three models fail to capture essential aspects of the data. Prompted by the observation that steady-state criterion values conformed well to a behavioral model of signal detection based on the generalized matching law, we constructed a trial-based version of this model and find that it provides a superior account of response bias fluctuations under changing reinforcement contingencies. (© 2024 The Authors. Journal of the Experimental Analysis of Behavior published by Wiley Periodicals LLC on behalf of Society for the Experimental Analysis of Behavior.) |
| References: | Alsop, B. (1991). Behavioral models of signal detection and detection models of choice. In M. L. Commons, J. A. Nevin, & M. C. Davison (Eds.), Signal detection: Mechanisms, models, and applications (pp. 39–55). Erlbaum. https://doi.org/10.4324/9780203772430. Alsop, B. (1998). Receiver operating characteristics from nonhuman animals: Some implications and directions for research with humans. Psychonomic Bulletin & Review, 5(2), 239–252. https://doi.org/10.3758/BF03212946. Baum, W. M. (1974). On two types of deviation from the matching law: Bias and undermatching. Journal of the Experimental Analysis of Behavior, 22(1), 231–242. https://doi.org/10.1901/jeab.1974.22-231. Benjamin, A. S., Diaz, M., & Wee, S. (2009). Signal detection with criterion noise: Applications to recognition memory. Psychological Review, 116(1), 84–115. https://doi.org/10.1037/a0014351. Boneau, C. A., & Cole, J. L. (1967). Decision theory, the pigeon, and the psychophysical function. Psychological Review, 74(2), 123–135. https://doi.org/10.1037/h0024287. Busemeyer, J. R., & Myung, I. J. (1992). An adaptive approach to human decision making: Learning theory, decision theory, and human performance. Journal of Experimental Psychology: General, 121(2), 177–194. https://doi.org/10.1037/0096-3445.121.2.177. Commons, M. L., Nevin, J. A., & Davison, M. C. (Eds.). (1991). Signal detection: Mechanisms, models, and applications. Erlbaum. https://doi.org/10.4324/9780203772430. Corrado, G. S., Sugrue, L. P., Sebastian Seung, H., & Newsome, W. T. (2005). Linear‐nonlinear‐Poisson models of primate choice dynamics. Journal of the Experimental Analysis of Behavior, 84(3), 581–617. https://doi.org/10.1901/jeab.2005.23-05. Green, D. M., & Swets, J. A. (1988). Signal detection theory and psychophysics. Peninsula Publishing. Davison, M. (1991). Stimulus discriminability, contingency discriminability, and complex stimulus control. In M. L. Commons, J. A. Nevin, & M. C. Davison (Eds.), Signal detection: Mechanisms, models, and applications (pp. 57–78). Erlbaum. https://doi.org/10.4324/9780203772430. Davison, M., & Jenkins, P. E. (1985). Stimulus discriminability, contingency discriminability, and schedule performance. Animal Learning & Behavior, 13(1), 77–84. https://doi.org/10.3758/BF03213368. Davison, M., & McCarthy, D. (1980). Reinforcement for errors in a signal‐detection procedure. Journal of the Experimental Analysis of Behavior, 34(1), 35–47. https://doi.org/10.1901/jeab.1980.34-35. Davison, M., & McCarthy, D. (1981). Undermatching and structural relations. Behaviour Analysis Letters, 1(1), 67–72. Davison, M., & Nevin, J. (1999). Stimuli, reinforcers, and behavior: An integration. Journal of the Experimental Analysis of Behavior, 71(3), 439–482. https://doi.org/10.1901/jeab.1999.71-439. Davison, M., & Tustin, R. D. (1978). The relation between the generalized matching law and signal‐detection theory. Journal of the Experimental Analysis of Behavior, 29(2), 331–336. https://doi.org/10.1901/jeab.1978.29-331. Dorfman, D. D. (1969). Probability matching in signal detection. Psychonomic Science, 17(2), 103–103. https://doi.org/10.3758/BF03336468. Dorfman, D. D. (1973). Likelihood function of additive learning models: Sufficient conditions for strict log‐concavity and uniqueness of maximum. Journal of Mathematical Psychology, 10(1), 73–85. https://doi.org/10.1016/0022-2496(73)90005-9. Dorfman, D. D., & Biderman, M. (1971). A learning model for a continuum of sensory states. Journal of Mathematical Psychology, 8(2), 264–284. https://doi.org/10.1016/0022-2496(71)90017-4. Dorfman, D. D., Saslow, C. F., & Simpson, J. C. (1975). Learning models for a continuum of sensory states reexamined. Journal of Mathematical Psychology, 12(2), 178–211. https://doi.org/10.1016/0022-2496(75)90056-5. Erev, I. (1998). Signal detection by human observers: A cutoff reinforcement learning model of categorization decisions under uncertainty. Psychological Review, 105(2), 280–298. https://doi.org/10.1037/0033-295X.105.2.280. Estes, W. K. (2002). Traps in the route to models of memory and decision. Psychonomic Bulletin and Review, 9(1), 3–25. https://doi.org/10.3758/BF03196254. Feng, S., Holmes, P., Rorie, A., & Newsome, W. T. (2009). Can monkeys choose optimally when faced with noisy stimuli and unequal rewards? PLoS Computational Biology, 5(2), Article e1000284. https://doi.org/10.1371/journal.pcbi.1000284. Friedman, M. P., Carterette, E. C., Nakatani, L., & Ahumada, A. (1968). Comparisons of some learning models for response bias in signal detection. Perception & Psychophysics, 3, 5–11. https://doi.org/10.3758/BF03212703. Funamizu, A. (2021). Integration of sensory evidence and reward expectation in mouse perceptual decision‐making task with various sensory uncertainties. IScience, 24, Article 102826. https://doi.org/10.1016/j.isci.2021.102826. Gold, J. I., & Ding, L. (2013). How mechanisms of perceptual decision‐making affect the psychometric function. Progress in Neurobiology, 103, 98–114. https://doi.org/10.1016/j.pneurobio.2012.05.008. Harley, C. B. (1981). Learning the evolutionarily stable strategy. Journal of Theoretical Biology, 89(4), 611–633. https://doi.org/10.1016/0022-5193(81)90032-1. Hautus, M. J., Macmillan, N. A., & Creelman, C. D. (2022). Detection theory: A user's guide (3rd ed.). Routledge. https://doi.org/10.4324/9781003203636. Hernández‐Navarro, L., Hermoso‐Mendizabal, A., Duque, D., de la Rocha, J., & Hyafil, A. (2021). Proactive and reactive accumulation‐to‐bound processes compete during perceptual decisions. Nature Communications, 12(1), Article 7148. https://doi.org/10.1038/s41467-021-27302-8. Herrnstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4, 267–272. https://doi.org/10.1901/jeab.1961.4-267. Kac, M. (1962). A note on learning signal detection. IRE Transactions on Information Theory, 8(2), 126–128. https://doi.org/10.1109/TIT.1962.1057687. Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90(430), 773–795. https://doi.org/10.1080/01621459.1995.10476572. Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008). Neural correlates, computation and behavioural impact of decision confidence. Nature, 455(7210), 227–231. https://doi.org/10.1038/nature07200. Killeen, P. R., Taylor, T. J., & Treviño, M. (2018). Subjects adjust criterion on errors in perceptual decision tasks. Psychological Review, 125(1), 117–130. https://doi.org/10.1037/rev0000056. Lak, A., Hueske, E., Hirokawa, J., Masset, P., Ott, T., Urai, A. E., Donner, T. H., Carandini, M., Tonegawa, S., Uchida, N., & Kepecs, A. (2020). Reinforcement biases subsequent perceptual decisions when confidence is low: A widespread behavioral phenomenon. ELife, 9, 1–26. https://doi.org/10.7554/eLife.49834. Lak, A., Nomoto, K., Keramati, M., Sakagami, M., & Kepecs, A. (2017). Midbrain dopamine neurons signal belief in choice accuracy during a perceptual decision. Current Biology, 27(6), 821–832. https://doi.org/10.1016/j.cub.2017.02.026. Lak, A., Okun, M., Moss, M. M., Gurnani, H., Farrell, K., Wells, M. J., Reddy, C. B., Kepecs, A., Harris, K. D., & Carandini, M. (2020). Dopaminergic and prefrontal basis of learning from sensory confidence and reward value. Neuron, 105(4), 700–711. https://doi.org/10.1016/j.neuron.2019.11.018. Luce, R. D. (1963). A threshold theory for simple detection experiments. Psychological Review, 70(1), 61–79. https://doi.org/10.1037/h0039723. McCarthy, D., & Davison, M. (1979). Signal probability, reinforcement and signal‐detection. Journal of the Experimental Analysis of Behavior, 32(3), 373–386. https://doi.org/10.1901/jeab.1979.32-373. McCarthy, D., & Davison, M. (1980). Independence of sensitivity to relative reinforcement rate and discriminability in signal‐detection. Journal of the Experimental Analysis of Behavior, 34(3), 273–284. https://doi.org/10.1901/jeab.1980.34-273. McCarthy, D., & Davison, M. (1981). Towards a behavioral theory of bias in signal detection. Perception & Psychophysics, 29(4), 371–382. https://doi.org/10.3758/bf03207347. McCarthy, D., & Davison, M. (1984). Isobias and alloiobias functions in animal psycophysics. Journal of Experimental Psychology: Animal Behavior Processes, 10(3), 390–409. https://doi.org/10.1037/0097-7403.10.3.390. McCarthy, D., Davison, M., & Jenkins, P. E. (1982). Stimulus discriminability in free‐operant and discrete‐trial detection procedures. Journal of the Experimental Analysis of Behavior, 37(2), 199–215. https://doi.org/10.1901/jeab.1982.37-199. Mill, R. W., Alves‐Pinto, A., & Sumner, C. J. (2014). Decision criterion dynamics in animals performing an auditory detection task. PLoS ONE, 9(12), Article e114076. https://doi.org/10.1371/journal.pone.0114076. Pisupati, S., Chartarifsky‐Lynn, L., Khanal, A., & Churchland, A. K. (2021). Lapses in perceptual decisions reflect exploration. ELife, 10, 1–27. https://doi.org/10.7554/ELIFE.55490. Stoilova, V. V., Knauer, B., Berg, S., Rieber, E., Jäkel, F., & Stüttgen, M. C. (2020). Auditory cortex reflects goal‐directed movement but is not necessary for behavioral adaptation in sound‐cued reward tracking. Journal of Neurophysiology, 124(4), 1056–1071. https://doi.org/10.1152/jn.00736.2019. Stubbs, D. A., & Pliskoff, S. S. (1969). Concurrent responding with fixed relative rate of reinforcement. Journal of the Experimental Analysis of Behavior, 12(6), 887–895. https://doi.org/10.1901/jeab.1969.12-887. Stüttgen, M. C., Kasties, N., Lengersdorf, D., Starosta, S., Güntürkün, O., & Jäkel, F. (2013). Suboptimal criterion setting in a perceptual choice task with asymmetric reinforcement. Behavioural Processes, 96, 59–70. https://doi.org/10.1016/j.beproc.2013.02.014. Stüttgen, M. C., Schwarz, C., & Jäkel, F. (2011). Mapping spikes to sensations. Frontiers in Neuroscience, 5, Article 125. Stüttgen, M. C., Yildiz, A., & Güntürkün, O. (2011). Adaptive criterion setting in perceptual decision making. Journal of the Experimental Analysis of Behavior, 96(2), 155–176. https://doi.org/10.1901/jeab.2011.96-155. Swets, J. A. (1961a). Detection theory and psychophysics: A review. Psychometrika, 26(1), 49–63. https://doi.org/10.1007/BF02289684. Swets, J. A. (1961b). Is there a sensory threshold? Science, 134(3473), 168–177. https://doi.org/10.1126/science.134.3473.168. Teichert, T., & Ferrera, V. P. (2010). Suboptimal integration of reward magnitude and prior reward likelihood in categorical decisions by monkeys. Frontiers in Neuroscience, 4, Article 186. https://doi.org/10.3389/fnins.2010.00186. Thomas, E. A. C. (1975). Criterion adjustment and probability matching. Perception & Psychophysics, 18, 158–162. https://doi.org/10.3758/BF03204104. Treisman, M., & Williams, T. C. (1984). A theory of criterion setting with an application to sequential dependencies. Psychological Review, 91(1), 68–111. https://doi.org/10.1037//0033-295X.91.1.68. Vandevelde, J. R., Yang, J.‐W., Albrecht, S., Lam, H., Kaufmann, P., Luhmann, H. J., & Stüttgen, M. C. (2023). Layer‐ and cell‐type‐specific differences in neural activity in mouse barrel cortex during a whisker detection task. Cerebral Cortex, 33, 1361–1382. https://doi.org/10.1093/cercor/bhac141. Wichmann, F. A., & Jäkel, F. (2018). Methods in psychophysics. In J. T. Wixted (Ed.), Stevens' handbook of experimental psychology and cognitive neuroscience (4th ed., pp. 1–42). John Wiley & Sons. https://doi.org/10.1002/9781119170174.epcn507. Yang, T., & Shadlen, M. N. (2007). Probabilistic reasoning by neurons. Nature, 447(7148), 1075–1080. https://doi.org/10.1038/nature05852. |
| معلومات مُعتمدة: | JA 1878/2-1 Deutsche Forschungsgemeinschaft; STU 544/6-1 Deutsche Forschungsgemeinschaft |
| فهرسة مساهمة: | Keywords: criterion; rat; response bias; reward; signal detection theory |
| تواريخ الأحداث: | Date Created: 20240301 Date Completed: 20240508 Latest Revision: 20240508 |
| رمز التحديث: | 20240508 |
| DOI: | 10.1002/jeab.908 |
| PMID: | 38426657 |
| قاعدة البيانات: | MEDLINE |
Full Text Finder | تدمد: | 1938-3711 |
|---|---|
| DOI: | 10.1002/jeab.908 |