Hostname: page-component-78c5997874-mlc7c Total loading time: 0 Render date: 2024-11-18T01:27:11.085Z Has data issue: false hasContentIssue false
  • English
  • Français

Beyond “incentive hope”: Information sampling and learning under reward uncertainty

Published online by Cambridge University Press:  19 March 2019

Maya Zhe Wang Open the ORCID record for Maya Zhe Wang [Opens in a new window]  and
Benjamin Y. Hayden Open the ORCID record for Benjamin Y. Hayden [Opens in a new window]
Maya Zhe Wang
Affiliation:
Department of Brain and Cognitive Sciences and Center for Visual Sciences, University of Rochester, Rochester, NY 14627 Department of Neuroscience and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455. wang8200@umn.edubenhayden@gmail.comhaydenlab.com
Benjamin Y. Hayden
Affiliation:
Department of Neuroscience and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455. wang8200@umn.edubenhayden@gmail.comhaydenlab.com
Get access Rights & Permissions [Opens in a new window]

Abstract

Information seeking, especially when motivated by strategic learning and intrinsic curiosity, could render the new mechanism “incentive hope” proposed by Anselme & Güntürkün sufficient, but not necessary to explain how reward uncertainty promotes reward seeking and consumption. Naturalistic and foraging-like tasks can help parse motivational processes that bridge learning and foraging behaviors and identify their neural underpinnings.

Type
Open Peer Commentary
Information
Behavioral and Brain Sciences , Volume 42 , 2019 , e56
Check for updates
Copyright
Copyright © Cambridge University Press 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abe, H. & Lee, D. (2011) Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron 70(4):731–41. http://doi.org/10.1016/j.neuron.2011.03.026.Google Scholar
Bateson, M. & Kacelnik, A. (1997) Starlings’ preference for predictable and unpredictable delays to food. Animal Behaviour 53(6):1129–42. https://doi.org/10.1006/anbe.1996.0388.Google Scholar
Behrens, T. E. J., Woolrich, M. W., Walton, M. E. & Rushworth, M. F. S. (2007) Learning the value of information in an uncertain world. Nature Neuroscience 10(9):1214–21. http://doi.org/10.1038/nn1954.Google Scholar
Blanchard, T. C., Hayden, B. Y. & Bromberg-Martin, E. S. (2015a) Orbitofrontal cortex uses distinct codes for different choice attributes in decisions motivated by curiosity. Neuron 85(3):602–14. http://doi.org/10.1016/j.neuron.2014.12.050.Google Scholar
Blanchard, T. C., Strait, C. E. & Hayden, B. Y. (2015b) Ramping ensemble activity in dorsal anterior cingulate neurons during persistent commitment to a decision. Journal of Neurophysiology 114:2439–49. http://doi:10.1152/jn.00711.2015.Google Scholar
Blanchard, T. C., Wilke, A. & Hayden, B. Y. (2014) Hot-hand bias in rhesus monkeys. Journal of Experimental Psychology: Animal Learning and Cognition 40(3):280–86. http://doi.org/10.1037/xan0000033.Google Scholar
Boorman, E. D., Behrens, T. E. & Rushworth, M. F. (2011) Counterfactual choice and learning in a neural network centered on human lateral frontopolar cortex. PLoS Biology 9(6):e1001093–13. http://doi.org/10.1371/journal.pbio.1001093.Google Scholar
Bromberg-Martin, E. S., Matsumoto, M. & Hikosaka, O. (2010) Dopamine in motivational control: Rewarding, aversive, and alerting. Neuron 68(5):815–34. http://doi.org/10.1016/j.neuron.2010.11.022.Google Scholar
Calhoun, A. J. & Hayden, B. Y. (2015) The foraging brain. Current Opinion in Behavioral Science 5:24–31. http://dx.doi.org/10.1016/j.cobeha.2015.07.003.Google Scholar
Chang, C. Y., Gardner, M., Di Tillio, M. G. & Schoenbaum, G. (2017) Optogenetic blockade of dopamine transients prevents learning induced by changes in reward features. Curbio 27(22):3480–86.Google Scholar
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. (2006) Cortical substrates for exploratory decisions in humans. Nature 441(7095):876–79. http://doi.org/10.1038/nature04766.Google Scholar
De Petrillo, F., Ventricelli, M., Ponsi, G. & Addessi, E. (2015) Do tufted capuchin monkeys play the odds? Flexible risk preferences in Sapajus spp. Animal Cognition 18(1):119–30.Google Scholar
Gershman, S. J. & Schoenbaum, G. (2017) Rethinking dopamine prediction errors. bioRxiv 239731 preprint. doi: https://doi.org/10.1101/239731.Google Scholar
Hayden, B. Y. (2018) Economic choice: The foraging perspective. Current Opinion in Behavioral Sciences 24:1–6.Google Scholar
Hayden, B. Y., Pearson, J. M. & Platt, M. L. (2009) Fictive reward signals in the anterior cingulate cortex. Science 324(5929):948–50. http://doi.org/10.1126/science.1168488.Google Scholar
Kacelnik, A. & Bateson, M. (1997) Risk-sensitivity: Crossroads for theories of decision-making. Trends in Cognitive Sciences 1(8):304–309. http://doi.org/10.1016/S1364-6613(97)01093-0.Google Scholar
Kaplan, R., Schuck, N. W. & Doeller, C. F. (2017) The role of mental maps in decision-making. Trends in Neurosciences 40(5):1–4. http://doi.org/10.1016/j.tins.2017.03.002.Google Scholar
Kiani, R. & Shadlen, M. N. (2009) Representation of confidence associated with a decision by neurons in the parietal cortex. Science 324(5928):759–64. http://doi.org/10.1126/science.1169405.Google Scholar
Kidd, C. & Hayden, B. Y. (2015) The psychology and neuroscience of curiosity. Neuron 88(3):449–60. http://doi.org/10.1016/j.neuron.2015.09.010.Google Scholar
Killian, N. J., Jutras, M. J. & Buffalo, E. A. (2012) A map of visual space in the primate entorhinal cortex. Nature 491(7426):761.Google Scholar
Kornell, N., Son, L. K. & Terrace, H. S. (2007) Transfer of metacognitive skills and hint seeking in monkeys. Psychological Science 18(1):64–71.Google Scholar
Langdon, A. J., Sharpe, M. J., Schoenbaum, G. & Niv, Y. (2018) Model-based predictions for dopamine. Current Opinion in Neurobiology 49:1–7.Google Scholar
Minderer, M. & Harvey, C. D. (2016) Neuroscience: Virtual reality explored. Nature 533(7603):324–24. http://doi.org/10.1038/nature17899.Google Scholar
Mobbs, D., Trimmer, P. C., Blumstein, D. T. & Dayan, P. (2018) Foraging for foundations in decision neuroscience: Insights from ethology. Neuroscience 13(18):19.Google Scholar
Musall, S., Kaufman, M. T., Gluf, S. & Churchland, A. K. (2018) Movement-related activity dominates cortex during sensory-guided decision making. bioRxiv preprint. https://doi.org/10.1101/308288.Google Scholar
Noonan, M. P., Walton, M. E., Behrens, T. E. J., Sallet, J., Buckley, M. J. & Rushworth, M. F. S. (2010) Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proceedings of the National Academy of Sciences USA 107(47):20547–52. http://doi.org/10.1073/pnas.1012246107.Google Scholar
Pearson, J. M., Hayden, B. Y., Raghavachari, S. & Platt, M. L. (2009) Neurons in posterior cingulate cortex signal exploratory decisions in a dynamic multioption choice task. Current Biology 19(18):1532–37. http://doi.org/10.1016/j.cub.2009.07.048.Google Scholar
Pitkow, X. & Angelaki, D. (2017) How the brain might work: Statistics flowing in redundant population codes. arXiv:1702.03492Google Scholar
Pouget, A., Drugowitsch, J. & Kepecs, A. (2016) Confidence and certainty: Distinct probabilistic quantities for different goals. Nature Neuroscience 19(3):366–74. http://doi.org/10.1038/nn.4240.Google Scholar
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E. & Behrens, T. E. (2011) Frontal cortex and reward-guided learning and decision-making. Neuron 70(6):1054–69. http://doi.org/10.1016/j.neuron.2011.05.014.Google Scholar
Sadacca, B. F., Jones, J. L. & Schoenbaum, G. (2016) Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework. eLife 016;5:e13665. http://doi.org/10.7554/eLife.13665.Google Scholar
Schonberg, T., Fox, C. R. & Poldrack, R. A. (2011) Mind the gap: Bridging economic and naturalistic risk-taking with cognitive neuroscience. Trends in Cognitive Sciences 15(1):11–19. http://doi.org/10.1016/j.tics.2010.10.002.Google Scholar
Stephens, D. W. & Krebs, J. R. (1986) Foraging theory. Princeton University Press.Google Scholar
Strait, C. E., Sleezer, B. J., Blanchard, T. C., Azab, H., Castagno, M. D. & Hayden, B. Y. (2016) Neuronal selectivity for spatial position of offers and choices in five reward regions. Journal of Neurophysiology 115:1098–1111.Google Scholar
Takahashi, Y. K., Batchelor, H. M., Liu, B., Khanna, A., Morales, M. & Schoenbaum, G. (2017) Dopamine neurons respond to errors in the prediction of sensory features of expected rewards. Neuron 95(6):1395–1405.Google Scholar
Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H. & Rushworth, M. F. S. (2010) Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron 65(6):927–39. http://doi.org/10.1016/j.neuron.2010.02.027.Google Scholar
Wang, M. Z. & Hayden, B. (2018) Monkeys are curious about counterfactual outcomes. bioRxiv preprint. http://doi.org/10.1101/291708.Google Scholar
Wang, M. Z. & Hayden, B. Y. (2017) Reactivation of associative structure specific outcome responses during prospective evaluation in reward-based choices. Nature Communications 8:15821. http://doi.org/10.1038/ncomms15821.Google Scholar
Wirth, S., Baraduc, P., Planté, A., Pinède, S., & Duhamel, J.-R. (2017) Gaze-informed, task-situated representation of space in primate hippocampus during virtual navigation. PLoS Biology 15(2):e2001045. http://doi.org/10.1371/journal.pbio.2001045.Google Scholar