This special topics course will examine the neural mechanisms of reward, decision making, and reward-based learning. The course covers the anatomy and physiology of reward and action systems. Special emphasis will be placed on the representation of reward expectation; the interplay between reward, motivation, and attention; and the selection of actions. Links between concepts in economics and the neural mechanisms of decision making will be explored. Data from animal and human studies collected using behavioral, neurophysiological, and functional magnetic resonance techniques will be reviewed. Instructor Andersen.
Students will present papers during the class (two per class, about 4 papers per student over the course of the term). In addition, two students per paper will be assigned as "discussants". Discussants will prepare questions for the paper presenter, to help the group get to the core issues of each paper.
Date |
Topic / Papers |
Presenter (Discussants) |
| 2010/04/01 | Class Orientation | N/A |
| 2010/04/08 |
Awareness of Intent |
|
| Desmurget, M., Reilly, K.T., Richard, N., Szathmari, A., Mottolese, C., Sirigu, A., Movement Intention After Parietal Cortex Stimulation in Humans. Science, 2009. 324: p. 811-813. | Britton (Boris) | |
| Sirigu, A., Daprati, C., Ciancia, S., Giraux, P., Nighoghossian, N., Posada, A., Haggard, P., Altered awareness of voluntary action after damage to the parietal cortex. Nature Neuroscience, 2004. 7: p. 80-84. | Laura (Alison) | |
|
Background Reading for Those Who are Interested: Karnath, H.O., Borchers, S., Himmelbach, M., Comment on "Movement Intention After Parietal Cortex Stimulation in Humans". Science, 2010. 327: p. DOI: 10.1126/science.1183735 Sirigu, A., Mottolese, C., Desmurget, M. , Response to Comment on "Movement Intention After Parietal Cortex Stimulation in Humans". Science, Science. 327: p. DOI: 10.1126/science.1183758 Haggard, P., The sources of human volution. Science, 2009. 324: p. 731-733. Snyder, L.H., A.P. Batista, and R.A. Andersen, Coding of intention in the posterior parietal cortex. Nature, 1997. 386(6621): p. 167-70. Andersen, R.A. and C.A. Buneo, Intentional maps in posterior parietal cortex. Annual Review of Neuroscience 2002. 25: p. 189-220. Haggard, P., Human volition: towards a neuroscience of will. Nature Reviews Neuroscience, 2008. 9: p. 934-946. Libet, B., Gleason, C.A., Wright, E.W., Pearl, D.K., Time of conscious intention to act in relation to onset of cerebral-activity (readiness-potential)-the unconscious initiation of a freely voluntary act. Brain, 1983. 106: p. 623-642. |
||
| 2010/04/16 |
Effort |
|
| Gan, J.O., Walton, M.E., and Phillips, P.E.M., Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine. Nature Neuroscience, 2009. 13: p. 25-27. | Laura (Boris) | |
| Croxson, P.L., Walton, M.E., O'Reilly, J.X., Behrens, T.E.J., Rushworth, M.F.S., Effort-based cost-benefit valuation and the human brain. Journal of Neuroscience, 2009. 29: p. 4531-4541. | Boris (Laura, Britton) | |
|
Background Reading for Those Who are Interested: Scherberger, H., M.A. Goodale, and R.A. Andersen, Target selection for reaching and saccades share a similar behavioral reference frame in the macaque. Journal of Neurophysiology, 2003. 89: p. 1456-1466. Scherberger, H.a.A., R.A., Target selection signals for arm reaching in the posterior parietal cortex. J. Neurosci., 2007. 27: p. 2001-2012. |
||
| 2010/04/22 |
Value |
|
| Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. SCIENCE Volume: 275 Issue: 5306 Pages: 1593-1599 Published: MAR 14 1997 | Boris (Laura, Alison) | |
| O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. SCIENCE Volume: 304 Issue: 5669 Pages: 452-454 Published: APR 16 2004 | Laura (Boris, Britton) | |
|
Background Reading for Those Who are Interested: Sutton & Barto, “Reinforcement Learning: An Introduction”. Section 6. |
||
| 2010/04/29 |
Goods and Uncertainty |
|
| Padoa-Schioppa, C. and J.A. Assad, Neurons in the orbitofrontal cortex encode economic value Nature, 2006. 441: p. 223-226. | Alison (Laura, Boris) | |
| Kennerley SW, Dahmubed AF, Lara AH, Wallis JD. Neurons in the Frontal Lobe Encode the Value of Multiple Decision Variables | Ben (Britton, Laura) | |
|
Background Reading for Those Who are Interested: Padoa-Schioppa, C., Assad, J.A., The representation of economic value in the orbitofrontal cortex is invariant to changes in menus. Nature Neuroscience, 2008. 11: p. 95-109. |
||
| 2010/05/06 |
Internal Models |
|
| Wolpert DM, Goodbody SJ, Husain M Source: NATURE NEUROSCIENCE . Maintaining internal representations the role of the human superior parietal lobe . Volume: 1 Issue: 6 Pages: 529-533 Published: OCT 1998 May 16 Affordance competition | Alison (Britton, Ben) | |
| Mulliken GH (Mulliken, Grant H.), Musallam S, Andersen RA. Forward estimation of movement state in posterior parietal cortex. PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA Volume: 105 Issue: 24 Pages: 8170-8177 Published: JUN 17 2008 | Britton (Ben, Alison) | |
|
Background Reading for Those Who are Interested: Haruno M, Wolpert DM, Kawato M . MOSAIC model for sensorimotor learning and control. NEURAL COMPUTATION Volume: 13 Issue: 10 Pages: 2201-2220 Published: OCT 2001 |
||
| 2010/05/13 |
Affordance competition |
|
| Cisek, P., Integrated neural processes for defining potential actions and deciding between them: a computational model. Journal of Neuroscience, 2006. 26: p. 9761-9770. | Ben (Boris) | |
| Rickert, J., Riehle, A., Aertsen, A., Rotter, S., Nawrot, M.P., Dynamic encoding of movement direction in motor cortical neurons. Journal of Neuroscience, 2009. 29: p. 13870-13882. | Britton (Boris, Ben) | |
|
Background Reading for Those Who are Interested: Cui, H. and R.A. Andersen, Posterior parietal cortex encodes autonomously selected motor plans. Neuron, 2007. 56(3): p. 552-9. |
||
| 2010/05/20 |
Accumulators and race models |
|
| Huk, A.C., Shadlen, M.N., Neural activity in macaque parietal cortex reflects temporal integration of visual motion signals during perceptual decision making. Journal of Neuroscience, 2005. 25: p. 10420-10436. | Boris (Britton, Ben) | |
| lo, C.C., Boucher, L., Pare, M., Schall, J.D., Wang, X.J., Proactive inhibitory control and attractor dynamics in countermanding action: a spiking neural circuit model. Journal of Neuroscience, 2009. 29: p. 9059-0971. | Britton (Boris, Ben) | |
| 2010/05/27 |
Action value |
|
| Wunderlich, K., Rangel, A., O'Doherty, P.P., Neural computations underlying action-based decision making in the human brain. PNAS, 2009. 106: p. 17199-17204. | Alison (Britton, Laura) | |
| Lau, B., and Glimcher, P.W., Value representations in the primate striatum during matching behavior. Neuron, 2008. 58: p. 451-463. | Boris (Laura, Ben) | |
|
Background Reading for Those Who are Interested: Samejima K, Ueda Y, Doya K, Kimura M. Representation of action-specific reward values in the striatum. SCIENCE Volume: 310 Issue: 5752 Pages: 1337-1340 Published: NOV 25 2005 |
||
| 2010/06/03 |
Reinforcement learning and subjective desirability |
|
| Seo, H., Barraclough, D.J., Lee, D., Lateral intraparietal cortex and reinforcement learning during a mixed-strategy game. Journal of Neuroscience, 2009. 22: p. 7278-7289. | Ben (Britton, Laura) | |
| Dorris, M.C. and P.W. Glimcher, Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron, 2004. 44: p. 365-378. | Laura (Britton, Ben) | |
|
Background Reading for Those Who are Interested: Barraclough, D.J., M.L. Conroy, and D. Lee, Prefrontal cortex and decision making in a mixed-strategy game. Nat Neurosci 2004. 7: p. 404-410. Sugrue LP, Corrado GS, Newsome WT. Matching behavior and the representation of value in the parietal cortex. Source: SCIENCE Volume: 304 Issue: 5678 Pages: 1782-1787 Published: JUN 18 2004 |