Laurent, Patryk Alix
(2010)
Basal Ganglia Involvement in the Reinforcement Learning of Physical and Cognitive Actions.
Doctoral Dissertation, University of Pittsburgh.
(Unpublished)
Abstract
Work is presented aimed at understanding the function of the basal ganglia in reward-related learning. Behavioral, fMRI, and computational techniques are used to examine basal ganglia activation during the reinforcement of physical (i.e., motoric) and cognitive (i.e., non-motoric) actions. In a single experiment design, participants received positive and negative reinforcement for performing actions in one of four possible directions depending on a color cue stimulus. During different phases of the experiment, participants performed either hand movements, eye movements, or covert attention shifts. Behavioral and fMRI data collected during the task were used to test predictions from simulated Reinforcement Learning (RL) agents trained on the same sequences of stimulus, action, and outcome experienced by the human participants. Behavioral data showed that participants were able to learn the three types of action equally well and at similar rates, providing behavioral evidence that a common algorithm might be involved. Further, RL simulations fit the learning of the three types of action equally well, suggesting that RL might be that common algorithm. A deconvolution analysis of striatal fMRI BOLD data suggested that: (1) the striatum computed reward prediction errors for both physical and cognitive actions, and (2) this computation was localized to different regions of the striatum depending on the type of action that was being rewarded. The localization of these computations replicates prior findings implicating those regions in action-specific voluntary control, but extends them to include the fact that these regions compute action-specific reward-prediction errors. Together, these data suggest that distinct circuits linking neocortex with the basal ganglia are involved in RL-related computations for the actions controlled by those circuits. The results point to a theoretical framework in which the basal ganglia mediate the reinforcement of actions whose control is delegated to local neocortical regions. Implications for theories of learning, theories of cognitive control, and mapping RL to the basal ganglia are discussed. Finally, the possibility that phasic dopamine might mediate the prediction error signals is considered in view of some theoretical challenges such as its non-specificity, its asymmetric representation of negative reinforcement, and its presence during non-rewarding events.
Share
| Citation/Export: |
|
| Social Networking: |
|
Details
| Item Type: |
University of Pittsburgh ETD
|
| Status: |
Unpublished |
| Creators/Authors: |
|
| ETD Committee: |
|
| Date: |
21 June 2010 |
| Date Type: |
Completion |
| Defense Date: |
13 November 2009 |
| Approval Date: |
21 June 2010 |
| Submission Date: |
21 April 2010 |
| Access Restriction: |
5 year -- Restrict access to University of Pittsburgh for a period of 5 years. |
| Institution: |
University of Pittsburgh |
| Schools and Programs: |
Dietrich School of Arts and Sciences > Neuroscience |
| Degree: |
PhD - Doctor of Philosophy |
| Thesis Type: |
Doctoral Dissertation |
| Refereed: |
Yes |
| Uncontrolled Keywords: |
attention; basal ganglia; cognition; dopamine; reinforcement learning; saccades |
| Other ID: |
http://etd.library.pitt.edu/ETD/available/etd-04212010-142229/, etd-04212010-142229 |
| Date Deposited: |
10 Nov 2011 19:40 |
| Last Modified: |
15 Nov 2016 13:41 |
| URI: |
http://d-scholarship.pitt.edu/id/eprint/7446 |
Metrics
Monthly Views for the past 3 years
Plum Analytics
Actions (login required)
 |
View Item |
![[feed]](/images/feed-icon-32x32.png){kind=icon}