Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

Basal Ganglia Involvement in the Reinforcement Learning of Physical and Cognitive Actions

Laurent, Patryk Alix (2010) Basal Ganglia Involvement in the Reinforcement Learning of Physical and Cognitive Actions. Doctoral Dissertation, University of Pittsburgh. (Unpublished)

[img]
Preview
PDF
Primary Text

Download (6MB) | Preview

Abstract

Work is presented aimed at understanding the function of the basal ganglia in reward-related learning. Behavioral, fMRI, and computational techniques are used to examine basal ganglia activation during the reinforcement of physical (i.e., motoric) and cognitive (i.e., non-motoric) actions. In a single experiment design, participants received positive and negative reinforcement for performing actions in one of four possible directions depending on a color cue stimulus. During different phases of the experiment, participants performed either hand movements, eye movements, or covert attention shifts. Behavioral and fMRI data collected during the task were used to test predictions from simulated Reinforcement Learning (RL) agents trained on the same sequences of stimulus, action, and outcome experienced by the human participants. Behavioral data showed that participants were able to learn the three types of action equally well and at similar rates, providing behavioral evidence that a common algorithm might be involved. Further, RL simulations fit the learning of the three types of action equally well, suggesting that RL might be that common algorithm. A deconvolution analysis of striatal fMRI BOLD data suggested that: (1) the striatum computed reward prediction errors for both physical and cognitive actions, and (2) this computation was localized to different regions of the striatum depending on the type of action that was being rewarded. The localization of these computations replicates prior findings implicating those regions in action-specific voluntary control, but extends them to include the fact that these regions compute action-specific reward-prediction errors. Together, these data suggest that distinct circuits linking neocortex with the basal ganglia are involved in RL-related computations for the actions controlled by those circuits. The results point to a theoretical framework in which the basal ganglia mediate the reinforcement of actions whose control is delegated to local neocortical regions. Implications for theories of learning, theories of cognitive control, and mapping RL to the basal ganglia are discussed. Finally, the possibility that phasic dopamine might mediate the prediction error signals is considered in view of some theoretical challenges such as its non-specificity, its asymmetric representation of negative reinforcement, and its presence during non-rewarding events.


Share

Citation/Export:
Social Networking:
Share |

Details

Item Type: University of Pittsburgh ETD
Status: Unpublished
Creators/Authors:
CreatorsEmailPitt UsernameORCID
Laurent, Patryk Alixfloat@pakl.net
ETD Committee:
TitleMemberEmail AddressPitt UsernameORCID
Committee ChairReichle, Erik Dreichle@pitt.eduREICHLE
Committee MemberRedish, A Davidredish@umn.edu
Committee MemberFiez, Julie Afiez@pitt.eduFIEZ
Committee MemberSommer, Marc Amas@cnbc.cmu.edu
Committee MemberWheeler, Mark Ewheeler.me@gmail.com
Committee MemberStrick, Peter Lstrickp@pitt.eduSTRICKP
Date: 21 June 2010
Date Type: Completion
Defense Date: 13 November 2009
Approval Date: 21 June 2010
Submission Date: 21 April 2010
Access Restriction: 5 year -- Restrict access to University of Pittsburgh for a period of 5 years.
Institution: University of Pittsburgh
Schools and Programs: Dietrich School of Arts and Sciences > Neuroscience
Degree: PhD - Doctor of Philosophy
Thesis Type: Doctoral Dissertation
Refereed: Yes
Uncontrolled Keywords: attention; basal ganglia; cognition; dopamine; reinforcement learning; saccades
Other ID: http://etd.library.pitt.edu/ETD/available/etd-04212010-142229/, etd-04212010-142229
Date Deposited: 10 Nov 2011 19:40
Last Modified: 15 Nov 2016 13:41
URI: http://d-scholarship.pitt.edu/id/eprint/7446

Metrics

Monthly Views for the past 3 years

Plum Analytics


Actions (login required)

View Item View Item