Sosis, Baram
(2024)
A Mathematical Analysis of Effective Learning and Decision-Making in Neuronal Circuits.
Doctoral Dissertation, University of Pittsburgh.
(Unpublished)
Abstract
Learning and decision-making are fundamental aspects of cognition, but much remains unknown about how they are accomplished by the brain. We investigate these topics through the framework of effective task performance. We first study action selection on forced choice tasks, in which it is generally assumed that agents attempt to maximize reward rate. We compare the standard formulation of reward rate, expected reward divided by expected time or E[R]/E[T], to an alternative formulation, the expectation of reward divided by time or E[R/T], where R denotes reward size and T denotes decision time. Both theoretical and empirical results suggest that E[R/T] may in many cases better describe behavior. We derive a formula for E[R/T] in the context of drift-diffusion models and find parameter regimes where it differs from the classical formula. These results may provide a new lens through which to analyze experimental data related to decision-making.
Next, we investigate how the learning mechanisms present in the basal ganglia impact task performance in a variety of settings. Corticostriatal synapses, which serve as the primary input to the basal ganglia, undergo spike-timing-dependent plasticity (STDP) modulated by dopamine. We introduce three forms of dopamine-modulated STDP and analyze, both mathematically and with simulations, their performance in several biologically relevant scenarios. We find that each plasticity rule is well-suited to some of the scenarios studied but falls short in others. This demonstrates that different tasks require different forms of synaptic plasticity, and suggests that regions of the striatum -- or other brain areas impacted by dopamine -- with distinct computational functions may show variations in the STDP mechanisms they implement.
Share
| Citation/Export: |
|
| Social Networking: |
|
Details
| Item Type: |
University of Pittsburgh ETD
|
| Status: |
Unpublished |
| Creators/Authors: |
|
| ETD Committee: |
|
| Date: |
20 December 2024 |
| Date Type: |
Publication |
| Defense Date: |
22 November 2024 |
| Approval Date: |
20 December 2024 |
| Submission Date: |
6 December 2024 |
| Access Restriction: |
No restriction; Release the ETD for access worldwide immediately. |
| Number of Pages: |
152 |
| Institution: |
University of Pittsburgh |
| Schools and Programs: |
Dietrich School of Arts and Sciences > Mathematics |
| Degree: |
PhD - Doctor of Philosophy |
| Thesis Type: |
Doctoral Dissertation |
| Refereed: |
Yes |
| Uncontrolled Keywords: |
Reward prediction, Action selection, Speed-accuracy tradeoff, Drift diffusion model, Dopamine, Synaptic plasticity, STDP, Basal ganglia |
| Date Deposited: |
20 Dec 2024 14:44 |
| Last Modified: |
20 Dec 2024 14:44 |
| URI: |
http://d-scholarship.pitt.edu/id/eprint/47171 |
Metrics
Monthly Views for the past 3 years
Plum Analytics
Actions (login required)
 |
View Item |
![[feed]](/images/feed-icon-32x32.png){kind=icon}