+ Site Statistics
+ Search Articles
+ PDF Full Text Service
How our service works
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ Translate
+ Recently Requested

How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis

How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis

European Journal of Neuroscience 35(7): 1024-1035

Instrumental learning involves corticostriatal circuitry and the dopaminergic system. This system is typically modeled in the reinforcement learning (RL) framework by incrementally accumulating reward values of states and actions. However, human learning also implicates prefrontal cortical mechanisms involved in higher level cognitive functions. The interaction of these systems remains poorly understood, and models of human behavior often ignore working memory (WM) and therefore incorrectly assign behavioral variance to the RL system. Here we designed a task that highlights the profound entanglement of these two processes, even in simple learning problems. By systematically varying the size of the learning problem and delay between stimulus repetitions, we separately extracted WM-specific effects of load and delay on learning. We propose a new computational model that accounts for the dynamic integration of RL and WM processes observed in subjects' behavior. Incorporating capacity-limited WM into the model allowed us to capture behavioral variance that could not be captured in a pure RL framework even if we (implausibly) allowed separate RL systems for each set size. The WM component also allowed for a more reasonable estimation of a single RL process. Finally, we report effects of two genetic polymorphisms having relative specificity for prefrontal and basal ganglia functions. Whereas the COMT gene coding for catechol-O-methyl transferase selectively influenced model estimates of WM capacity, the GPR6 gene coding for G-protein-coupled receptor 6 influenced the RL learning rate. Thus, this study allowed us to specify distinct influences of the high-level and low-level cognitive functions on instrumental learning, beyond the possibilities offered by simple RL models.

Please choose payment method:

(PDF emailed within 0-6 h: $19.90)

Accession: 036374419

Download citation: RISBibTeXText

PMID: 22487033

DOI: 10.1111/j.1460-9568.2011.07980.x

Related references

Working Memory and Reinforcement Schedule Jointly Determine Reinforcement Learning in Children: Potential Implications for Behavioral Parent Training. Frontiers in Psychology 9: 394, 2018

An integrated computational model of dopamine function in reinforcement learning and working memory. Society for Neuroscience Abstracts 23(1-2): 775, 1997

Learning to use working memory: a reinforcement learning gating model of rule acquisition in rats. Frontiers in Computational Neuroscience 6: 87, 2012

Modeling choice and reaction time during arbitrary visuomotor learning through the coordination of adaptive working memory and reinforcement learning. Frontiers in Behavioral Neuroscience 9: 225, 2015

Externalizing psychopathology and behavioral disinhibition: working memory mediates signal discriminability and reinforcement moderates response bias in approach-avoidance learning. Journal of Abnormal Psychology 120(2): 336-351, 2011

Working memory contributions to reinforcement learning impairments in schizophrenia. Journal of Neuroscience 34(41): 13747-13756, 2014

The Tortoise and the Hare: Interactions between Reinforcement Learning and Working Memory. Journal of Cognitive Neuroscience 30(10): 1422-1432, 2018

Somatosensory working memory in human reinforcement-based motor learning. Journal of Neurophysiology 120(6): 3275-3286, 2018

Reinforcement Learning with Limited Reinforcement: Using Bayes Risk for Active Learning in POMDPs. Proceedings of the .. International Conference on Machine Learning. International Conference on Machine Learning 301: 256-263, 2008

A biologically plausible computational model of meta learning in reinforcement learning. Society for Neuroscience Abstract Viewer & Itinerary Planner : Abstract No 283 20, 2002

Beyond simple reinforcement learning: the computational neurobiology of reward-learning and valuation. European Journal of Neuroscience 35(7): 987-990, 2012

Within- and across-trial dynamics of human EEG reveal cooperative interplay between reinforcement learning and working memory. Proceedings of the National Academy of Sciences of the United States of America 115(10): 2502-2507, 2018

Linking Individual Learning Styles to Approach-Avoidance Motivational Traits and Computational Aspects of Reinforcement Learning. Plos one 11(11): E0166675, 2016

Interactions Among Working Memory, Reinforcement Learning, and Effort in Value-Based Choice: A New Paradigm and Selective Deficits in Schizophrenia. Biological Psychiatry 82(6): 431-439, 2017

Dopamine-mediated learning and switching in cortico-striatal circuit explain behavioral changes in reinforcement learning. Frontiers in Behavioral Neuroscience 5: 15, 2011