+ Site Statistics
+ Search Articles
+ PDF Full Text Service
How our service works
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ Translate
+ Recently Requested

A neural network model of adaptively timed reinforcement learning and hippocampal dynamics



A neural network model of adaptively timed reinforcement learning and hippocampal dynamics



Brain Research. Cognitive Brain Research 1(1): 3-38



A neural model is described of how adaptively timed reinforcement learning occurs. The adaptive timing circuit is suggested to exist in the hippocampus, and to involve convergence of dentate granule cells on CA3 pyramidal cells, and N-methyl-D-aspartate (NMDA) receptors. This circuit forms part of a model neural system for the coordinated control of recognition learning, reinforcement learning, and motor learning, whose properties clarify how an animal can learn to acquire a delayed reward. Behavioral and neural data are summarized in support of each processing stage of the system. The relevant anatomical sites are in thalamus, neocortex, hippocampus, hypothalamus, amygdala and cerebellum. Cerebellar influences on motor learning are distinguished from hippocampal influences on adaptive timing of reinforcement learning. The model simulates how damage to the hippocampal formation disrupts adaptive timing, eliminates attentional blocking and causes symptoms of medial temporal amnesia. Properties of learned expectations, attentional focussing, memory search and orienting reactions to novel events are used to analyze the blocking and amnesia data. The model also suggests how normal acquisition of subcortical emotional conditioning can occur after cortical ablation, even though extinction of emotional conditioning is retarded by cortical ablation. The model simulates how increasing the duration of an unconditioned stimulus increases the amplitude of emotional conditioning, but does not change adaptive timing; and how an increase in the intensity of a conditioned stimulus 'speeds up the clock', but an increase in the intensity of an unconditioned stimulus does not. Computer simulations of the model fit parametric conditioning data, including a Weber law property and an inverted U property. Both primary and secondary adaptively timed conditionings are simulated, as are data concerning conditioning using multiple interstimulus intervals (ISIs), gradually or abruptly changing ISIs, partial reinforcement and multiple stimuli that lead to time-averaging of responses. Neurobiologically testable predictions are made to facilitate further tests of the model.

Please choose payment method:






(PDF emailed within 0-6 h: $19.90)

Accession: 048083546

Download citation: RISBibTeXText

PMID: 15497433

DOI: 10.1016/0926-6410(92)90003-a


Related references

Neural Dynamics of Autistic Repetitive Behaviors and Fragile X Syndrome: Basal Ganglia Movement Gating and mGluR-Modulated Adaptively Timed Learning. Frontiers in Psychology 9: 269, 2018

A neural model of normal and abnormal learning and memory consolidation: adaptively timed conditioning, hippocampus, amnesia, neurotrophins, and consciousness. Cognitive Affective and Behavioral Neuroscience 17(1): 24-76, 2017

Adaptively timed conditioned responses and the cerebellum a neural network approach. Society for Neuroscience Abstracts 15(1): 506, 1989

Synaptic plasticity model of a spiking neural network for reinforcement learning. Neurocomputing 71(13-15): 3037-3043, 2008

The Use of Artificial Neural Networks to Study Perception in Animals || A Neural-Network Reinforcement-Learning Model of Domestic Chicks That Learn to Localize the Centre of Closed Arenas. Philosophical Transactions Biological Sciences 362(1479): 383-401, 2007

A spiking neural network model of model-free reinforcement learning with high-dimensional sensory input and perceptual ambiguity. Plos one 10(3): E0115620, 2015

Reinforcement learning of songbird premotor representations in a spiking neural network model. Society for Neuroscience Abstract Viewer & Itinerary Planner : Abstract No 680 1, 2002

Learning of sequential movements by neural network model with dopamine-like reinforcement signal. Experimental Brain Research. 121(3): 350-354,., 1998

A model to explain the emergence of reward expectancy neurons using reinforcement learning and neural network. Neurocomputing 69(10-12): 1327-1331, 2006

A neural network model for the orbitofrontal cortex and task space acquisition during reinforcement learning. Plos Computational Biology 14(1): E1005925, 2018

A neural-network reinforcement-learning model of domestic chicks that learn to localize the centre of closed arenas. Philosophical Transactions of the Royal Society of London. Series B Biological Sciences 362(1479): 383-401, 2007

A neural network learning for adaptively extracting cross-correlation features between two high-dimensional data streams. IEEE Transactions on Neural Networks 15(6): 1541-1554, 2004

New neural multiprocess memory model for adaptively regulating associative learning. Neural Networks 7(9): 1351-1378, 1994

Simulation of spatial learning in the Morris water maze by a neural network model of the hippocampal formation and nucleus accumbens. Hippocampus 5(3): 171-188, 1995

The hippocampus and cerebellum in adaptively timed learning, recognition, and movement. Journal of Cognitive Neuroscience 8(3): 257-277, 1996