+ Site Statistics
+ Search Articles
+ PDF Full Text Service
How our service works
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ Translate
+ Recently Requested

Using reinforcement learning to understand the emergence of "intelligent" eye-movement behavior during reading



Using reinforcement learning to understand the emergence of "intelligent" eye-movement behavior during reading



Psychological Review 113(2): 390-408



The eye movements of skilled readers are typically very regular (K. Rayner, 1998). This regularity may arise as a result of the perceptual, cognitive, and motor limitations of the reader (e.g., limited visual acuity) and the inherent constraints of the task (e.g., identifying the words in their correct order). To examine this hypothesis, reinforcement learning was used to allow an artificial "agent" to learn to move its eyes to read as efficiently as possible. The resulting patterns of simulated eye movements resembled those of skilled readers and suggest that important aspects of eye-movement behavior might emerge as a consequence of satisfying the constraints that are imposed on readers. These results also suggest novel interpretations of some contentious empirical results, such as the fixation duration costs associated with word skipping (R. Kliegl & R. Engbert, 2005), and theoretical assumptions, for example the familiarity check in the E-Z Reader model of eye-movement control (E. D. Reichle, A. Pollatsek, D. L. Fisher, & K. Rayner, 1998).

Please choose payment method:






(PDF emailed within 0-6 h: $19.90)

Accession: 050918605

Download citation: RISBibTeXText

PMID: 16637766

DOI: 10.1037/0033-295x.113.2.390


Related references

An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning. Frontiers in Computational Neuroscience 8: 47, 2014

An intelligent mobile vehicle navigator based on fuzzy logic and reinforcement learning. IEEE Transactions on Systems Man and Cybernetics. Part B Cybernetics 29(2): 314-321, 1999

Simulation of rat behavior by a reinforcement learning algorithm in consideration of appearance probabilities of reinforcement signals. Bio Systems 80(1): 83-90, 2005

Reading the tea leaves in an intelligent Coffee Corner: challenges for understanding behavior. Behavior Research Methods 41(3): 820-826, 2009

Using reinforcement learning to examine dynamic attention allocation during reading. Cognitive Science 37(8): 1507-1540, 2014

Integrating reinforcement learning, equilibrium points, and minimum variance to understand the development of reaching: a computational model. Psychological Review 121(3): 389-421, 2014

The rules of information aggregation and emergence of collective intelligent behavior. Topics in Cognitive Science 1(4): 598-620, 2009

Using epsilon-greedy reinforcement learning methods to further understand ventromedial prefrontal patients' deficits on the Iowa Gambling Task. Neural Networks 20(6): 676-689, 2007

The emergence of saliency and novelty responses from Reinforcement Learning principles. Neural Networks 21(10): 1493-1499, 2008

Credit assignment during movement reinforcement learning. Plos one 8(2): E55352, 2013

Credit assignment in movement-dependent reinforcement learning. Proceedings of the National Academy of Sciences of the United States of America 113(24): 6797-6802, 2016

From Creatures of Habit to Goal-Directed Learners: Tracking the Developmental Emergence of Model-Based Reinforcement Learning. Psychological Science 27(6): 848-858, 2016

How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. European Journal of Neuroscience 35(7): 1024-1035, 2012

Intelligent thought: science versus the intelligent design movement. Unknown, 2006

Reinforcement learning of targeted movement in a spiking neuronal model of motor cortex. Plos one 7(10): E47251, 2012