+ Site Statistics
+ Search Articles
+ PDF Full Text Service
How our service works
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ Translate
+ Recently Requested

Novelty and Inductive Generalization in Human Reinforcement Learning



Novelty and Inductive Generalization in Human Reinforcement Learning



Topics in Cognitive Science 7(3): 391-415



In reinforcement learning (RL), a decision maker searching for the most rewarding option is often faced with the question: What is the value of an option that has never been tried before? One way to frame this question is as an inductive problem: How can I generalize my previous experience with one set of options to a novel option? We show how hierarchical Bayesian inference can be used to solve this problem, and we describe an equivalence between the Bayesian model and temporal difference learning algorithms that have been proposed as models of RL in humans and animals. According to our view, the search for the best option is guided by abstract knowledge about the relationships between different options in an environment, resulting in greater search efficiency compared to traditional RL algorithms previously applied to human cognition. In two behavioral experiments, we test several predictions of our model, providing evidence that humans learn and exploit structured inductive knowledge to make predictions about novel options. In light of this model, we suggest a new interpretation of dopaminergic responses to novelty.

Please choose payment method:






(PDF emailed within 0-6 h: $19.90)

Accession: 058434774

Download citation: RISBibTeXText

PMID: 25808176

DOI: 10.1111/tops.12138


Related references

The effect of novelty on reinforcement learning. Progress in Brain Research 202: 415-439, 2013

Negative evidence and inductive reasoning in generalization of associative learning. Journal of Experimental Psychology. General 148(2): 289-303, 2019

Generalization of value in reinforcement learning by humans. European Journal of Neuroscience 35(7): 1092-1104, 2012

The emergence of saliency and novelty responses from Reinforcement Learning principles. Neural Networks 21(10): 1493-1499, 2008

Generalization of secondary reinforcement in discrimination learning. Journal of Comparative and Physiological Psychology 47(4): 311-314, 1954

Conditioning parameter model for reinforcement generalization in probabilistic discrimination learning. Journal of Mathematical Psychology 3(1): 184-196, 1966

State generalization method with support vector machines in reinforcement learning. Systems and Computers in Japan 37(9): 77-86, 2006

Toward Generalization of Automated Temporal Abstraction to Partially Observable Reinforcement Learning. IEEE Transactions on Cybernetics 45(8): 1414-1425, 2015

Exploiting Generalization in the Subspaces for Faster Model-Based Reinforcement Learning. IEEE Transactions on Neural Networks and Learning Systems 30(6): 1635-1650, 2019

A study of teaching-learning principle based on generalization and abstraction The definition and properties of generalization and the relations of logic and generalization. Memoirs of the Faculty of Education Shiga University Natural Science & Pedagogic Science 0(45): 1-15, 1995

Association between locomotor response to novelty and light reinforcement: sensory reinforcement as a rodent model of sensation seeking. Behavioural Brain Research 230(2): 380-388, 2012

Grid cells, place cells, and geodesic generalization for spatial reinforcement learning. Plos Computational Biology 7(10): E1002235, 2011

Extinction following partial reinforcement with control of stimulus-generalization and secondary reinforcement. American Journal of Psychology 69(3): 359-368, 1956

How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. European Journal of Neuroscience 35(7): 1024-1035, 2012

Reinforcement Learning with Limited Reinforcement: Using Bayes Risk for Active Learning in POMDPs. Proceedings of the .. International Conference on Machine Learning. International Conference on Machine Learning 301: 256-263, 2008