+ Site Statistics
+ Search Articles
+ PDF Full Text Service
How our service works
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ Translate
+ Recently Requested

Contrasting temporal difference and opportunity cost reinforcement learning in an empirical money-emergence paradigm



Contrasting temporal difference and opportunity cost reinforcement learning in an empirical money-emergence paradigm



Proceedings of the National Academy of Sciences of the United States of America 115(49): E11446



Money is a fundamental and ubiquitous institution in modern economies. However, the question of its emergence remains a central one for economists. The monetary search-theoretic approach studies the conditions under which commodity money emerges as a solution to override frictions inherent to interindividual exchanges in a decentralized economy. Although among these conditions, agents' rationality is classically essential and a prerequisite to any theoretical monetary equilibrium, human subjects often fail to adopt optimal strategies in tasks implementing a search-theoretic paradigm when these strategies are speculative, i.e., involve the use of a costly medium of exchange to increase the probability of subsequent and successful trades. In the present work, we hypothesize that implementing such speculative behaviors relies on reinforcement learning instead of lifetime utility calculations, as supposed by classical economic theory. To test this hypothesis, we operationalized the Kiyotaki and Wright paradigm of money emergence in a multistep exchange task and fitted behavioral data regarding human subjects performing this task with two reinforcement learning models. Each of them implements a distinct cognitive hypothesis regarding the weight of future or counterfactual rewards in current decisions. We found that both models outperformed theoretical predictions about subjects' behaviors regarding the implementation of speculative strategies and that the latter relies on the degree of the opportunity costs consideration in the learning process. Speculating about the marketability advantage of money thus seems to depend on mental simulations of counterfactual events that agents are performing in exchange situations.

Please choose payment method:






(PDF emailed within 0-6 h: $19.90)

Accession: 065739620

Download citation: RISBibTeXText

PMID: 30442672

DOI: 10.1073/pnas.1813197115


Related references

Hippocampal replay contributes to within session learning in a temporal difference reinforcement learning model. Neural Networks 18(9): 1163-1171, 2005

Reinforcement learning via kernel temporal difference. Conference Proceedings 2011: 5662-5665, 2011

Temporal-difference reinforcement learning with distributed representations. Plos one 4(10): E7362, 2009

Deterministic limit of temporal difference reinforcement learning for stochastic games. PhysicalReview.e99(4-1):043305, 2019

Reinforcement learning: computing the temporal difference of values via distinct corticostriatal pathways. Trends in Neurosciences 35(8): 457-467, 2012

Biological implementation of the temporal difference algorithm for reinforcement learning: theoretical comment on O'Reilly et al. (2007). Behavioral Neuroscience 121(1): 231-232, 2007

Integrating temporal difference methods and self-organizing neural networks for reinforcement learning with delayed evaluative feedback. IEEE Transactions on Neural Networks 19(2): 230-244, 2008

Reinforcement Learning: Computing the Temporal Difference of Values via Distinct Corticostriatal Pathways: (Trends in Neurosciences 35, 457-467; 2012). Trends in Neurosciences 40(7): 453, 2017

Properties of a temporal difference reinforcement learning brain machine interface driven by a simulated motor cortex. Conference Proceedings 2012: 3284-3287, 2012

Average cost temporal-difference learning. Automatica 35(11): 1799-1808, 1999

Does newborn screening save money? The difference between cost-effective and cost-saving interventions. Journal of Pediatrics 146(2): 168-170, 2005

A Reinforcement-Based Learning Paradigm Increases Anatomical Learning and Retention-A Neuroeducation Study. Frontiers in Human Neuroscience 12: 38, 2018

Time is money: opportunity cost and physicians' provision of charity care 1996-2005. Health Services Research 45(6 Pt 1): 1670-1692, 2010

Empirical priors for reinforcement learning models. Journal of Mathematical Psychology 71: 1-6, 2016

The opportunity cost of coastal land-use controls an empirical analysis. Land economics 67(3): 308-316, 1991