+ Site Statistics
+ Search Articles
+ PDF Full Text Service
How our service works
Request PDF Full Text
+ Follow Us
Follow on Facebook
Follow on Twitter
Follow on LinkedIn
+ Subscribe to Site Feeds
Most Shared
PDF Full Text
+ Translate
+ Recently Requested

Temporal-kernel recurrent neural networks

Temporal-kernel recurrent neural networks

Neural Networks 23(2): 239-243

A Recurrent Neural Network (RNN) is a powerful connectionist model that can be applied to many challenging sequential problems, including problems that naturally arise in language and speech. However, RNNs are extremely hard to train on problems that have long-term dependencies, where it is necessary to remember events for many timesteps before using them to make a prediction. In this paper we consider the problem of training RNNs to predict sequences that exhibit significant long-term dependencies, focusing on a serial recall task where the RNN needs to remember a sequence of characters for a large number of steps before reconstructing it. We introduce the Temporal-Kernel Recurrent Neural Network (TKRNN), which is a variant of the RNN that can cope with long-term dependencies much more easily than a standard RNN, and show that the TKRNN develops short-term memory that successfully solves the serial recall task by representing the input string with a stable state of its hidden units.

Please choose payment method:

(PDF emailed within 0-6 h: $19.90)

Accession: 056143285

Download citation: RISBibTeXText

PMID: 19932002

DOI: 10.1016/j.neunet.2009.10.009

Related references

Spatio-Temporal Dynamics of Intrinsic Networks in Functional Magnetic Imaging Data Using Recurrent Neural Networks. Frontiers in Neuroscience 12: 600, 2018

Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks. Biological Cybernetics 106(4-5): 201-217, 2012

Improving EEG-Based Motor Imagery Classification via Spatial and Temporal Recurrent Neural Networks. Conference Proceedings 2018: 1903-1906, 2018

Non-linear autoregressive modelling by Temporal Recurrent Neural Networks for the prediction of freshwater phytoplankton dynamics. Ecological Modelling 211(3-4): 292-300, 2008

Deep Recurrent Neural Networks for Prostate Cancer Detection: Analysis of Temporal Enhanced Ultrasound. IEEE Transactions on Medical Imaging 37(12): 2695-2703, 2018

Recurrent neural networks for predicting outcomes after liver transplantation: representing temporal sequence of clinical observations. Methods of Information in Medicine 40(5): 386-391, 2001

Recurrent Neural Networks for Predicting Outcomes after Liver Transplantation: Representing Temporal Sequence of Clinical Observations. Methods of Information in Medicine 40(5): 386-391, 2001

Protein secondary structure prediction improved by recurrent neural networks integrated with two-dimensional convolutional neural networks. Journal of Bioinformatics and Computational Biology 16(5): 1850021, 2018

Spatio-temporal prediction of crop disease severity for agricultural emergency management based on recurrent neural networks. Geoinformatica 22(2): 363-381, 2018

Learning Spatial-Spectral-Temporal EEG Features With Recurrent 3D Convolutional Neural Networks for Cross-Task Mental Workload Assessment. IEEE Transactions on Neural Systems and Rehabilitation Engineering 27(1): 31-42, 2019

Identification of Temporal Transition of Functional States Using Recurrent Neural Networks from Functional MRI. Medical Image Computing and Computer-Assisted Intervention 11072: 232-239, 2018

Estimating Missing Data in Temporal Data Streams Using Multi-Directional Recurrent Neural Networks. IEEE Transactions on Bio-Medical Engineering 66(5): 1477-1490, 2019

Land cover classification from multi-temporal, multi-spectral remotely sensed imagery using patch-based recurrent neural networks. Neural Networks 105: 346-355, 2018

Construction of Gene Regulatory Networks Using Recurrent Neural Networks and Swarm Intelligence. Scientifica 2016: 1060843, 2016

Fractional Hopfield Neural Networks: Fractional Dynamic Associative Recurrent Neural Networks. IEEE Transactions on Neural Networks and Learning Systems 28(10): 2319-2333, 2017