Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
, 11 (3), 697-709

New Results on Recurrent Network Training: Unifying the Algorithms and Accelerating Convergence

Affiliations

New Results on Recurrent Network Training: Unifying the Algorithms and Accelerating Convergence

A F Atiya et al. IEEE Trans Neural Netw.

Abstract

How to efficiently train recurrent networks remains a challenging and active research topic. Most of the proposed training approaches are based on computational ways to efficiently obtain the gradient of the error function, and can be generally grouped into five major groups. In this study we present a derivation that unifies these approaches. We demonstrate that the approaches are only five different ways of solving a particular matrix equation. The second goal of this paper is develop a new algorithm based on the insights gained from the novel formulation. The new algorithm, which is based on approximating the error gradient, has lower computational complexity in computing the weight update than the competing techniques for most typical problems. In addition, it reaches the error minimum in a much smaller number of iterations. A desirable characteristic of recurrent network training algorithms is to be able to update the weights in an on-line fashion. We have also developed an on-line version of the proposed algorithm, that is based on updating the error gradient approximation in a recursive manner.

Similar articles

See all similar articles

Cited by 10 PubMed Central articles

See all "Cited by" articles

LinkOut - more resources

Feedback