Part of Advances in Neural Information Processing Systems 11 (NIPS 1998)
A. Düring, Anthony Coolen, D. Sherrington
We solve the dynamics of Hopfield-type neural networks which store se(cid:173) quences of patterns, close to saturation. The asymmetry of the interaction matrix in such models leads to violation of detailed balance, ruling out an equilibrium statistical mechanical analysis. Using generating functional methods we derive exact closed equations for dynamical order parame(cid:173) ters, viz. the sequence overlap and correlation and response functions. in the limit of an infinite system size. We calculate the time translation invariant solutions of these equations. describing stationary limit-cycles. which leads to a phase diagram. The effective retarded self-interaction usually appearing in symmetric models is here found to vanish, which causes a significantly enlarged storage capacity of eYe ~ 0.269. com(cid:173) pared to eYe ~ 0.139 for Hopfield networks s~oring static patterns. Our results are tested against extensive computer simulations and excellent agreement is found.
212
A. Diiring, A. C. C. Coo/en and D. Sherrington
1 INTRODUCTION AND DEFINITIONS
We consider a system of N neurons O'(t) = {ai(t) = ±1}, which can change their states collectively at discrete times (parallel dynamics). Each neuron changes its state with a probability Pi(t) = ~[l-tanh,Bai(t)[Lj Jijaj(t)+Oi(t)]], so that the transition matrix is W[o'(s + l)IO'(s)] = II e.BO',(s+l)[E;=l J, j O'} (s)+ o,( s)]-ln2cosh(i3[E; =1 J'JO'} ( s)+(J, (s )))