Part of Advances in Neural Information Processing Systems 10 (NIPS 1997)
Volker Tresp, Thomas Briegel
We consider neural network models for stochastic nonlinear dynamical systems where measurements of the variable of interest are only avail(cid:173) able at irregular intervals i.e. most realizations are missing. Difficulties arise since the solutions for prediction and maximum likelihood learn(cid:173) ing with missing data lead to complex integrals, which even for simple cases cannot be solved analytically. In this paper we propose a spe(cid:173) cific combination of a nonlinear recurrent neural predictive model and a linear error model which leads to tractable prediction and maximum likelihood adaptation rules. In particular, the recurrent neural network can be trained using the real-time recurrent learning rule and the linear error model can be trained by an EM adaptation rule, implemented us(cid:173) ing forward-backward Kalman filter equations. The model is applied to predict the glucose/insulin metabolism of a diabetic patient where blood glucose measurements are only available a few times a day at irregular intervals. The new model shows considerable improvement with respect to both recurrent neural networks trained with teacher forcing or in a free running mode and various linear models.