Part of Advances in Neural Information Processing Systems 2 (NIPS 1989)
Conrad Galland, Geoffrey E. Hinton
A new form of the deterministic Boltzmann machine (DBM) learn(cid:173) ing procedure is presented which can efficiently train network mod(cid:173) ules to discriminate between input vectors according to some cri(cid:173) terion. The new technique directly utilizes the free energy of these "mean field modules" to represent the probability that the criterion is met, the free energy being readily manipulated by the learning procedure. Although conventional deterministic Boltzmann learn(cid:173) ing fails to extract the higher order feature of shift at a network bottleneck, combining the new mean field modules with the mu(cid:173) tual information objective function rapidly produces modules that perfectly extract this important higher order feature without direct external supervision.