Information, Prediction, and Query by Committee

Part of Advances in Neural Information Processing Systems 5 (NIPS 1992)

Authors

Yoav Freund, H. Sebastian Seung, Eli Shamir, Naftali Tishby

Abstract

We analyze the "query by committee" algorithm, a method for fil(cid:173) tering informative queries from a random stream of inputs. We show that if the two-member committee algorithm achieves infor(cid:173) mation gain with positive lower bound, then the prediction error decreases exponentially with the number of queries. We show that, in particular, this exponential decrease holds for query learning of thresholded smooth functions.