DOE OSTI2013
Since their inception, additive ensembles have been shown to provide a highly effective and robust frame-work for a variety of classification and prediction tasks. Their successful application in a practical setting, however, usually depends on specifying a priori several operational parameters that can be difficult or time-consuming to estimate. In particular, the number of constituent models (e.g., decision trees, logistic models) that comprise the ensemble induces a tradeoff between performance and efficiency that may not be readily characterized a priori. This dichotomy varies with several key factors (e.g., the nature of the underlying models, availability of training data, etc.) and is especially critical in contexts where memory is at a premium (e.g., limited hardware [19]) and/or computational speed is paramount (e.g., real-time stock trading, cybersecurity). We have frequently observed that the classification performance of the ensemble tends to plateau (i.e., achieve saturation) as the number of members in the ensemble (i.e., ensemble size) increases, so that eventually the marginal gain from adding further models falls below a desired tolerance. This observation suggests that superior performance may be achieved via overspecification of the ensemble size. In practice, however, this strategy can easily produce simultaneously redundant and overly complex ensembles at a significant expense to computational efficiency.
97 MATHEMATICS AND COMPUTING↗