Bagging Ensemble Selection - a new ensemble learning strategy

When to use Bagging Ensemble Selection?

We have two papers published on the Bagging Ensemble Selection algorithm for both Classification and Regression problems.

The BESTrees algorithm

A tree-ensemble-based BES implementation, which is relatively fast and supports multi-cores (using default parameters and 4 cores, the algorithm took 1 min to train on a dataset with 100,000 instances and 100 attributes on an AMD 2.8G PC)

The user can choose a target evaluation metric (out of 9) to optimize. For regression problems, the algorithm supports the NNLS ensemble pruning strategy.

NNLS (Non-negative least-squares) WEKA source code
BESTrees can be downloaded from:

How to install BESTrees? (for Java 1.6+ and WEKA 3.7.7+)

Download WEKA (I have tested WEKA 3.7.7, which works fine) for your system from:
Open the WEKA package manager, click the "File/URL" button, and type:

BESTrees - A WEKA-based BES-OOB-nnls Implementation

After installation you will see the BESTrees algorithm displayed on the list

BESTrees - A WEKA-based BES-OOB-nnls Implementation

Use BESTrees in WEKA

BESTrees - A WEKA-based BES-OOB-nnls Implementation
BESTrees supports 9 metrics.
BESTrees - A WEKA-based BES-OOB-nnls Implementation


What should I do if the default parameter setting doesn't work well on my dataset?

Try to set the 'numFeatures' option to the total number of features of your dataset, or set it to a very large number, e.g., 99999, the base learner (REPTree) of BESTrees will use all features automatically.

Try to increase the values for 'numBags' and 'numTreesPerBag', e.g., 30 and 100, respectively.

Any questions?

Please feel free to contact me if you have any questions:

Last updated 28/01/2013