AdaBoost

Adaboost

This example is available as part of milk as milk/demos/adaboost.py.

Adaboost is based on a weak learner. For this example, we are going to use a stump learner:

import milk.supervised.tree
import milk.supervised.adaboost

weak = milk.supervised.tree.stump_learner()
learner = milk.supervised.adaboost.boost_learner(weak)

Currently, only binary classification is implemented for boost_learner. Therefore, we need to use a converter, in this case, using the one versus one strategy:

import milk.supervised.multi
learner = milk.supervised.multi.one_against_one(learner)

Now, we can use this learner as we would normally do. For example, for cross-validation:

from milksets import wine
features, labels = wine.load()
cmat,names,predictions = \
    milk.nfoldcrossvalidation(features, \
                                labels, \
                                classifier=learner, \
                                return_predictions=True)

We display just the first two dimensions here using circles for correct predictions and crosses for mis-matches. The colour represents the underlying class:

import pylab as plt
colors = "rgb"
codes = "xo"
for y,x,r,p in zip(features.T[0], features.T[1], labels, predictions):
    code = codes[int(r == p)]
    plt.plot([y],[x], colors[p]+code)
plt.show()

import pylab as plt
import milk.supervised.tree
import milk.supervised.adaboost
from milksets import wine
import milk.supervised.multi

weak = milk.supervised.tree.stump_learner()
learner = milk.supervised.adaboost.boost_learner(weak)
learner = milk.supervised.multi.one_against_one(learner)

features, labels = wine.load()
cmat,names,predictions = milk.nfoldcrossvalidation(features,labels, classifier=learner, return_predictions=True)
colors = "rgb"
codes = "xo"
for y,x,r,p in zip(features.T[0], features.T[1], labels, predictions):
    code = codes[int(r == p)]
    plt.plot([y],[x], colors[p]+code)
plt.show()

(Source code)

API Documentation

class milk.supervised.adaboost.boost_learner(weak_learner_type(), max_iters=100) model = learner.train(features, labels) test = model.apply(f)

AdaBoost learner

Attributes

base	(learner) Weak learner
max_iters	(integer) Nr of iterations (default: 100)

Methods