Getting Started

Tuning an optimization algorithm under multiple OFE budgets using optTune consists of three parts:

1. setting up the tuning problem
2. applying the tuning algorithm
3. analysing the results

Setting up the tuning problem

optTune runs the optimization algorithm using the CPV tuple being assessed as to gauge performance at multiple OFE budgets. The assessment function for tMOPSO is in the form of

optAlg(CPV_tuple, OFE_budgets, randomSeed):

Parameters:

• CPV_tuple (numpy.array) – CPV_tuple
• OFE_budgets (numpy.array) – OFE_budgets at which to record utility (i.e. solution error)
• randomSeed (int) – seed to be used for optimization run

Returns:

Two lists : [ Utility_values, OFE_eval_made ]

Other factors such as the tuning bounds, OFE budgets to tune under, resampling size, also need to be decided upon.

Example setup for tuning Differential Evolution ( DE ; see DE Code ) to the gerneralised Rossenbrock function.

from DE_code import DE_opt, numpy
from optTune import get_F_vals_at_specified_OFE_budgets

def Ros_ND(x) :
    "gerneralised Rossenbrock function"
    return sum([100*(x[ii+1]-x[ii]**2)**2 + (1-x[ii])**2 for ii in range(len(x)-1)])
prob_d = 6

def run_DE_on_Ros_ND(CPV_tuple, OFE_budgets, randomSeed):
    X_min, f_best_hist, X_hist, F_hist = DE_opt(
        objfun = Ros_ND,
        x_lb = -5.0 * numpy.ones(prob_d),
        x_ub =  5.0 * numpy.ones(prob_d),
        Np = int(CPV_tuple[0]),
        Cr = CPV_tuple[1],
        F = CPV_tuple[2],
        evals = max(OFE_budgets),
        printLevel=0
        )
    F =  numpy.array(f_best_hist)
    OFEs_made = int(CPV_tuple[0])*numpy.arange(1,len(X_hist)+1)
    return  get_F_vals_at_specified_OFE_budgets(F, OFEs_made, OFE_budgets)

def CPV_validity_checks(CPV_array, OFE_budget):
    'check tuning constraints'
    N, Cr, F = CPV_array
    if OFE_budget < N :
        return False, 'OFE_budget < N'
    if N < 5:
        return False, 'N < 5'
    if Cr < 0 or Cr > 1 :
        return False, 'Cr not in [0,1]'
    if F < 0:
        return False, 'F < 0'
    return True, ""

#initilization bounds
CPV_lb = numpy.array([  5, 0.0, 0.0 ]) 
CPV_ub = numpy.array([ 50, 1.0, 1.0 ]) 

OFE_budgets_to_tune_under = numpy.logspace(1,3,30).astype(int)

sampleSizes = [2,8,20]

tuningBudget = 50*1000*30 # tuning budget is equvialent of assessing 50 CPV tuples upto 1000 OFEs using 30 resampling runs each

Refer to the Quirks section as to why the get_F_vals_at_specified_OFE_budgets function is nessary.

Applying the tuning algorithm

After the tuning problem has been defined, then tuning algorithm can applied. Example:

import DE_tuning_setup 
from optTune import tMOPSO, linearFunction

tuningOpt = tMOPSO( 
    optAlg = DE_tuning_setup.run_DE_on_Ros_ND, 
    CPV_lb = DE_tuning_setup.CPV_lb, 
    CPV_ub = DE_tuning_setup.CPV_ub,
    CPV_validity_checks = DE_tuning_setup.CPV_validity_checks,
    OFE_budgets= DE_tuning_setup.OFE_budgets_to_tune_under,
    sampleSizes =  DE_tuning_setup.sampleSizes,
    gammaBudget = DE_tuning_setup.tuningBudget,
    )

Analysing the results

continuing with the tMOPSO tuning DE example

from matplotlib import pyplot

OFE_budgets = [ d.fv[0] for d in tuningOpt.PFA.designs ] 
Fmin_values = [ d.fv[1] for d in tuningOpt.PFA.designs ] 

log_OFE_budgets = [ d.xv[0] for d in tuningOpt.PFA.designs ]
N_values =        [ int(d.xv[1]) for d in tuningOpt.PFA.designs ]
Cr_values =       [ d.xv[2] for d in tuningOpt.PFA.designs ]
F_values =        [ d.xv[3] for d in tuningOpt.PFA.designs ]

line_Cr = pyplot.semilogx(OFE_budgets, Cr_values, 'b^')[0]
line_F  = pyplot.semilogx(OFE_budgets, F_values, 'rx')[0]
pyplot.ylabel('Cr, F')
pyplot.twinx()
line_N = pyplot.semilogx(OFE_budgets, N_values, 'go')[0]
pyplot.ylim( 0, max(N_values)*1.1)
pyplot.ylabel('N')
pyplot.legend([line_Cr,line_F,line_N], ['Cr','F','N'], loc='upper center')
pyplot.xlabel('OFE budget')
pyplot.xlim(min(OFE_budgets)-1,max(OFE_budgets)+60)
pyplot.title('Optimal CPVs for different OFE budgets')

pyplot.show()