Navigation

Optunity API

optunity submodules

Key features

Standalone process

Optunity

Provides
  1. Routines to efficiently optimize hyperparameters
  2. Function decorators to implicitly log evaluations, constrain the domain and more.
  3. Facilities for k-fold cross-validation to estimate generalization performance.

Available modules

solvers
contains all officially supported solvers
functions
a variety of useful function decorators for hyperparameter tuning
cross_validation
k-fold cross-validation

Available subpackages

tests
regression test suite
solvers
solver implementations and auxillary functions

Utilities

__version__
Optunity version string
__revision__
Optunity revision string
__author__
Main authors of the package
optunity.manual(solver_name=None)

Prints the manual of requested solver.

Parameters:solver_name – (optional) name of the solver to request a manual from. If none is specified, a general manual is printed.

Raises KeyError if solver_name is not registered.

optunity.maximize(f, num_evals=50, solver_name=None, pmap=<built-in function map>, **kwargs)

Basic function maximization routine. Maximizes f within the given box constraints.

Parameters:
  • f – the function to be maximized
  • num_evals – number of permitted function evaluations
  • solver_name (string) – name of the solver to use (optional)
  • pmap (callable) – the map function to use
  • kwargs – box constraints, a dict of the following form {'parameter_name': [lower_bound, upper_bound], ...}
Returns:

retrieved maximum, extra information and solver info

This function will implicitly choose an appropriate solver and its initialization based on num_evals and the box constraints.

optunity.minimize(f, num_evals=50, solver_name=None, pmap=<built-in function map>, **kwargs)

Basic function minimization routine. Minimizes f within the given box constraints.

Parameters:
  • f – the function to be minimized
  • num_evals – number of permitted function evaluations
  • solver_name (string) – name of the solver to use (optional)
  • pmap (callable) – the map function to use
  • kwargs – box constraints, a dict of the following form {'parameter_name': [lower_bound, upper_bound], ...}
Returns:

retrieved minimum, extra information and solver info

This function will implicitly choose an appropriate solver and its initialization based on num_evals and the box constraints.

optunity.optimize(solver, func, maximize=True, max_evals=0, pmap=<built-in function map>)

Optimizes func with given solver.

Parameters:
  • solver – the solver to be used, for instance a result from optunity.make_solver()
  • func (callable) – the objective function
  • maximize (bool) – maximize or minimize?
  • max_evals (int) – maximum number of permitted function evaluations
  • pmap (function) – the map() function to use, to vectorize use optunity.pmap()

Returns the solution and a namedtuple with further details.

Result details includes the following:

optimum
optimal function value f(solution)
stats
statistics about the solving process
call_log
the call log
report
solver report, can be None

Statistics gathered while solving a problem:

num_evals
number of function evaluations
time
wall clock time needed to solve
optunity.wrap_call_log(f, call_dict)

Wraps an existing call log (as dictionary) around f.

This allows you to communicate known function values to solvers. (currently available solvers do not use this info)

optunity.wrap_constraints(f, default=None, ub_o=None, ub_c=None, lb_o=None, lb_c=None, range_oo=None, range_co=None, range_oc=None, range_cc=None, custom=None)

Decorates f with given input domain constraints.

Parameters:
  • f (callable) – the function that will be constrained
  • default (number) – function value to default to in case of constraint violations
  • ub_o (dict) – open upper bound constraints, e.g. \(x < c\)
  • ub_c (dict) – closed upper bound constraints, e.g. \(x \leq c\)
  • lb_o (dict) – open lower bound constraints, e.g. \(x > c\)
  • lb_c (dict) – closed lower bound constraints, e.g. \(x \geq c\)
  • range_oo (dict with 2-element lists as values ([lb, ub])) – range constraints (open lb and open ub) \(lb < x < ub\)
  • range_co (dict with 2-element lists as values ([lb, ub])) – range constraints (closed lb and open ub) \(lb \leq x < ub\)
  • range_oc (dict with 2-element lists as values ([lb, ub])) – range constraints (open lb and closed ub) \(lb < x \leq ub\)
  • range_cc (dict with 2-element lists as values ([lb, ub])) – range constraints (closed lb and closed ub) \(lb \leq x \leq ub\)
  • custom (list of constraints) – custom, user-defined constraints

*custom constraints are binary functions that yield False in case of violations.

>>> def f(x):
...     return x
>>> fc = wrap_constraints(f, default=-1, range_oc={'x': [0, 1]})
>>> fc(x=0.5)
0.5
>>> fc(x=1)
1
>>> fc(x=5)
-1
>>> fc(x=0)
-1

We can define any custom constraint that we want. For instance, assume we have a binary function with arguments x and y, and we want to make sure that the provided values remain within the unit circle.

>>> def f(x, y):
...     return x + y
>>> circle_constraint = lambda x, y: (x ** 2 + y ** 2) <= 1
>>> fc = wrap_constraints(f, default=1234, custom=[circle_constraint])
>>> fc(0.0, 0.0)
0.0
>>> fc(1.0, 0.0)
1.0
>>> fc(0.5, 0.5)
1.0
>>> fc(1, 0.5)
1234
optunity.make_solver(solver_name, *args, **kwargs)

Creates a Solver from given parameters.

Parameters:
  • solver_name (string) – the solver to instantiate
  • args – positional arguments to solver constructor.
  • kwargs – keyword arguments to solver constructor.

Use optunity.manual() to get a list of registered solvers. For constructor arguments per solver, please refer to Solver overview.

Raises KeyError if

  • solver_name is not registered
  • *args and **kwargs are invalid to instantiate the solver.
optunity.suggest_solver(num_evals=50, solver_name=None, **kwargs)
optunity.cross_validated(x, num_folds=10, y=None, strata=None, folds=None, num_iter=1, regenerate_folds=False, clusters=None, aggregator=<function mean at 0x7f38189b0e60>)

Function decorator to perform cross-validation as configured.

Parameters:
  • x – data to be used for cross-validation
  • num_folds – number of cross-validation folds (default 10)
  • y – (optional) labels to be used for cross-validation. If specified, len(labels) must equal len(x)
  • strata – (optional) strata to account for when generating folds. Strata signify instances that must be spread across folds. Not every instance must be in a stratum. Specify strata as a list of lists of instance indices.
  • folds – (optional) prespecified cross-validation folds to be used (list of lists (iterations) of lists (folds)).
  • num_iter – (optional) number of iterations to use (default 1)
  • regenerate_folds – (optional) whether or not to regenerate folds on every evaluation (default false)
  • clusters – (optional) clusters to account for when generating folds. Clusters signify instances that must be assigned to the same fold. Not every instance must be in a cluster. Specify clusters as a list of lists of instance indices.
  • aggregator – function to aggregate scores of different folds (default: mean)
Returns:

a cross_validated_callable with the proper configuration.

This resulting decorator must be used on a function with the following signature (+ potential other arguments):

Parameters:
  • x_train (iterable) – training data
  • y_train (iterable) – training labels (optional)
  • x_test (iterable) – testing data
  • y_test (iterable) – testing labels (optional)

y_train and y_test must be available of the y argument to this function is not None.

These 4 keyword arguments will be bound upon decoration. Further arguments will remain free (e.g. hyperparameter names).

>>> data = list(range(5))
>>> @cross_validated(x=data, num_folds=5, folds=[[[i] for i in range(5)]], aggregator=identity)
... def f(x_train, x_test, a):
...     return x_test[0] + a
>>> f(a=1)
[1, 2, 3, 4, 5]
>>> f(1)
[1, 2, 3, 4, 5]
>>> f(a=2)
[2, 3, 4, 5, 6]
optunity.generate_folds(num_rows, num_folds=10, strata=None, clusters=None)

Generates folds for a given number of rows.

Parameters:
  • num_rows – number of data instances
  • num_folds – number of folds to use (default 10)
  • strata – (optional) list of lists to indicate different sampling strata. Not all rows must be in a stratum. The number of rows per stratum must be larger than or equal to num_folds.
  • clusters – (optional) list of lists indicating clustered instances. Clustered instances must be placed in a single fold to avoid information leaks.
Returns:

a list of folds, each fold is a list of instance indices

 
>>> folds = generate_folds(num_rows=6, num_folds=2, clusters=[[1, 2]], strata=[[3,4]])
>>> len(folds)
2
>>> i1 = [idx for idx, fold in enumerate(folds) if 1 in fold]
>>> i2 = [idx for idx, fold in enumerate(folds) if 2 in fold]
>>> i1 == i2
True
>>> i3 = [idx for idx, fold in enumerate(folds) if 3 in fold]
>>> i4 = [idx for idx, fold in enumerate(folds) if 4 in fold]
>>> i3 == i4
False

Warning

Instances in strata are not necessarily spread out over all folds. Some folds may already be full due to clusters. This effect should be negligible.

optunity.pmap(f, *args)

Parallel map using multiprocessing.

Parameters:
  • f – the callable
  • args – arguments to f, as iterables
Returns:

a list containing the results

Warning

This function will not work in IPython: https://github.com/claesenm/optunity/issues/8.

Warning

Python’s multiprocessing library is incompatible with Jython.

optunity.available_solvers()

Returns a list of all available solvers.

These can be used in optunity.make_solver().

Table Of Contents

This Page

Previous topic

MATLAB

Next topic

Solvers

Navigation

© Copyright 2014, Marc Claesen, Jaak Simm and Dusan Popovic. Created using Sphinx 1.2.3.