# paramz package¶

## paramz.caching module¶

class Cache_this(limit=5, ignore_args=(), force_kwargs=())[source]

Bases: object

A decorator which can be applied to bound methods in order to cache them

class Cacher(operation, limit=3, ignore_args=(), force_kwargs=(), cacher_enabled=True)[source]

Bases: object

Cache an operation. If the operation is a bound method we will create a cache (FunctionCache) on that object in order to keep track of the caches on instances.

that Cacher will be overwritten by this Cacher!
Parameters: operation (callable) – function to cache limit (int) – depth of cacher ignore_args ([int]) – list of indices, pointing at arguments to ignore in *args of operation(*args). This includes self, so make sure to ignore self, if it is not cachable and you do not want this to prevent caching! force_kwargs ([str]) – list of kwarg names (strings). If a kwarg with that name is given, the cacher will force recompute and wont cache anything. verbose (int) – verbosity level. 0: no print outs, 1: casual print outs, 2: debug level print outs
add_to_cache(cache_id, inputs, output)[source]

This adds cache_id to the cache, with inputs and output

combine_inputs(args, kw, ignore_args)[source]

Combines the args and kw in a unique way, such that ordering of kwargs does not lead to recompute

disable_cacher()[source]

Disable the caching of this cacher. This also removes previously cached results

enable_cacher()[source]

Enable the caching of this cacher.

ensure_cache_length()[source]

Ensures the cache is within its limits and has one place free

id(obj)[source]

returns the self.id of an object, to be used in caching individual self.ids

on_cache_changed(direct, which=None)[source]

A callback funtion, which sets local flags when the elements of some cached inputs change

this function gets ‘hooked up’ to the inputs when we cache them, and upon their elements being changed we update here.

prepare_cache_id(combined_args_kw)[source]

get the cacheid (conc. string of argument self.ids in order)

reset()[source]

Totally reset the cache

class FunctionCache(*args, **kwargs)[source]

Bases: dict

disable_caching()[source]

Disable the cache of this object. This also removes previously cached results

enable_caching()[source]

Enable the cache of this object.

reset()[source]

Reset (delete) the cache of this object

## paramz.domains module¶

(Hyper-)Parameter domains defined for paramz.transformations. These domains specify the legitimate realm of the parameters to live in.

_REAL :
real domain, all values in the real numbers are allowed
_POSITIVE:
positive domain, only positive real values are allowed
_NEGATIVE:
same as _POSITIVE, but only negative values are allowed
_BOUNDED:
only values within the bounded range are allowed, the bounds are specified withing the object with the bounded range

## paramz.model module¶

class Model(name)[source]
objective_function()[source]

The objective function for the given algorithm.

This function is the true objective, which wants to be minimized. Note that all parameters are already set and in place, so you just need to return the objective function here.

For probabilistic models this is the negative log_likelihood (including the MAP prior), so we return it here. If your model is not probabilistic, just return your objective to minimize here!

objective_function_gradients()[source]

The gradients for the objective function for the given algorithm. The gradients are w.r.t. the negative objective function, as this framework works with negative log-likelihoods as a default.

You can find the gradient for the parameters in self.gradient at all times. This is the place, where gradients get stored for parameters.

This function is the true objective, which wants to be minimized. Note that all parameters are already set and in place, so you just need to return the gradient here.

For probabilistic models this is the gradient of the negative log_likelihood (including the MAP prior), so we return it here. If your model is not probabilistic, just return your negative gradient here!

optimize(optimizer=None, start=None, messages=False, max_iters=1000, ipython_notebook=True, clear_after_finish=False, **kwargs)[source]

Optimize the model using self.log_likelihood and self.log_likelihood_gradient, as well as self.priors.

kwargs are passed to the optimizer. They can be:

Parameters: max_iters (int) – maximum number of function evaluations optimizer (string) – which optimizer to use (defaults to self.preferred optimizer) True: Display messages during optimisation, “ipython_notebook”:
Valid optimizers are:
• ‘scg’: scaled conjugate gradient method, recommended for stability.
• ‘fmin_tnc’: truncated Newton method (see scipy.optimize.fmin_tnc)
• ‘simplex’: the Nelder-Mead simplex method (see scipy.optimize.fmin),
• ‘lbfgsb’: the l-bfgs-b method (see scipy.optimize.fmin_l_bfgs_b),
• ‘lbfgs’: the bfgs method (see scipy.optimize.fmin_bfgs),
• ‘sgd’: stochastic gradient decsent (see scipy.optimize.sgd). For experts only!
optimize_restarts(num_restarts=10, robust=False, verbose=True, parallel=False, num_processes=None, **kwargs)[source]

Perform random restarts of the model, and set the model to the best seen solution.

If the robust flag is set, exceptions raised during optimizations will be handled silently. If _all_ runs fail, the model is reset to the existing parameter values.

**kwargs are passed to the optimizer.

Parameters: num_restarts (int) – number of restarts to use (default 10) robust (bool) – whether to handle exceptions silently or not (default False) parallel (bool) – whether to run each restart as a separate process. It relies on the multiprocessing module. num_processes – number of workers in the multiprocessing pool max_f_eval (int) – maximum number of function evaluations max_iters (int) – maximum number of iterations messages (bool) – whether to display during optimisation

Note

If num_processes is None, the number of workes in the multiprocessing pool is automatically set to the number of processors on the current machine.

opt_wrapper(args)[source]

## paramz.param module¶

class Param(name, input_array, default_constraint=None, *a, **kw)[source]

Parameter object for GPy models.

Parameters: name (str) – name of the parameter to be printed input_array (np.ndarray) – array which this parameter handles default_constraint – The default constraint for this parameter

You can add/remove constraints by calling constrain on the parameter itself, e.g:

• self[:,1].constrain_positive()
• self[0].tie_to(other)
• self.untie()
• self[:3,:].unconstrain()
• self[1].fix()

Fixing parameters will fix them to the value they are right now. If you change the fixed value, it will be fixed to the new value!

Important Notes:

The array given into this, will be used as the Param object. That is, the memory of the numpy array given will be the memory of this object. If you want to make a new Param object you need to copy the input array!

Multilevel indexing (e.g. self[:2][1:]) is not supported and might lead to unexpected behaviour. Try to index in one go, using boolean indexing or the numpy builtin np.index function.

See GPy.core.parameterized.Parameterized for more details on constraining etc.

build_pydot(G)[source]

Build a pydot representation of this model. This needs pydot installed.

Example Usage:

np.random.seed(1000) X = np.random.normal(0,1,(20,2)) beta = np.random.uniform(0,1,(2,1)) Y = X.dot(beta) m = RidgeRegression(X, Y) G = m.build_pydot() G.write_png(‘example_hierarchy_layout.png’)

The output looks like:

Rectangles are parameterized objects (nodes or leafs of hierarchy).

Trapezoids are param objects, which represent the arrays for parameters.

Black arrows show parameter hierarchical dependence. The arrow points from parents towards children.

Orange arrows show the observer pattern. Self references (here) are the references to the call to parameters changed and references upwards are the references to tell the parents they need to update.

copy()[source]
get_property_string(propname)[source]
parameter_names(add_self=False, adjust_for_printing=False, recursive=True, **kw)[source]
flattened_parameters
gradient

Return a view on the gradient, which is in the same shape as this parameter is. Note: this is not the real gradient array, it is just a view on it.

is_fixed
num_params
param_array

As we are a leaf, this just returns self

parameters = []
values

Return self as numpy array view

class ParamConcatenation(params)[source]

Bases: object

Parameter concatenation for convenience of printing regular expression matched arrays you can index this concatenation as if it was the flattened concatenation of all the parameters it contains, same for setting parameters (Broadcasting enabled).

See GPy.core.parameter.Param for more details on constraining.

checkgrad(verbose=False, step=1e-06, tolerance=0.001)[source]
constrain(constraint, warning=True)[source]
Parameters: transform – the paramz.transformations.Transformation to constrain the this parameter to. warning – print a warning if re-constraining parameters.

Constrain the parameter to the given paramz.transformations.Transformation.

constrain_bounded(lower, upper, warning=True)[source]
Parameters: upper (lower,) – the limits to bound this parameter to warning – print a warning if re-constraining parameters.

Constrain this parameter to lie within the given range.

constrain_fixed(value=None, warning=True, trigger_parent=True)[source]

Constrain this parameter to be fixed to the current value it carries.

This does not override the previous constraints, so unfixing will restore the constraint set before fixing.

Parameters: warning – print a warning for overwriting constraints.
constrain_negative(warning=True)[source]
Parameters: warning – print a warning if re-constraining parameters.

Constrain this parameter to the default negative constraint.

constrain_positive(warning=True)[source]
Parameters: warning – print a warning if re-constraining parameters.

Constrain this parameter to the default positive constraint.

fix(value=None, warning=True, trigger_parent=True)

Constrain this parameter to be fixed to the current value it carries.

This does not override the previous constraints, so unfixing will restore the constraint set before fixing.

Parameters: warning – print a warning for overwriting constraints.
unconstrain(*constraints)[source]
Parameters: transforms – The transformations to unconstrain from.

remove all paramz.transformations.Transformation transformats of this parameter object.

unconstrain_bounded(lower, upper)[source]
Parameters: upper (lower,) – the limits to unbound this parameter from

Remove (lower, upper) bounded constrain from this parameter/

unconstrain_fixed()[source]

This parameter will no longer be fixed.

If there was a constraint on this parameter when fixing it, it will be constraint with that previous constraint.

unconstrain_negative()[source]

Remove negative constraint of this parameter.

unconstrain_positive()[source]

Remove positive constraint of this parameter.

unfix()

This parameter will no longer be fixed.

If there was a constraint on this parameter when fixing it, it will be constraint with that previous constraint.

update_all_params()[source]
values()[source]

## paramz.parameterized module¶

class Parameterized(name=None, parameters=[])[source]

Say m is a handle to a parameterized class.

Printing parameters:

- print m:           prints a nice summary over all parameters
- print m.name:      prints details for param with name 'name'
- print m[regexp]: prints details for all the parameters
which match (!) regexp
- print m['']:       prints details for all parameters


Fields:

Name:       The name of the param, can be renamed!
Value:      Shape or value, if one-valued
Constrain:  constraint of the param, curly "{c}" brackets indicate
some parameters are constrained by c. See detailed print
to get exact constraints.
Tied_to:    which paramter it is tied to.


Getting and setting parameters:

- Set all values in param to one:      m.name.to.param = 1
- Set all values in parameterized:     m.name[:] = 1
- Set values to random values:         m[:] = np.random.norm(m.size)


Handling of constraining, fixing and tieing parameters:

- You can constrain parameters by calling the constrain on the param itself, e.g:

- m.name[:,1].constrain_positive()
- m.name[0].tie_to(m.name[1])

- Fixing parameters will fix them to the value they are right now. If you change
the parameters value, the param will be fixed to the new value!

- If you want to operate on all parameters use m[''] to wildcard select all paramters
and concatenate them. Printing m[''] will result in printing of all parameters in detail.

build_pydot(G=None)[source]

Build a pydot representation of this model. This needs pydot installed.

Example Usage:

np.random.seed(1000)
X = np.random.normal(0,1,(20,2))
beta = np.random.uniform(0,1,(2,1))
Y = X.dot(beta)
m = RidgeRegression(X, Y)
G = m.build_pydot()
G.write_png('example_hierarchy_layout.png')


The output looks like:

Rectangles are parameterized objects (nodes or leafs of hierarchy).

Trapezoids are param objects, which represent the arrays for parameters.

Black arrows show parameter hierarchical dependence. The arrow points from parents towards children.

Orange arrows show the observer pattern. Self references (here) are the references to the call to parameters changed and references upwards are the references to tell the parents they need to update.

copy(memo=None)[source]
get_property_string(propname)[source]
grep_param_names(regexp)[source]

create a list of parameters, matching regular expression regexp

Parameters: parameters (list of or one paramz.param.Param) – the parameters to add [index] – index of where to put parameters

Add all parameters to this param class, you can insert parameters at any given index using the list.insert syntax

Parameters: param – param object to remove from being a parameter of this parameterized object.
flattened_parameters
class ParametersChangedMeta[source]

Bases: type

## paramz.ties_and_remappings module¶

class Fix(name=None, parameters=[])[source]
class Remapping(name=None, parameters=[])[source]
callback()[source]
mapping()[source]

The return value of this function gives the values which the re-mapped parameters should take. Implement in sub-classes.

parameters_changed()[source]
class Tie(name='tie')[source]

The new parameter tie framework. (under development)

All the parameters tied together get a new parameter inside the Tie object. Its value should always be equal to all the tied parameters, and its gradient is the sum of all the tied parameters.

Implementation Details:

The Tie object should only exist on the top of param tree (the highest parent).

self.label_buf: It uses a label buffer that has the same length as all the parameters (self._highest_parent_.param_array). The buffer keeps track of all the tied parameters. All the tied parameters have a label (an interger) higher than 0, and the parameters that have the same label are tied together.

self.buf_index: An auxiliary index list for the global index of the tie parameter inside the Tie object.

Warning

This is not implemented yet and will be different, with high degree of certainty. Do not use!

add_tied_parameter(p, p2=None)[source]

Tie the list of parameters p together (p2==None) or Tie the list of parameters p with the list of parameters p2 (p2!=None)

collate_gradient()[source]
getTieFlag(p=None)[source]
parameters_changed()[source]
propagate_val()[source]

## paramz.transformations module¶

class Exponent[source]
f(x)[source]
finv(x)[source]
gradfactor(f, df)[source]
initialize(f)[source]
log_jacobian(model_param)[source]
log_jacobian_grad(model_param)[source]
domain = 'positive'
class Logexp[source]
f(x)[source]
finv(f)[source]
gradfactor(f, df)[source]
initialize(f)[source]
log_jacobian(model_param)[source]
log_jacobian_grad(model_param)[source]
domain = 'positive'
class LogexpClipped(lower=1e-06)[source]
f(x)[source]
finv(f)[source]
gradfactor(f, df)[source]
initialize(f)[source]
domain = 'positive'
log_max_bound = 230.25850929940458
log_min_bound = -23.025850929940457
max_bound = 1e+100
min_bound = 1e-10
class LogexpNeg[source]
f(x)[source]
finv(f)[source]
gradfactor(f, df)[source]
initialize(f)[source]
domain = 'positive'
class Logistic(lower, upper)[source]
f(x)[source]
finv(f)[source]
gradfactor(f, df)[source]
initialize(f)[source]
domain = 'bounded'
class NegativeExponent[source]
f(x)[source]
finv(f)[source]
gradfactor(f, df)[source]
initialize(f)[source]
domain = 'negative'
class NegativeLogexp[source]
f(x)[source]
finv(f)[source]
gradfactor(f, df)[source]
initialize(f)[source]
domain = 'negative'
logexp = Logexp
class NormalEta(mu_indices, var_indices)[source]

Do not use, not officially supported!

Do not use, not officially supported!

f(theta)[source]
finv(muvar)[source]
gradfactor(muvar, dmuvar)[source]
initialize(f)[source]
class NormalNaturalAntti(mu_indices, var_indices)[source]

Do not use, not officially supported!

Do not use, not officially supported!

gradfactor(muvar, dmuvar)[source]
initialize(f)[source]
class NormalNaturalThroughEta(mu_indices, var_indices)[source]

Do not use, not officially supported!

Do not use, not officially supported!

gradfactor(muvar, dmuvar)[source]
class NormalNaturalThroughTheta(mu_indices, var_indices)[source]

Do not use, not officially supported!

Do not use, not officially supported!

gradfactor(muvar, dmuvar)[source]
gradfactor_non_natural(muvar, dmuvar)[source]
class NormalNaturalWhooot(mu_indices, var_indices)[source]

Do not use, not officially supported!

Do not use, not officially supported!

gradfactor(muvar, dmuvar)[source]
class NormalTheta(mu_indices, var_indices)[source]

Do not use, not officially supported!

Do not use, not officially supported!

f(theta)[source]
finv(muvar)[source]
gradfactor(muvar, dmuvar)[source]
initialize(f)[source]
class Square[source]
f(x)[source]
finv(x)[source]
gradfactor(f, df)[source]
initialize(f)[source]
domain = 'positive'
class Transformation[source]

Bases: object

f(opt_param)[source]
finv(model_param)[source]
gradfactor(model_param, dL_dmodel_param)[source]

df(opt_param)_dopt_param evaluated at self.f(opt_param)=model_param, times the gradient dL_dmodel_param,

i.e.: define



rac{ rac{partial L}{partial f}left(left.partial f(x)}{partial x} ight|_{x=f^{-1}(f) ight)}

gradfactor_non_natural(model_param, dL_dmodel_param)[source]
initialize(f)[source]

produce a sensible initial value for f(x)

log_jacobian(model_param)[source]

compute the log of the jacobian of f, evaluated at f(x)= model_param

log_jacobian_grad(model_param)[source]

compute the drivative of the log of the jacobian of f, evaluated at f(x)= model_param

plot(xlabel='transformed $\\theta$', ylabel='$\\theta$', axes=None, *args, **kw)[source]
domain = None

## Module contents¶

load(file_or_path)[source]

Load a previously pickled model, using m.pickle(‘path/to/file.pickle)’

Parameters: file_name – path/to/file.pickle