Bulk (Re)Decoration, (Re)Decorating Imports¶
This chapter discusses the log_calls.decorate_*
classmethods. These methods allow you to:
- decorate or redecorate functions and classes,
- decorate an entire class hierarchy (a class and all its subclasses), and even
- decorate all classes and/or functions in a module.
These methods are handy in situations where altering source code is impractical (too many things to decorate) or questionable practice (third-party modules and packages). They can also help you learn a new codebase, by shedding light on its internal operations.
The decorate_*
methods provide another way to dynamically change the settings
of already-decorated functions and classes.
Like any decorator, log_calls is a functional — a function that takes a function argument and returns a function. The following typical use:
@log_calls()
def f(): pass
is equivalent to:
f = log_calls()(f)
If f
occurs in your own code, then no doubt you’ll prefer the former. The log_calls.decorate_*
methods let you decorate f
when its definition does not necessarily appear in your code.
Note
You can’t decorate Python builtins. Attempting to do is harmless (anyway, it’s supposed to be!), and log_calls will return the builtin class or callable unchanged. For example, the following have no effect:
log_calls.decorate_class(dict)
log_calls.decorate_class(dict, only='update')
log_calls.decorate_function(dict.update)
Decorating classes programmatically¶
Decorating a class and optionally, all of its subclasses¶
-
classmethod
log_calls.
decorate_class
(klass: type, decorate_subclasses=False, **setting_kwargs) → None¶ Decorate class
klass
and, optionally, all of its descendants recursively. Ifdecorate_subclasses == True
, and if any subclasses are decorated, their explicitly given settings remain unchanged by those insetting_kwargs
unlessoverride=True
is insetting_kwargs
.log_calls.decorate_class(C, **kwds)
is basically a syntactically sweetened version oflog_calls(**kwds)(C)
, with the addition of the flag parameterdecorate_subclasses
. There’s another difference, however:log_calls.decorate_class(...)
returnsNone
, whereaslog_calls(**kwds)(C)
returnsC
.
Decorating a class and all of its subclasses¶
-
classmethod
log_calls.
decorate_hierarchy
(baseclass: type, **setting_kwargs) → None¶ Decorate
baseclass
and, recursively, all of its descendants. If any subclasses are directly decorated, their explicitly given settings remain unchanged by those insetting_kwargs
unlessoverride=True
is insetting_kwargs
.This is just a shorthand for
log_calls.decorate_class(baseclass, decorate_subclasses=True, **setting_kwargs)
.
Decorating functions programmatically¶
Decorating a function in your namespace¶
-
classmethod
log_calls.
decorate_function
(f: 'Callable', **setting_kwargs) → None¶ Decorate
f
usingsettings_kwds
, and replace the definition off.__name__
with the decorated function (i.e. the wrapper) in the global namespace of the caller.Parameters: - f – a function object, with no package/module qualifier:
however it would be referred to in code at the point of the call
to
decorate_function
.f
itself refers to a function which is either defined in or imported into the module of the caller. - setting_kwargs – settings for decorator
log_calls.decorate_function(f, **kwds)
is basically a syntactically sweetened version oflog_calls(**kwds)(f)
. However,log_calls.decorate_function(...)
returnsNone
, whereaslog_calls(**kwds)(f)
returns the wrapper off
.- f – a function object, with no package/module qualifier:
however it would be referred to in code at the point of the call
to
Decorating an “external” function in a package¶
-
classmethod
log_calls.
decorate_package_function
(f: 'Callable', **setting_kwargs) → None¶ Decorate
f
using settings insettings_kwds
; replace the definition off.__name__
with the decorated function in the__dict__
of the module off
.Parameters: - f – a function object, qualified with a package, e.g.
somepackage.somefunc
, however it would be referred to in code at the point of a call todecorate_package_function
. - setting_kwargs – settings for decorator
- f – a function object, qualified with a package, e.g.
Decorating an “external” function in a module¶
-
classmethod
log_calls.
decorate_module_function
(f: 'Callable', **setting_kwargs) → None¶ Decorate
f
using settings insettings_kwds
; replace the definition off.__name__
with the decorated function in the__dict__
of the module off
.Parameters: - f – a function object, qualified with a module, e.g.
thatmodule.afunc
, however it would be referred to in code at the point of a call todecorate_module_function
. - setting_kwargs – settings for decorator
- f – a function object, qualified with a module, e.g.
Decorating all functions and/or classes in a module¶
decorate_module
lets you decorate the functions and/or classes of an imported module:
-
classmethod
log_calls.
decorate_module
(cls, mod: 'module', functions: bool=True, classes: bool=True, **setting_kwargs) → None¶ Parameters: - mod – module whose members are to be decorated
- functions – decorate all functions in
mod
if true - classes – decorate all classes in
mod
if true - setting_kwargs – keyword parameters for decorator
Raises: TypeError
Examples¶
These modules in the tests/
subdirectory contain several examples:
test_decorate_module.py
The docstring of the functiontest_decorate_module()
contains simple tests of decorating the moduletests/some_module.py
.
A few examples/tests use the Skikit-Learn package if it’s installed.
(The following subsection reproduces one of them.)
Those in these two modules are run by run_tests.py
:
test_decorate_sklearn_KMeans.py
test_decorate_sklearn_KMeans_functions.py
The test in the following module decorates an entire module of Skikit-Learn:
_test_decorate_module_of_sklearn.py
As the settings it imposes mess up the other sklearn
tests, it is not
run by run_tests.py
. It can be run separately.
Example — decorating a class in scikit-learn¶
This example demonstrates:
- decorating a class that’s not part of your project (unless you’re working on scikit-learn:), and
- using the
override
parameter with one of thelog_calls.decorate_*
functions to dynamically change the settings of (all the callables of) an already-decorated class.
Except for the log_calls.decorate_*
calls, the following code is excerpted
from the sklearn site, e.g. Demonstration of k-means assumptions.
The double backslashes in the two added lines accommodate doctest.
>>> from log_calls import log_calls
>>> from sklearn.cluster import KMeans
>>> from sklearn.datasets import make_blobs
>>> n_samples = 1500
>>> random_state = 170
>>> X, y = make_blobs(n_samples=n_samples, random_state=random_state)
First, let’s decorate the class hierarchy, with settings that show just the call tree:
>>> log_calls.decorate_hierarchy(KMeans, log_args=False) ### THIS LINE ADDED
Now let’s call KMeans.fit_predict
:
>>> y_pred = KMeans(n_clusters=2, random_state=random_state).fit_predict(X)
KMeans.__init__ <== called by <module>
KMeans.__init__ ==> returning to <module>
KMeans.fit_predict <== called by <module>
KMeans.fit <== called by KMeans.fit_predict
KMeans._check_fit_data <== called by KMeans.fit
KMeans._check_fit_data ==> returning to KMeans.fit
KMeans.fit ==> returning to KMeans.fit_predict
KMeans.fit_predict ==> returning to <module>
MiniBatchKMeans
is a subclass of KMeans
so that class is decorated too:
>>> mbk = MiniBatchKMeans(init='k-means++', n_clusters=2, batch_size=45,
... n_init=10, max_no_improvement=10)
MiniBatchKMeans.__init__ <== called by <module>
KMeans.__init__ <== called by MiniBatchKMeans.__init__
KMeans.__init__ ==> returning to MiniBatchKMeans.__init__
MiniBatchKMeans.__init__ ==> returning to <module>
Now let’s call MiniBatchKMeans.fit
:
>>> mbk.fit(X)
MiniBatchKMeans.fit <== called by <module>
MiniBatchKMeans._labels_inertia_minibatch <== called by MiniBatchKMeans.fit
MiniBatchKMeans._labels_inertia_minibatch ==> returning to MiniBatchKMeans.fit
MiniBatchKMeans.fit ==> returning to <module>
MiniBatchKMeans(batch_size=45, compute_labels=True, init='k-means++',
init_size=None, max_iter=100, max_no_improvement=10, n_clusters=2,
n_init=10, random_state=None, reassignment_ratio=0.01, tol=0.0,
verbose=0)
To view arguments as well (and trigger more output), change setting to log_args=True
and use override=True
. Here, we call log_calls.decorate_class
for class KMeans
with the parameter decorate_subclasses=True
, which is equivalent to calling
log_calls.decorate_hierarchy
:
>>> log_calls.decorate_class(KMeans, decorate_subclasses=True,
... log_args=True, args_sep='\\n',
... override=True)
>>> mbk.fit(X)
MiniBatchKMeans.fit <== called by <module>
arguments:
self=MiniBatchKMeans(batch_size=45, compute_labels=True, init='k-means++',
init_size=None, max_iter=100, max_no_improvement=10, n_clusters=2,
n_init=10, random_state=None, reassignment_ratio=0.01, tol=0.0,
verbose=0)
X=array([[ -5.19811282e+00, 6.41869316e-01],
[ -5.75229538e+00, 4.18627111e-01],
[ -1.08448984e+01, -7.55352273e+00],
...,
[ 1.36105255e+00, -9.07491863e-01],
[ -3.54141108e-01, 7.12241630e-01],
[ 1.88577252e+00, 1.41185693e-03]])
defaults:
y=None
MiniBatchKMeans._labels_inertia_minibatch <== called by MiniBatchKMeans.fit
arguments:
self=MiniBatchKMeans(batch_size=45, compute_labels=True, init='k-means++',
init_size=None, max_iter=100, max_no_improvement=10, n_clusters=2,
n_init=10, random_state=None, reassignment_ratio=0.01, tol=0.0,
verbose=0)
X=array([[ -5.19811282e+00, 6.41869316e-01],
[ -5.75229538e+00, 4.18627111e-01],
[ -1.08448984e+01, -7.55352273e+00],
...,
[ 1.36105255e+00, -9.07491863e-01],
[ -3.54141108e-01, 7.12241630e-01],
[ 1.88577252e+00, 1.41185693e-03]])
MiniBatchKMeans._labels_inertia_minibatch ==> returning to MiniBatchKMeans.fit
MiniBatchKMeans.fit ==> returning to <module>
MiniBatchKMeans(batch_size=45, compute_labels=True, init='k-means++',
init_size=None, max_iter=100, max_no_improvement=10, n_clusters=2,
n_init=10, random_state=None, reassignment_ratio=0.01, tol=0.0,
verbose=0)
Note: the ellipses in the values of the numpy
array X
are produced by its repr
.