This python package implements some recipes from the python cookbook. Some of the recipes are copied verbatim from the python cookbook and some are modified versions of those at the cookbook. Others I (John Reid) have written from scratch.
All novel parts of this software are released under the BSD. See the accompanying file python/cookbook/LICENSE. Please also refer to original code for their licensing terms.
I have tested this software on python 2.6 and 2.7.
A bidirectional map that maintains 2 dicts, one to map each way between elements of type 1 and type 2.
The map is built by adding pairs of elements that are related.
Add a relationship between v1 of type 1 and v2 of type 2.
Removed a relationship between v1 of type 1 and v2 of type 2.
Will do nothing if relationship is not in map.
Maps from keys of type 1 to values of type 2.
Maps from keys of type 2 to values of type 1.
Removed a relationship between v1 of type 1 and v2 of type 2.
Will throw key error if relationship is not in map.
From: http://code.activestate.com/recipes/52308-the-simple-but-handy-collector-of-a-bunch-of-named/ (released under PSF license).
Often we want to just collect a bunch of stuff together, naming each item of the bunch; a dictionary’s OK for that, but a small do-nothing class is even handier, and prettier to use.
Adapted from: http://code.activestate.com/recipes/325205-cache-decorator-in-python-24/ (released under PSF license).
The latest version of Python introduced a new language feature, function and method decorators (PEP 318, http://www.python.org/peps/pep-0318.html). This recipe show a common callable transformation that can benefit from the new syntax, often referred to as Memoization pattern.
Memoization is a useful pattern for improving the amortized performance of a computationally expensive function or method. The new syntax allows making explicit that a callable is memoized.
A restriction of the recipe above is that the arguments of the function (and also the method receiver object for methods) have to be hashable.
The decorator can be generalized by allowing different caching policies (e.g. a FIFO cache or a cache implementing an LRU policy) apart from the implied “cache-forever” policy of a dict.
A hashable dict.
Like Memoize but pickles its cache/memo for use in next python session
Dump the memo to the cache.
@return: The file we pickle to and from.
Load the memo from the cache.
Pickles the result of the method (ignoring arguments) and uses this if possible on the next call.
Get console terminal size if possible.
From: http://stackoverflow.com/questions/566746/how-to-get-console-window-width-in-python
Get console terminal size if possible.
In Python, any variable can be re-bound at will – and modules don’t let you define special methods such as an instance’s __setattr__ to stop attribute re-binding. Easy solution (in Python 2.1 and up): use an instance as ‘module’...
Adapted from http://code.activestate.com/recipes/65207-constants-in-python/ (released under PSF license).
Decorator helpers.
Decorator that allows other decorators to decorate instance methods.
This decorator can be used to turn simple functions into well-behaved decorators, so long as the decorators are fairly simple. If a decorator expects a function and returns a function (no descriptors), and if it doesn’t modify function attributes or docstring, then it is eligible to use this. Simply apply @simple_decorator to your decorator and it will automatically preserve the docstring and function attributes of functions to which it is applied.
Dictionary that assigns one value in a sequence of values to each key. Can be useful for assigning colours/line styles in plots for example.
Iterator over values.
A dictionary where missing values are created by an __init__ time supplied function.
A dictionary where the values are dictionarys. If an accessed key is missing, the value is initialised to an empty dict
A dictionary where the values are ints. If an accessed key is missing, the value is initialised to 0. Useful for counting
A dictionary where the values are lists. If an accessed key is missing, the value is initialised to an empty list
A dictionary where the values are sets. If an accessed key is missing, the value is initialised to an empty set
Holds a unique integer id for each key that is accessed.
@return: A function that creates DictOf using these args.
Taken from http://code.activestate.com/recipes/413486-first-class-enums-in-python/ (released under PSF license)
From http://code.activestate.com/recipes/499354-equivalence-partition/
Checks that a partition is consistent under the relationship
Partitions a set of objects into equivalence classes
Same as equivalence_partition() but also numbers the classes.
| Parameters: |
|
|---|---|
| Returns: | (classes, partitions, ids) classes is a sequence of sets. Each one is an equivalence class partitions is a dictionary mapping objects to equivalence classes ids is a dictionary mapping objects to the indices of their equivalence classes |
Partitions a set of objects into equivalence classes
| Parameters: |
|
|---|---|
| Returns: | (classes, partitions) classes is a sequence of sets. Each one is an equivalence class. partitions is a dictionary mapping objects to equivalence classes. |
Code to run a function as a task. Useful to make sequential code parallel.
Uses pickling to pass arguments and get result back.
This exception is raised when a process run by check_call() or check_output() returns a non-zero exit status. The exit status will be stored in the returncode attribute; check_output() will also store the output in the output attribute.
Run command with arguments and return its output as a byte string.
If the exit code was non-zero it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute and output in the output attribute.
The arguments are the same as for the Popen constructor. Example:
>>> check_output(["ls", "-l", "/dev/null"])
'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
The stdout argument is not allowed as it is used internally. To capture standard error in the result, use stderr=STDOUT.
>>> check_output(["/bin/sh", "-c",
... "ls -l non_existent_file ; exit 0"],
... stderr=STDOUT)
'ls: non_existent_file: No such file or directory\n'
Create a queue and a number of worker threads on it.
Create a worker function that takes tasks off a queue.
Actually do the task.
Log an exception. Can be used as replacement for sys.excepthook.
Run the named function as a subprocess.
Function to test run function as task functionality.
http://code.activestate.com/recipes/259173-groupby/ Licensed under the PSF License
Code to handle intervals.
Represents an interval. Defined as half-open interval [start,end), which includes the start position but not the end. Start and end do not have to be numeric types.
@return: A slice object that the interval indexes.
@return: True iff self is empty.
The interval’s end
@return: Interval containing both self and other.
Intersection. @return: An empty intersection if there is none.
@return: True iff self intersects other.
@return: True iff self is proper subset of other.
@return: The distance between self and other.
@return: True iff self.end - self.start == 1.
The interval’s start
@return: True iff self is subset of other.
@return: True iff 0 in self.
One-line decorator call adds caching to functions with hashable arguments and no keyword arguments. When the maximum size is reached, the least recently used entry is discarded – appropriate for long-running processes which cannot allow caches to grow without bound. Includes built-in performance instrumentation.
Adapted from http://code.activestate.com/recipes/498245-lru-and-lfu-cache-decorators/ (released under PSF license).
Decorator applying a least-recently-used cache with the given maximum size.
Arguments to the cached function must be hashable. Cache performance statistics stored in f.hits and f.misses.
From http://code.activestate.com/recipes/500261-named-tuples/ Released under PSF license.
Code to implement output contexts.
An output context. Uses the Borg design pattern: http://code.activestate.com/recipes/66531-singleton-we-dont-need-no-stinkin-singleton-the-bo/
@return: A callable that returns a filename suitable for caching an object.
@return: A decorator that caches the results of methods.
Set up the output context, i.e. tell it where any output should go.
Make sure the given directory exists.
An object we can use to access the output context.
Code to handle permutations and their inverses.
@return: A permutation that consists of the 2 permutations chained together.
@return: The identity permutation.
@return: The inversion of the permutation.
@return: A list that is the permuted version of x.
A generator that yields a permuted version of x.
Code for lazy cached initialisation.
Caches the result of calling a function. The function is evaluated once on the first invocation of self() and the cached result is returned for every invocation thereafter.
Caches the result of calling a function. The function is evaluated once on the first invocation of self() and the cached result is returned for every invocation thereafter. The persist method stores the cached result in the given file. This pickled object is reused in preference to function invocation the next time the object is loaded.
Pickle object to disk
Persist all the PersistedCaches in the given set of variables. E.g. persist_all_in(vars().values())
From http://wiki.python.org/moin/PythonDecoratorLibrary
Provide pre-/postconditions as function decorators.
Example usage:
>>> def in_ge20(inval):
... assert inval >= 20, 'Input value < 20'
...
>>> def out_lt30(retval, inval):
... assert retval < 30, 'Return value >= 30'
...
>>> @precondition(in_ge20)
... @postcondition(out_lt30)
... def inc(value):
... return value + 1
...
>>> inc(5)
Traceback (most recent call last):
...
AssertionError: Input value < 20
>>> inc(29)
Traceback (most recent call last):
...
AssertionError: Return value >= 30
>>> inc(20)
21
You can define as many pre-/postconditions for a function as you like. It is also possible to specify both types of conditions at once:
>>> @conditions(in_ge20, out_lt30)
... def add1(value):
... return value + 1
...
>>> add1(5)
Traceback (most recent call last):
...
AssertionError: Input value < 20
An interesting feature is the ability to prevent the creation of pre-/postconditions at function definition time. This makes it possible to use conditions for debugging and then switch them off for distribution.
>>> debug = False
>>> @precondition(in_ge20, debug)
... def dec(value):
... return value - 1
...
>>> dec(5)
4
From: http://code.activestate.com/recipes/473899-progress-meter/ Licensed under the PSF License
Here is a silly example of its usage:
import progress import time import random
total = 1000 p = progress.ProgressMeter(total=total)
Here is an example of its output:
[————————-> ] 41% 821.2/sec
2006-02-20 Denis Barmenkov: ANSI codes replaced by Backspace (0x08) characters
Some utilities to use with matplotlib.
Cycle through various format strings so that many different data can be displayed together.
If specifying many formats it is best to make them relatively prime to ensure the largest total cycle length.
Get fig_width_pt from LaTeX using showthecolumnwidth
@return: figure size tuple if fig_width_pt was specified
Specify either num_rows or num_cols.
A way of turning off pylab’s interactive mode with a ‘with’ statement. For example you could do:
with pylab_context_ioff(): ... pylab.figure()
to ensure the figure was created in non-interactive mode. After the context has ended, pylab’s interactive status is returned to what it was before the context was entered.
Decorator that turns pylab interactive mode off for duration of call to decorated function.
Set the rcParams in matplotlib to use postscript output.
If fig_width_pt is given then set default figure size to match.
Basic pylab colours.
create violin plots on an axis
Code to ease using pytables.
@return: A decorator that adds arguments and the result of the function to the table.
@return: A decorator that caches the arguments and the result of the function in the table.
An augmented red-black tree that handles intervals.
A node of a red black tree of half-open intervals.
The maximum value of any endpoint in the subtree rooted at this node.
@return: An interval that the subtree rooted here spans.
A red black tree of half-open intervals. See Cormen, Leiserson, Rivest, Stein 2nd edition pg 311.
Check the max invariant of the nodes and the red-black properties.
@return: The closest intervals in the interval tree to the given interval.
Search for a node whose interval overlaps i.
A red-black tree implementation.
A node in a red black tree. See Cormen, Leiserson, Rivest, Stein 2nd edition pg 273.
The node’s key
The node’s left child
The node’s parent
Is the node red?
The node’s right child
A red black tree. See Cormen, Leiserson, Rivest, Stein 2nd edition pg 273.
@return: True iff satisfies all criteria to be red-black tree.
Insert the key into the tree.
Insert node z into the tree.
@return: The maximum value in the subtree rooted at x.
@return: The minimum value in the subtree rooted at x.
The tree’s nil node
The tree’s root node
Search the subtree rooted at x (or the root if not given) iteratively for the key.
@return: self.nil if it cannot find it.
Insert keys one by one checking invariants and membership as we go.
Write the tree in the dot language format to f.
From http://code.activestate.com/recipes/415903-two-dict-classes-which-can-lookup-keys-by-value-an/ Released under PSF license
A dictionary which can lookup values by key, and keys by value. The lookup method returns a list of keys with matching values to the value argument.
A dictionary which can lookup values by key, and keys by value. All values and keys must be hashable, and unique.
Code to simplify logging and options in scripts.
A context manager that temporarily sets the dlopen flags for loading multiple boost.python modules and then returns them to previous values.
Log an exception. Used as replacement for sys.excepthook.
Log the values of the options.
Prepend a value to colon separated list in the named environment variable.
A context manager that temporarily sets the dlopen flags and then returns them to previous values.
Set up logging.
Print some details about the environment python is running in.
Utility functions for simple logging operations.
Adds a file handler to the logger if it does not already exist. If no logger is given, defaults to root logger. Returns the handler if one is created.
Add logging to named file.
Set up simple logging to stderr.
@return: Decorator to log exceptions raised inside a function.
Log the current exception.
Log the traceback for the current exception.
Set up simple logging to named file.
A context manager that can be used to temporarily add a handler to a logger.
Typical usage:
from cookbook.simple_logging import use_handler from logging import getLogger, FileHandler, Formatter
- with use_handler(getLogger(), FileHandler(‘output.log’)) as handler:
- handler.setFormatter(Formatter(“%(asctime)s:%(levelname)s: %(message)s”)) <do some stuff>
Code to time things.
Times things.
@return: The duration since start.
Restarts the timer.
Times things and implements the context management protocol (see PEP 343). A typical use case would be:
from cookbook.timer import Timer
- with Timer(msg=’description of task’, level=logging.INFO, logger=getLogger(__name__)) as timer:
- <do some stuff>
Creates Venn diagrams from up to 4 sets.
Calculate the areas given by these bounds.
Calculate the error associated with these areas.
Calculate the areas given by the grid bounds.
@return: A separation value. The closer to 0 it is, the better the bounds will be separated.
@return: A square-ness value. The closer to 0 it is, the more square the bounds will be.
Create a SVG figure from the bounds.
Use an optimizer to find the bounds.
Fit a Venn diagram to the 4 sets.
Return a suitable set of initial bounds.
Calculate mid_points
Code to set up logging and options in a workflow
@return: A decorator that caches the results of the function.
Makes a directory if it does not already exist.
Initialise the workflow:
Log the attributes in a module.
@return: Does the logger have a file handler already?
@return: Decorator that stores output of methods in cache directory.
A path to the given filename in the output directory.