Pyheapdump

Create and debug heap dumps.

Warning

This is alpha quality code.

Note

The pyheapdump package currently requires Python 2.7.

pyheapdump.dump_on_unhandled_exceptions(function=None, dump_dir=None, message=None, files=None, reraise=None, daisy_chain_sys_excepthook=None)

Create a heap dump for each unhandled exception.

This function acts as a function decorator or, if no function is given, registers a sys.excepthook handler. In both cases it causes python to create heap dumps on otherwise unhandled exceptions.

If function is given, dump_on_unhandled_exceptions wraps function and returns the newly created wrapper. The wrapper catches exceptions of type Exception and writes a heap dump. If reraise is true, it re-raises the exception. Otherwise it returns None.

If function is None, dump_on_unhandled_exceptions registers a sys.excepthook handler. If the handler receives an Exception, it writes a heap dump. If daisy_chain_sys_excepthook is true, the handler finally calls the previous exception handler.

Parameters:
  • function – a callable. If given dump_on_unhandled_exceptions returns a wrapper for function.
  • dump_dir (a string.) – the directory where to create the dump. If set to None, the dump is created in the directory given by the environment variable PYHEAPDUMP_DIR or, if the variable is unset or empty, the dump is created in the directory returned by tempfile.gettempdir().
  • message (string) – a message to print on sys.stderr.
  • files –

    this argument controls, if the heap dump contains the source code of those Python files, that contribute to the dumped frames. If files is None and the environment variable PYHEAPDUMP_WITH_FILES is defined, files is set from PYHEAPDUMP_WITH_FILES as follows: if PYHEAPDUMP_WITH_FILES contains os.path.pathsep, files is set to the list of path entries. Otherwise, files is set to True if PYHEAPDUMP_WITH_FILES is “yes” or “true”. All other non empty values set files to False.

    If files is a container, a file is included, if its name is in the container. Otherwise, if files is a callable, it will be called with a file name as its single argument. If the callable returns a true-value, the file will be included. Otherwise, if files is a true-value, all files are included. Lastly, if files is None, a default behaviour applies which is currently to include all files.

  • reraise (bool) – if true, a wrapper re-raises the caught exception.
  • daisy_chain_sys_excepthook (bool) – if true, daisy chain the original sys.excepthook handler.
pyheapdump.create_dump(exc_info=None, threads=None, tasklets=None, headers=None, files=None, lock_function=<function lock_function>)

Create a Python heap-dump.

This function creates a Python dump. The dump contains various informations about the currently executing Python program:

  • exception information
  • threads
  • tasklets

The exact content depends on the function arguments.

This function tries to prevent thread context switches during the creation of the dump. On Stackless it uses the tasklet flag atomic. Otherwise it temporarily manipulates the check-interval. Additionally this function temporarily increases the recursion limit.

This function will usually be called from an except block to allow post-mortem debugging of a failed process. The function returns a byte string, that can be loaded with load_dump() or debug_dump().

Arguments

Parameters:
  • filename (str) – name of the dump-file
  • exc_info – optional exception info tuple as returned by sys.exc_info(). If set to None, the value returned by sys.exc_info() will be used. If set to False, save_dump() does not add exception information to the dump.
  • threads – A single thread id (type is int) or a sequence of thread ids (type is collections.Sequence) or None for all threads (default) or False to omit thread information altogether.
  • tasklets – Stackless Python only: either None for all tasklets (default) or False to omit tasklet information altogether.
  • headers (collections.MutableMapping) – a mutable mapping, that contains header lines. All keys must be unicode strings. Values must be text, bytes or None.
  • files – this argument controls, if the heap dump contains the source code of those Python files, that contribute to the dumped frames. If files is a container, a file is included, if its name is in the container. Otherwise, if files is a callable, it will be called with a file name as its single argument. If the callable returns a true-value, the file will be included. Otherwise, if files is a true-value, all files are included. Lastly, if files is None, a default behaviour applies which is currently to include all files.
  • lock_function – Used internally. Do not set this argument!
Returns:

the compressed pickle of the heap-dump, the updated headers collection
Return type:

tuple
pyheapdump.save_dump(filename, tb=None, exc_info=None, threads=None, tasklets=None, headers=None, files=None, formatter=None, lock_function=<function lock_function>)

Create and save a Python heap-dump.

This function will usually be called from an except block to allow post-mortem debugging of a failed process. The function calls the function create_dump() to create the dump.

The saved file can be loaded with load_dump() or debug_dump(), which recreates traceback / thread and tasklet objects and passes them to a Python debugger.

The simplest way to do that is to run:

$ python -m pyheapdump my_dump_file.dump

Arguments

Parameters:
  • filename (str) – name of the dump-file
  • tb (types.TracebackType) – an optional traceback. This parameter exists for compatibility with the Python module pydump.
  • exc_info – optional exception info tuple as returned by sys.exc_info(). If set to None, the value returned by sys.exc_info() will be used. If set to True, save_dump() does not add exception information to the dump.
  • threads – A single thread id (type is int) or a sequence of thread ids (type is collections.Sequence) or None for all threads (default) or False to omit thread information altogether.
  • tasklets – Stackless Python only: either None for all tasklets (default) or False to omit tasklet information altogether.
  • headers (collections.MutableMapping) – a mutable mapping, that contains header lines. All keys must be unicode strings. Values must be text, bytes or None.
  • files – this argument controls, if the heap dump contains the source code of those Python files, that contribute to the dumped frames. If files is a container, a file is included, if its name is in the container. Otherwise, if files is a callable, it will be called with a file name as its single argument. If the callable returns a true-value, the file will be included. Otherwise, if files is a true-value, all files are included. Lastly, if files is None, a default behaviour applies which is currently to include all files.
  • formatter – the formatter to be used. The formatter must be a callable with two arguments pickle and headers that returns the bytes of the heap dump file. See MimeMessageFormatter for an example.
  • lock_function – Used internally. Do not set this argument!
Returns:

A tuple of length 2: the name of the dump file and the headers dictionary.
pyheapdump.load_dump(dumpfile=None, dump=None)

Load a Python heap dump

This function loads and preprocesses a Python dump file or a Python dump string or an already unpickled heap dump dictionary.

Arguments

Parameters:
  • dumpfile (string or file-like) – the name of the heap-dump file or a file-like object open for reading bytes.
  • dump (bytes or Message or dict) – The content of a heap-dump file (bytes) or a compressed pickle (bytes, starts with b"BZh9") or a MIME-message with content type application/x.python-heapdump or the already unpickled heap dump dictionary. Exactly one of the two arguments dumpfile and dump must be given.
Returns:

the preprocessed heap dump dictionary
Return type:

dict
pyheapdump.debug_dump(dumpfile, dump=None, debugger_options=None, invoke_debugger_func=None)

Load and debug a Python heap dump

This function loads and debugs a Python dump. First it calls load_dump() to load the dump and then it invokes a debugger to analyse the loaded dump.

This function currently looks for the following debuggers in the given order. It uses the first debugger found:

  • pydevd, the debugger from the PyDev eclipse plugin. It requires PyDev version 3.3.3 or later and you must start the PyDev debug server in the debug perspective of eclipse.
  • pdb, the debugger from the Python library. Unfortunately, pdb supports neither threads nor tasklets.

You can invoke this function directly from your shell:

$ python -m pyheapdump --help

Arguments

Parameters:
  • dumpfile (str or file-like) – the name of the heap-dump file or a file-like object open for reading bytes.
  • dump (bytes or Message or dict) – The content of a heap-dump file (bytes) or a compressed pickle (bytes, starts with b"BZh9") or a MIME-message with content type application/x.python-heapdump or the already unpickled heap dump dictionary. Exactly one of the two arguments filename and dump must be given.
  • debugger_options – Subject to change. Intentionally not documented.
  • invoke_debugger_func – Subject to change. Intentionally not documented.
Returns:

None
class pyheapdump.MimeMessageFormatter(ensure_ascii=True, unixfrom=False, preamble=None)

Bases: object

This class formats a heap-dump as a MIME-message.

Methods:

__call__(pickle, headers)

Format the pickle and the headres as a flat byte string.

Parameters:
  • pickle (bytes) – the compressed pickle of the heap dump
  • headers (collections.Mapping) – a mapping
Returns:

a MIME message
Return type:

bytes

Pyheapdump.__main__

Debug heap dumps.

Warning

This is alpha quality code.

pyheapdump.__main__.main(argv=None)

Debug a Python heap dump file.

You can invoke this function using the following command:

python -m pyheapdump [OPTIONS] pyheapdump

Use the option ‘-h’ to get help:

python -m pyheapdump -h