Do not hang around in Pythran code base without your developer guide! It is the compass that will guide you in the code jungle!
This document is a never ending work-in-progress draft. Please contribute!
Additional packages may be needed to use pythran to its fullest: colored logs, tests on several cores, doc generation
$> sudo apt-get install python-setuptools python-pytest-xdist python-docutils $> sudo pip install colorlog pytest-pep8
Pythran can be configured with a rc file. An example is found in pythran/pythran.cfg. Look at it! To customize it:
$> cp pythran/pythran.cfg ~/.pythranrc
In particular, you may want to add
-g -O0 to the
NT2 is required to compile the C++ code generated by Pythran. It is automatically installed along Pythran. If you don’t have NT2 installed in a system directory and you try to run Pythran directly from the source directory, the NT2 includes won’t be found.
A trick to ease the development and to enable running directly from the source
is to install Pythran once and to add
-I<install_prefix>/lib/<python-version>/site-packages/pythran to the
cppflags in order to point to the NT2 headers.
All Python code must be conform to the PEP 8, and the
pep8 command must not
yield any message when run on our database. Additionally, avoid backslashes,
and try to make your code as concise as possible.
C++ code use spaces (no tabs) and a tab width of 4.
Listing the top level directory yields the following entries:
- The files that describes what gets installed, that holds
PyPIentries and such.
- If you’re reading this document, you know what it’s all about!
MANUALis the user documentation and
DEVGUIDEis the developer documentation.
- Boring but important stuff.
- Describe additional stuff to package there.
- Quick introduction and description of _pythran_. The
README.rstfile is just a symbolic link that pleases
- The source of all things.
- The source of all issues.
- Where C++ back-end lies.
- Where Python scripts calling the
pythran uses the
unittest module to manage test cases. The whole
validation suite is run through the command:
$> python setup.py test
It is tested using the pytest <http://pytest.org/latest/> package:
$> apt-get install python-pytest
To run it faster we use the pytest extension xdist from debian package python-pytest-xdist , the test suite will run using all available cores. Otherwise it might run very slowly, something like four hours on a decent laptop :’(.
pep8 is checked using the pytest extension
pytest-pep8. It can be
$> sudo pip install pytest-pep8
Note that it is still possible to use the
pytest module directly, for
instance to pass a subset of the test suite:
$> PYTHONPATH=. py.test -n 8 pythran/tests/test_list.py
runs all the tests found in
Only compiler tests can be check using test filtering:
$> PYTHONPATH=. py.test -n 8 pythran/tests -m "not module"
There are two kinds of tests in
unit tests that test a specific feature of the implementation. Such tests are listed as method of a class deriving from
test_env.TestEnvand must call the
run_test(function_to_translate, *effective_parameters, **name_to_signature)method . It translates
function_to_translateinto a native function using the type annotations given in the
name_to_signaturedictionary, runs both the python and the native version with
effective_parametersas arguments and asserts the results are the same.
See examples in
pythran/tests/test_base.pyfor more details.
test cases that are just plain python modules to be converted in native module by
pythran. It is used to test complex situations, codes or benchmarks found on the web etc. They are just translated, not run. These test cases lie in
pythran/tests/cases/and are listed in
The C++ code generated by
pythran relies on a specific back-end,
pythonic. It is a set of headers that mimics Python’s intrinsics and
collections behavior in C++. It lies in
pythran/pythonic/. There is one
directory per module, e.g.
pythran/pythonic/numpy for the
and one file per function, e.g.
pythran/pythonic/numpy/ones.hpp for the
numpy.ones function. Type definitions are stored in the seperate
pythran/pythonic/types directory, one header per type. Each function header
#includ-able independently, i.e. it itself includes all the type
and function definition it needs. This helps keeping compilation time low.
All Pythran functions and types live in the
pythonic namespace. Each extra
module defines a new namespace, like
pythonic::random, and each type is defined in the
DEFINE_FUNCTOR macros from
pythonic/utils/functor.hpp is commonly used to convert functions into
functors and put them into the mandatory
The pythonic runtime can be used without Python support, so it is important to
protect all Python-specific stuff inside
All methods are represented by functions in Pythran. The associated
pseudo-modules are prefixed and suffixed by a double underscore
__, as in
Benchmarking and Testing¶
Stand-alone algorithms are put into
pythran/tests/cases. They must be valid
Pythran input (including spec annotations). To be taken into account by the
validation suite, they must be listed in
pythran/tests/test_cases.py. To be
taken into account by the benchmarking suite, they must have a line starting
#runas directive. Check
pythran/tests/matmul.py for a complete
To run the benchmark suite, one can rely on:
$> python setup.py bench --mode=<mode>
where <mode> is one among:
- Uses the interpreter used to run
- Uses the Pythran compiler.
- Uses the Pythran compiler in OpenMP mode.
All measurements are made using the
timeit module. The number of iterations
is customizable through the
|Add support for a new module:|
|Add a new analysis:|
|Push changes into the holy trunk:|