Asynchronous Test Suite

An asynchronous parallel testing suite pulsar.Application. Used for testing pulsar itself but it can be used as a test suite for any other library.

Requirements

• unittest2 for python 2.6
• mock for python < 3.3

Introduction

Create a script on the top level directory of your library, let’s call it runtests.py:

from pulsar.apps import TestSuite

if __name__ == '__main__':
    TestSuite(description='Test suite for my library',
              modules=('regression',
                       ('examples','tests'))).start()

where modules is an iterable for discovering test cases. Check the TestLoader for details. In the above example the test suite will look for all python files in the regression module (in a recursive fashion), and for modules called tests in the example module.

Loading Tests

Loading test cases is accomplished via the TestLoader class. In this context we refer to an object as a module (including a directory module) or a class.

These are the rules for loading tests:

• Directories that aren’t packages are not inspected.
• Any class that is a unittest.TestCase subclass is collected.
• if an object starts with _ or . it won’t be collected, nor will any objects it contains.
• If an object defines a __test__ attribute that does not evaluate to True, that object will not be collected, nor will any objects it contains.

Test Case

Only subclasses of unittest.TestCase are collected by this application. When running a test, pulsar looks for two extra method: _pre_setup and _post_teardown. If the former is available, it is run just before the setUp method while if the latter is available, it is run just after the tearDown method.

API

TestSuite

class pulsar.apps.test.TestSuite(callable=None, description=None, name=None, epilog=None, argv=None, script=None, version=None, parse_console=True, commands_set=None, cfg=None, **kwargs)

An asynchronous test suite which works like a task queue where each task is a group of tests specified in a test class.

Parameters:

• modules –

  An iterable over modules where to look for tests. Alternatively it can be a callable returning the iterable over modules. For example:

  suite = TestSuite(modules=('regression',
                             ('examples','tests'),
                             ('apps','test_*')))
  
  def get_modules(suite):
      ...
  
  suite = TestSuite(modules=get_modules)
  

  If not provided it is set as default to ["tests"] which loads all python module from the tests module in a recursive fashion. Check the the TestLoader for detailed information.

• result_class – Optional class for collecting test results. By default it used the standard unittest.TextTestResult.
• plugins – Optional list of TestPlugin instances.

runner

Instance of TestRunner driving the test case configuration and plugins.

TestPlugin

class pulsar.apps.test.TestPlugin

Base class for plugins which can be added to a TestSuite to extend its functionalities. pulsar.Setting are specified as class attributes and collected into the config attribute.

If the name is not specified or its value validate as True, an additional settings is added to the configuration.

TestLoader

class pulsar.apps.test.TestLoader(root, modules, runner, logger=None)

Classes used by the TestSuite to aggregate tests from a list of paths. The way it works is simple, you give a root directory and a list of submodules where to look for tests.

Parameters:

• root – root path passed by the TestSuite.
• modules –

  list (or tuple) of entries where to look for tests. An entry can be a string indicating the dotted path relative to the root directory or a two element tuple with the same dotted path as first argument and a pattern which files must match in order to be included in the search. A third element in the tuple can be used to specify the top level tag for all tests in this entry. For example:

  modules = ['test', ('bla', None, 'foo'), ('examples','test')]
  

  loads

  • all tests from modules in the test directory,
  • all tests from the bla directory with top level tag foo,
  • all tests from the examples directory matching the test pattern.

  All top level modules will be added to the python path.

• runner – Instance of the test runner.

The are very simple rules followed for importing tests.

checktag(tag, import_tags)

return True if tag is in import_tags.

testmodules(tags=None)

Generator of tag, test modules pairs.

get_tests(path, dotted_path, pattern, import_tags=None, tags=(), parent=None)

Collect python modules for testing and return a generator of tag,module pairs.

Parameters:

• path – directory path where to search. Files starting with _ or . are excluded from the search, as well as non-python files.
• dotted_path – the dotted python path equivalent of path.
• parent – the parent module for the current one. This parameter is passed by this function recursively.

Plugin

class pulsar.apps.test.Plugin

Interface for all classes which are part of of the TestRunner, including TestRunner itself, TestResult and Plugin.

configure(cfg)

Configure the instance. This method is called once just after construction of a TestRunner and can be used to configure the plugin. If it returns something other than None (for example an abort message) it will stop the configuration of all subsequent plugins and quit the test.

Parameters:cfg – instance of pulsar.Config.

on_start()

Called once by TestSuite just before it starts running tests.

on_end()

Called once by TestSuite just before it stops.

startTest(test)

Called just before a test is executed. This is run just before _pre_setup function.

stopTest(test)

Called just after a test has finished. This is run just after the _post_teardown function.

before_test_function_run(test, local)

This function can be used by plugins to manipulate the test behaviour in the process domain where the test run.

after_test_function_run(test, local, result)

Given a test-function instance return a, possibly, modified test instance. This function can be used by plugins to modify the behaviour of test cases. By default it returns test.

TestRunner

class pulsar.apps.test.TestRunner(plugins, stream, writercls=None, descriptions=True, logger=None)

An asynchronous test runner

loadTestsFromTestCase(cls)

Load all test functions for the test class cls.

before_test_function_run(test)

Called before the test run, in the test process domain.

after_test_function_run(test, result)

Called before the test starts.

run_test_function(test, func)

Run function func which belong to test.

Parameters:

• test – test instance or class
• func – test function belonging to test

Returns:

an asynchronous result

TestRequest

class pulsar.apps.test.TestRequest(testcls, tag)

A class which wraps a test case class and runs all its test functions

testcls

A unittest.TestCase class to be run on this request.

tag

A string indicating the tag associated with testcls.

start(worker)

Run all test functions from the testcls using the following algorithm:

• Run the class method setUpClass of testcls if defined.
• Call run_test() for each test functions in testcls
• Run the class method tearDownClass of testcls if defined.

run_test(test, runner)

Run a test function using the following algorithm

• Run _pre_setup() method if available in testcls.
• Run setUp() method in testcls.
• Run the test function.
• Run tearDown() method in testcls.
• Run _post_teardown() method if available in testcls.

run_on_arbiter

pulsar.apps.test.run_on_arbiter(f)

Decorator for running a test function in the arbiter domain. This can be useful to test Arbiter mechanics.