Web2py Test Runner

About

Runs all tests that exist in the tests directory, up to one sub-level deep. Also runs all doctests from python files in the controllers directory.

Your tests will be executed in a local environment, with access to everything included in this file.

Create some tests and place them in the /<yourapp>/<tests>/<filename>.py

Follow the standard unittest.TestCase class unit tests.

Your tests will be responsible for creating their own fresh environment of web2py. They have two options for doing this.

1. Create a fake environment, execute the models and controller manually.
2. Use webtest, and hook into web2py’s wsgibase.

Warning

You must use web2py version 1.77.4 or greater.

test_runner will fail if you do not have at least these versions!

Injection

With either of the two methods mentioned above, the variables TEST_APP and TEST_METHOD will be injected into the fake WSGI environment.

TEST_APP will contain a secret key that you define when executing your tests. TEST_METHOD will contain the method used to execute the models. This will either be WebTest if you use the webtest method, or FakeEnv if you use the new_env method.

From your model, you can verify the existence of this key and optionally perform logic checks based on this, such as defining a test database as your main object instead of using your production database.

Here is an example of determining the existence of this key.:

# in db.py
if request.get('wsgi', {}).get('environ', {}).get('TEST_APP', None) == '<my secret key>':
    TEST_APP = True
else:
    TEST_APP = False

if TEST_APP:
    db = DAL('sqlite:memory:')
else:
    db = DAL('sqlite://my_real_database.db')

Warning

You might want to remove these checks in a production environment. The secret key is there to provide some security in case you forget, but the check does create a little overhead.

Note

This is the recommended way of using a test database. There are two reasons.

1. If you have quite a few functions declared in your models that rely on your db object, if you want to test these functions, they will break if you use the copy_db function, since they will refer to your actual database in this case.
2. The copy_db function introduces lag time into your testing, since it will effectively be creating a brand new database on each test function.

Another approach is to use the provided copy_db function, which will make a sqlite:memory: DAL instance using the tables provided by your main db object. More info on this in the Fake Environment section.

Fake Environment

This class contains helper functions for creating a brand new environment, including a testing database.

In your TestCase.setUp() function you can include the following code.:

def setUp(self):
    self.env = new_env(app='init', controller='default')
    self.db = copy_db(self.env, db_name='db', db_link='sqlite:memory')

Note

If controller==None it will only execute your models.

This will create a self.env that holds everything in the web2py environment.

Optionally, you can copy your main database by using the copy_db function. This way you can operate on a blank database (preferably in memory).

Let’s test the hello_world.py/index function.:

# in init/controllers/default.py
def index():
    return dict(msg = "hello world")

# in init/tests/controllers/default.py
# inside a TestCase class...
def test_hello(self):
    res = self.env['index']()

    assert res['msg'] == "hello world"

For convenience, there is a setup function that returns both a env and a copydb.:

def setUp():
    self.env, self.db = setup('init', 'default',
                            db_name='db', db_link='sqlite:memory:')

Instead of having to access the globals in the self.env dictionary, you can use the following code in a test to extract the dict into the locals() namespace.:

def test_hello(self):
    exec '\n'.join("%s=self.env['%s']" % (i, i) for i in self.env)

    res = index()

    assert res['msg'] == "hello world"

For testing against a form or needing specific request.args to be set, there are a couple of helper functions. You may use them as follows:

def test_post_hello(self):
    set_args(self.env, 'arg1', 'arg2', 'arg3')

    # Testing the unit test runner... I know :)
    assert self.env['request'].args(0) == 'arg1'

    set_vars(self.env, type='post',
        username = "admin",
        password = "rockets",
        _formname = "login",
    )

    assert self.env['request'].vars.username == 'admin'
    assert self.env['request'].post_vars.username == 'admin'

    # If you are dealing with a form made by Crud, you should use the following function.

    set_crudform("auth_user",
        {"username": "admin",
        "password": "rockets"},
        self.env['request'],
        action = "create",
        record_id = None)

WebTest

You can use paste.WebTest to test your web2py app. With WebTest you are testing just the response from your application. This is like testing your app as if you were a web browser, since you will not have access to any of your apps internal functions.

You can find more documentation on how to use WebTest here

http://pythonpaste.org/webtest/

Using the url from above, here is an example TestCase using WebTest.:

def __init__(self, *args, **kwargs):
    unittest.TestCase.__init__(self, *args, **kwargs)
    self.app = webtest()

def test_hello(self):
    res = self.app.get('/init/default/index')

    assert 200 == res.status_code
    assert 'hello world' in res.body

Execution

Create a execute_test.py script with the following:

from web2py_utils.test_runner import run

 run(path            = '/path/to/web2py',
     app             = 'welcome',
     test_key        = 'superSecret',
     test_options    = {'verbosity': 3},
     test_report     = '/path/to/a/report.txt',)

Note

If you are executing this file in /path/to/web2py, you do not need to pass the path keyword, since it can determine the existence of gluon and figure out the rest from there.

Then it is as simple as running:

python execute_tests.py

Refer below for the list of all options that can be passed to the run function.

Keyword Arguments:

• path – The path to the web2py directory.

• app – Name of application to test.

• test_key – Secret key to inject into the WSGI environment

• test_options – Dictionary of options to pass along to the test runner

  This is gets passed to either Nosetests or TextTestRunner, depending on what is available.

  eg: {‘verbosity’: 3}

• test_report – Path to a file, all output will be redirected here.

  This redirects sys.stdout and sys.stderr to this file.

• coverage_report – If test_report is none, this will print the coverage

  report to this file, if coverage is installed.

• coverage_exclude – List of omit_prefixes. If a filepath starts with this

  value it will be omitted from the report

• coverage_include – List of files to include in the report.

• DO_COVER – If False, will disable coverage even if it is installed

• DO_NOSE – If False, will use unittest.TestTextRunner, even if nosetests is

  is installed.

Warning

If you receive the following trackback:

Traceback (most recent call last):
File "test.py", line 5, in <module>
    test_options={'verbosity': 3},)
File "./web2py_utils/web2py_utils/test_runner.py", line 322, in run
    cov = coverage()
File "/usr/lib/python2.6/dist-packages/coverage.py", line 303, in __init__
    raise CoverageException("Only one coverage object allowed.")
coverage.CoverageException: Only one coverage object allowed.

You do not have the correct version of coverage installed. Make sure that you install the version from the PYPI, and not from your package manager.

Credits

• Thank you to Jon Romero for the idea for this module, as well as
• Jonathan Lundell for the idea on integrating coverage.
• Mathieu Clabaut for doc test support