Web2py Test Runner ================== About ----- Runs all tests that exist in the ``tests`` directory, up to one sub-level deep. Also runs all doctests from python files in the ``controllers`` directory. Your tests will be executed in a local environment, with access to everything included in this file. Create some tests and place them in the ///.py Follow the standard ``unittest.TestCase`` class unit tests. Your tests will be responsible for creating their own fresh environment of web2py. They have two options for doing this. 1. Create a fake environment, execute the models and controller manually. 2. Use webtest, and hook into web2py's ``wsgibase``. .. warning:: You must use web2py version 1.77.4 or greater. test_runner will fail if you do not have at least these versions! Injection --------- With either of the two methods mentioned above, the variables ``TEST_APP`` and ``TEST_METHOD`` will be injected into the fake WSGI environment. ``TEST_APP`` will contain a secret key that you define when executing your tests. ``TEST_METHOD`` will contain the method used to execute the models. This will either be ``WebTest`` if you use the webtest method, or ``FakeEnv`` if you use the new_env method. .. seealso: More details on setting ``TEST_APP_KEY`` in the :ref:`execution` section. From your model, you can verify the existence of this key and optionally perform logic checks based on this, such as defining a test database as your main object instead of using your production database. Here is an example of determining the existence of this key.:: # in db.py if request.get('wsgi', {}).get('environ', {}).get('TEST_APP', None) == '': TEST_APP = True else: TEST_APP = False if TEST_APP: db = DAL('sqlite:memory:') else: db = DAL('sqlite://my_real_database.db') .. warning:: You might want to remove these checks in a production environment. The secret key is there to provide some security in case you forget, but the check does create a little overhead. .. note:: This is the recommended way of using a test database. There are two reasons. 1. If you have quite a few functions declared in your models that rely on your db object, if you want to test these functions, they will break if you use the copy_db function, since they will refer to your actual database in this case. 2. The copy_db function introduces lag time into your testing, since it will effectively be creating a brand new database on each test function. Another approach is to use the provided ``copy_db`` function, which will make a `sqlite:memory:` DAL instance using the tables provided by your main `db` object. More info on this in the :ref:`fake_env` section. .. _fake_env: Fake Environment ---------------- This class contains helper functions for creating a brand new environment, including a testing database. In your ``TestCase.setUp()`` function you can include the following code.:: def setUp(self): self.env = new_env(app='init', controller='default') self.db = copy_db(self.env, db_name='db', db_link='sqlite:memory') .. note:: If ``controller==None`` it will only execute your models. This will create a ``self.env`` that holds everything in the web2py environment. Optionally, you can copy your main database by using the ``copy_db`` function. This way you can operate on a blank database (preferably in memory). Let's test the `hello_world.py/index` function.:: # in init/controllers/default.py def index(): return dict(msg = "hello world") # in init/tests/controllers/default.py # inside a TestCase class... def test_hello(self): res = self.env['index']() assert res['msg'] == "hello world" For convenience, there is a setup function that returns both a env and a copydb.:: def setUp(): self.env, self.db = setup('init', 'default', db_name='db', db_link='sqlite:memory:') Instead of having to access the globals in the ``self.env`` dictionary, you can use the following code in a test to extract the dict into the locals() namespace.:: def test_hello(self): exec '\n'.join("%s=self.env['%s']" % (i, i) for i in self.env) res = index() assert res['msg'] == "hello world" For testing against a form or needing specific ``request.args`` to be set, there are a couple of helper functions. You may use them as follows:: def test_post_hello(self): set_args(self.env, 'arg1', 'arg2', 'arg3') # Testing the unit test runner... I know :) assert self.env['request'].args(0) == 'arg1' set_vars(self.env, type='post', username = "admin", password = "rockets", _formname = "login", ) assert self.env['request'].vars.username == 'admin' assert self.env['request'].post_vars.username == 'admin' # If you are dealing with a form made by Crud, you should use the following function. set_crudform("auth_user", {"username": "admin", "password": "rockets"}, self.env['request'], action = "create", record_id = None) WebTest ------- You can use paste.WebTest to test your web2py app. With WebTest you are testing just the response from your application. This is like testing your app as if you were a web browser, since you will not have access to any of your apps internal functions. You can find more documentation on how to use WebTest here http://pythonpaste.org/webtest/ Using the url from above, here is an example TestCase using WebTest.:: def __init__(self, *args, **kwargs): unittest.TestCase.__init__(self, *args, **kwargs) self.app = webtest() def test_hello(self): res = self.app.get('/init/default/index') assert 200 == res.status_code assert 'hello world' in res.body .. _execution: Execution --------- Create a ``execute_test.py`` script with the following:: from web2py_utils.test_runner import run run(path = '/path/to/web2py', app = 'welcome', test_key = 'superSecret', test_options = {'verbosity': 3}, test_report = '/path/to/a/report.txt',) .. note:: If you are executing this file in /path/to/web2py, you do not need to pass the ``path`` keyword, since it can determine the existence of gluon and figure out the rest from there. Then it is as simple as running:: python execute_tests.py Refer below for the list of all options that can be passed to the ``run`` function. Keyword Arguments: - path -- The path to the web2py directory. - app -- Name of application to test. - test_key -- Secret key to inject into the WSGI environment - test_options -- Dictionary of options to pass along to the test runner This is gets passed to either Nosetests or TextTestRunner, depending on what is available. eg: {'verbosity': 3} - test_report -- Path to a file, all output will be redirected here. This redirects sys.stdout and sys.stderr to this file. - coverage_report -- If ``test_report`` is none, this will print the coverage report to this file, if coverage is installed. - coverage_exclude -- List of omit_prefixes. If a filepath starts with this value it will be omitted from the report - coverage_include -- List of files to include in the report. - DO_COVER -- If False, will disable coverage even if it is installed - DO_NOSE -- If False, will use unittest.TestTextRunner, even if nosetests is is installed. .. warning:: If you receive the following trackback:: Traceback (most recent call last): File "test.py", line 5, in test_options={'verbosity': 3},) File "./web2py_utils/web2py_utils/test_runner.py", line 322, in run cov = coverage() File "/usr/lib/python2.6/dist-packages/coverage.py", line 303, in __init__ raise CoverageException("Only one coverage object allowed.") coverage.CoverageException: Only one coverage object allowed. You do not have the correct version of coverage installed. Make sure that you install the version from the PYPI, and not from your package manager. Credits ------- - Thank you to Jon Romero for the idea for this module, as well as - Jonathan Lundell for the idea on integrating coverage. - Mathieu Clabaut for doc test support