Test Cases

Test cases is a list of dicts

{
    'test_name': 'Default',
    'params': {'param_name': 'param_value'},
    'outputs': {
        'test_output': [
            'ggo_out.complex.Sheet1.tsv',
            'galaxygetopt/tests/test_file.tsv'],
},

The test name is simply a human readable name for the test and printed in any error statements. params is a dict of commands passed during command line invocation. outputs is a dict with keys representing an output as defined in the outputs=[ section, and values being a list of two values.

• The first value is the generated filename, you may need to run the tool to identify this
• The second value is a path to a comparison file

These files will be diffd. More than one line will result in failure, zero lines (identical files) will be a pass.

In Galaxy

A test case like above generates:

<tests>
    <test>
        <param name="param_name" value="param_value"/>
        <output file="galaxygetopt/tests/test_file.tsv" name="test_output"/>
    </test>
</tests>

At the Command Line

Tests can be generated via python script.py --gen_test. Here is an example generated test:

#!/usr/bin/env python
import difflib
import unittest
import shlex, subprocess
import os

class TestScript(unittest.TestCase):

    def setUp(self):
        self.base = ['python', '../complex.py']
        self.tests = [{'outputs': {'test_output': ['ggo_out.complex.Sheet1.tsv', 'galaxygetopt/tests/test_file.tsv']}, 'command_line': '--param_name param_value', 'test_name': 'Default'}]

    def test_run(self):
        for test_case in self.tests:
            failed_test = False
            try:
                current_command = self.base + \
                    shlex.split(test_case['command_line'])
                # Should probably be wrapped in an assert as well
                subprocess.check_call(current_command)

                for fileset in test_case['outputs']:
                    failed_test = self.file_comparison(
                            test_case['outputs'][fileset][0],
                            test_case['outputs'][fileset][1])
            except:
                raise
            self.assertFalse(failed_test)

    def file_comparison(self, test_file, comp_file):
        failed_test = False
        diff=difflib.unified_diff(open(test_file).readlines(),
                           open(comp_file).readlines())
        try:
            while True:
                print diff.next(),
                failed_test = True
        except:
            pass
        try:
            # Attempt to remove the generated file to cut down on
            # clutter/other bad things
            os.unlink(test_file)
        except:
            pass
        return failed_test

if __name__ == '__main__':
    unittest.main()