str(object=’‘) -> string
Return a nice string representation of the object. If the argument is a string, the return value is the same object.
str(object=’‘) -> string
Return a nice string representation of the object. If the argument is a string, the return value is the same object.
str(object=’‘) -> string
Return a nice string representation of the object. If the argument is a string, the return value is the same object.
list() -> new empty list list(iterable) -> new list initialized from iterable’s items
list() -> new empty list list(iterable) -> new list initialized from iterable’s items
list() -> new empty list list(iterable) -> new list initialized from iterable’s items
str(object=’‘) -> string
Return a nice string representation of the object. If the argument is a string, the return value is the same object.
str(object=’‘) -> string
Return a nice string representation of the object. If the argument is a string, the return value is the same object.
list() -> new empty list list(iterable) -> new list initialized from iterable’s items
list() -> new empty list list(iterable) -> new list initialized from iterable’s items
str(object=’‘) -> string
Return a nice string representation of the object. If the argument is a string, the return value is the same object.
str(object=’‘) -> string
Return a nice string representation of the object. If the argument is a string, the return value is the same object.
str(object=’‘) -> string
Return a nice string representation of the object. If the argument is a string, the return value is the same object.
str(object=’‘) -> string
Return a nice string representation of the object. If the argument is a string, the return value is the same object.
str(object=’‘) -> string
Return a nice string representation of the object. If the argument is a string, the return value is the same object.
ScriptRunner instance.
Replaces invalid XML characters with ‘?’.
Escape a string for an XML CDATA section.
Return string with trailing whitespaces from each line removed.
Return properly quoted string according its content.
Compare two version strings, returns: 0 for lver = rver -1 for lver < rver 1 for lver > rver
Make sure that string is unicode. Accepts Unicode, strings encoded in UTF-8 or None and returns Unicode string or None.
Make sure that string is encoded in UTF-8. Accepts Unicode, strings or None. If paremeter is unicode, it’s returned as string encoded in UTF-8. String or None parameter is returned as is.
Run external program using Popen from subprocess module and capture stdin and stderr.
| Parameters: |
|
|---|---|
| Returns: | Tuple of (returncode, stdout, stderr) |
Recipe for test execution against specific Firebird version and OS platform.
| Parameters: |
|
|---|
Return recipe data as string definition of Python dictionary. This string could be evaluated back to Python dictionary and passed to TestVersion constructor to recreate the test version instance. String is encoded in UTF-8 if necessary.
Note
Only attributes that haven’t DEFAULT values are included.
Returns platforms supported by this test version as list of platform names.
Execute the recipe.
Important
Test run outcome is stored in Result instance.
| Parameters: |
|---|
Test definition.
Can contain multiple TestVersion recipes for test execution against different Firebird versions on various OS platforms.
| Parameters: |
|
|---|
Add TestVersion instance to test.
| Parameters: | version – TestVersion instance. |
|---|
Return test data as string definition of Python dictionary. This string could be evaluated back to Python dictionary and passed to Test constructor to recreate the test instance. String is encoded in UTF-8 if necessary.
Note
Only attributes that haven’t DEFAULT values are included.
Return test name.
Test name is last part of test ID.
Return test recipe suitable for specified platform and Firebird version.
| Parameters: | |
|---|---|
| Returns: | TestVersion or None. |
Base class for test resources.
| Parameters: | id (string) – Resource ID. |
|---|
Return resource data as string definition of Python dictionary. This string could be evaluated back to Python dictionary and passed to Resource constructor to recreate the resource instance. String is encoded in UTF-8 if necessary.
Resource finalization
Nothing in this class.
Class method to create right Resource instance from parameters.
| Parameters: | kind (string) – Resource class specification. |
|---|
Helper method to note fail of resource setup/cleanup in Result instance.
| Parameters: |
|---|
Resource initialization.
Nothing in this class.
Test resource that provides Firebird USER.
| Parameters: |
|---|
Remove specified Firebird user via GSEC.
Create Firebird user with specified name and password via GSEC.
Container for tests and sub-suites to organize tests in hierarchical structure.
| Parameters: |
|---|
Delete all tests and sub-suites from instance.
Return suite ID.
Consists from suite names from root to this one separated by dot.
Return parent suite or None
Return all tests for this suite (including tests from sub-suites).
Load tests and sub-suites from path.
Save test on disk.
| Parameters: | test_id – Test ID. |
|---|
Run results archive.
Archive is structured in subdirectories. Each directory is named after Firebird version and contains result collection dump (pickle) files with name:
<number_of_tests_executed>-<platform><cpuarch>-<fbarch>-<person-id>-<sequence>.trf
| Parameters: | archive_path (string) – Directory with archive. |
|---|
Remove specified run results from archive.
| Parameters: | result – File path in archive or RunResult instance stored in archive. |
|---|
Get File path for archived results. :param RunResult results: Run results. :returns: File path.
Get list of result files in archive. :returns: Sorted list of filenames (including FB version subdirectory name).
Get list of run results from archive.
| Parameters: | version (string) – Returns only results for specified Firebird version. |
|---|---|
| Returns: | List of RunResult instances. |
Store run results into archive. :param RunResult results: Run results.
Result from test run or resource setup/cleanup.
Note
Also acts as dictionary of annotations.
| Parameters: |
|---|
Set ERROR outcome with specified cause and annotations.
| Parameters: |
|---|
Set FAIL outcome with specified cause and annotations.
| Parameters: |
|---|
Return cause.
Return test run time as (float) number of secods.
Return test run time as datetime.time.
Set outcome to ERROR and annotate with exception info and traceback.
| Parameters: |
|
|---|
Set cause.
Set Result outcome.
| Parameters: |
|---|
Collection of test/resource Results.
Designed to hold Results from single QA run. Acts also as dictionary of Results with test/resource ID as key and Result object as value.
Add result
Remove all results.
Return number of ERROR outcomes
Return list of Results with ERROR outcome
Return number of FAIL outcomes
Return list of Results with FAIL outcome
Get list outcomes from stored Results
Return number of PASS outcomes
Return list of Results with PASS outcome
Return list of Results with SKIPPED outcome
Return number of SKIPPED outcomes
Return list of Results with UNTESTED outcome
Return number of UNTESTED outcomes
Load results from file.
| Parameters: | filename (string) – Filename with pickled Result. |
|---|
Print results summary to stdout.
QA Execution engine.
| Parameters: | repository – Repository instance. |
|---|
Run tests.
| Parameters: | |
|---|---|
| Returns: | RunResults object with results. |
Warning
QA environment must be properly configured!
Set QA person identification.
| Parameters: | person (string) – QA person name. |
|---|
Sets person_name and person_id attributes: person_name = person in lowercase person_id = first two letters from uppercased person
If person is “Unknown”, the person_is is set to “XX”.
Configures the QA environment to run on specified Firebird installation.
| Parameters: |
|---|
Helper class to organize script running code. Instantiated as singleton instance script_runner.
Analyze test run results and produce HTML output.
| Parameters: |
|---|
Filters out annotations we don’t want to compare.
Called by run_archive() for command execution.
Called by run_archive() for command execution.
Called by run_archive() for command execution.
Called by run_archive() for command execution.
Called by run_database() for command execution.
Called by run_database() for command execution.
Called by run_update() for command execution.
Called by run_update() for command execution.
Compare two results for single test. Compares kind, outcome and annotations.
| Parameters: | |
|---|---|
| Returns: | True if both results are equal. |
Connect to local or remote Test Repository.
| Parameters: | options – Command-line options. |
|---|---|
| Returns: | Repository instance. |
Return list of Result file filenames stored in directory.
| Parameters: | directory (string) – directory with result files. |
|---|---|
| Returns: | List with filenames. |
Return string that could be used as key to group tests by platform, cpu, FB architecture and run sequence.
Get Subversion login credentials from user.
Create HTML files with test run analysis.
| Parameters: |
|
|---|
Print IDs of tests that ended with ERROR.
| Parameters: |
|
|---|
Print IDs of tests that ended with FAIL.
| Parameters: |
|
|---|
Print information from result file.
| Parameters: |
|
|---|
Print IDs of tests that ended with SKIPPED.
| Parameters: |
|
|---|
Print IDs of tests that ended with UNTESTED.
| Parameters: |
|
|---|
Called by run_analyze() for command execution.
Called by run_server() for command execution.
Called by run_tests() for command execution.
Called by run_view() for command execution.
Notification callback from pysvn.
CLI Script function for test execution.
This is a ‘main’ function called by fbt_run script. Runs all tests in test reporsitory or specified test/suite. Result from test run is stored in current directory:
results.trf : Pickled RunResults instance. Pickled RunResults instance is also stored in ‘archive’.
results.xml : XUNIT XML report (when -x or –xunit is specified).
usage: fbt_run [options] [suite_or_test_name]
Be more verbose
Set verbosity; –verbosity=2 is the same as -v
Be less verbose
Location of Firebird binary tools (like gbak etc.).
Location for temporary databases used by tests.
Suite or test name or name of file with suite/test names to skip.
If specified, last run result file is copied to archive.
If specified, runs only tests that don’t PASSed in last run.
If specified, connects to remote fbtest server and runs specified tests on it.
If specified, updates last run results with re-run results.
SYSDBA password. (default ‘masterkey’)
Firebird or fbtest host machine identification. (default ‘localhost’)
QA person name. First two letters are used as person ID.
Firebird architecture (‘SS’, ‘CS’ or ‘SC’). (default ‘SS’ - SuperServer)
Run sequence number. Important to disctinguish repeated runs af the same tests on single ‘target’. Used by fbt_analyze. (default ‘1’)
Provides test results also in the standard XUnit XML format.
Test results file to be used as expeted outcomes
CLI Script function for test execution.
This is a ‘main’ function called by fbt_server script.
usage: fbt_server [options]
Location of Firebird binary tools (like gbak etc.).
Location for temporary databases used by tests.
SYSDBA password. (default ‘masterkey’)
Firebird host machine identification. (default ‘localhost’)
QA person name. First two letters are used as person ID.
Firebird architecture (‘SS’, ‘CS’ or ‘SC’). (default ‘SS’ - SuperServer)
If specified, automatically registers RPyC slave server.
CLI Script function for test analysis.
This is a ‘main’ function called by fbt_analyze script. Reads result file or all result files in specified directory and writes HTML report in current or specified directory.
usage: fbt_analyze [options] <file_or_directory>
Location where HTML report should be stored.
Show only diffs on detail pages.
CLI Script function to update metadata in Result file(s).
This is a ‘main’ function called by fbt_update script. Reads result file or all result files in specified directory and updates metadata with specified value(s).
usage:
fbt_update {result,repository} [options]
result Change result file metadata.
repository Update Test Repository.
Options for fbt_update result:
Results file or directory with result files.
Firebird architecture (‘SS’, ‘CS’ or ‘SC’).
QA person name. First two letters are used as person ID.
Run sequence number. Important to disctinguish repeated runs of the same tests on single ‘target’. Used by fbt_analyze.
Options for fbt_update repository:
None.
CLI Script function to update metadata in Result file(s).
This is a ‘main’ function called by fbt_view script. Reads result file or all result files in specified directory and prints information about stored results.
usage: fbt_view <file_or_directory>
Store processed test results in the standard XUnit XML format.
Print cause of fails and errors.
Print details for fails and errors.
CLI Script function to maintain archive of Result file(s).
This is a ‘main’ function called by fbt_archive script.
usage:
fbt_update {list,save,retrieve,delete} [options]
list List result(s) in archive.
save Save result(s) to archive.
retrieve Retrieve result(s) from archive.
delete Delete result(s) from archive.
If specified, connects to remote fbtest server (and archive).
Remote fbtest host machine identification. (default ‘localhost’)
Options for fbt_archive list:
None.
Options for fbt_archive save:
Results file.
Options for fbt_archive retrieve:
Retrieve results only for specified Firebird version.
Retrieve results only for currently tested Firebird version.
Location where result file(s) should be copied.
Retrieve results only for specified QA person name. First two letters are used as person ID.
Retrieve results only for specified Firebird architecture (‘SS’, ‘CS’ or ‘SC’).
Retrieve results only for specified run sequence number.
Options for fbt_archive delete:
Delete results only for specified Firebird version.
Delete results only for currently tested Firebird version.
Delete results only for specified QA person name. First two letters are used as person ID.
Delete results only for specified Firebird architecture (‘SS’, ‘CS’ or ‘SC’).
Delete results only for specified run sequence number.
CLI Script function to maintain database archive of Result file(s).
This is a ‘main’ function called by fbt_db script.
usage:
fbt_db [-h] [-w PASSWORD] [-o HOST]
[-d DATABASE] {import,create} ...
optional arguments:
-h, --help show this help message and exit
-w PASSWORD, --password PASSWORD
SYSDBA password
-o HOST, --host HOST Firebird host machine identification
-d DATABASE, --database DATABASE
Archive database name
Commands:
{import,create} Use <command> --help for more information about
command.
create Creates archive database.
import Import result(s) to database.
Firebird machine identification. (default ‘localhost’)
FBTest archive database name.
SYSDBA password. (default ‘masterkey’)
Options for fbt_archive create:
None.
Options for fbt_archive import:
Results file.