API documentation

The main entry points to the iospec package is the function iospec.parse(). it returns a iospec.IoSpec parse tree.

Reference

iospec.parse(file, commands=None)

Parse the content of string or file object.

Returns the parse tree as a iospec.IoSpec instance.

The IoSpec parse tree

The result of iospec.parse() is a iospec.IoSpec instance that behaves mostly like a sequence of test case nodes. Consider the string of iospec data:

data = """
Say your name: <John>
Hello, John!
"""

The contents can be parsed as:

from iospec import parse
tree = parse(data)
case = tree[0]

Each test case is a sequence of In/Out strings:

>>> list(case)
[Out('Say your name: '), In('John'), Out('Hello, John!')]

The main AST object

class iospec.types.IoSpec(data=(), *, commands=None, definitions=None)

Root node of an iospec AST.

Args:
data (seq):
A (possibly empty) sequence of test cases.
commands (dict):
A mapping of command names to their respective functions or Command subclasses.
definitions (list):
A list of command definitions declared in the IoSpec source.
Attributes:
has_errors (bool):
True if IoSpec structure has an ErrorTestCase child.
is_simple (bool):
True if all children are instances of SimpleTestCase.
is_expanded (bool):
True if all children are instances of SimpleTestCase that have all input commands expanded. This tells that each child is made of just a sequence of In and Out strings.

TestCase elements

We refer to each run of a program as a “test case”. iospec implements many different test case blocks in order to adapt to different situations. Perhaps the most simple block is a SimpleTestCase

class iospec.types.SimpleTestCase(data=(), *, priority=None, lineno=None, **kwds)

Regular input/output test case.

class iospec.types.InputTestCase(data=(), *, inline=True, **kwds)

Blocks that contain only input entries in which o outputs should be computed by third parties.

It is created by the @input and @plain decorators of the IoSpec language.

class iospec.types.ErrorTestCase(data=(), *, error_message='', error_type='runtime', **kwds)

Error test cases are created using a decorator:

@runtime-error
    The main body has a regular block of input/output interactions.

    This error describes the most common case of failure: when an error
    is triggered during a program execution. Errors can be anything from
    seg-faults to exceptions, or buffer overflows, etc.

    @error
        optional block of messages displayed to stderr


@timeout-error
    The main body has a regular block of input/output interactions.

    Timeout errors happen when execution takes longer than expected.
    It doesn't define an error message since the program did not fail,
    but the runner decided to terminate its execution.


@build-error
    The main body has a block of messages that should be displayed to
    stderr. Build errors cannot have a regular block of IO interactions
    since they happen prior to program execution.

    Build errors happen when program is being prepared to execute. Can
    be a syntax error, compilation error or anything that prevents
    the program to run.

Error test cases check if a program fails in some specific way. It is also necessary in order to use the IOSpec format to describe how a program actually ran, in case an error is found.

class iospec.types.TestCase(data=(), *, priority=None, lineno=None, **kwds)

Base class for all test cases.

Args:
data:
A sequence of In, Out and Command strings.
priority (float):
Relative priority of this test case for input expansion.
lineno (int):
The line number for this test case in the IoSpec source.