‘epyunit.SystemCalls’ - Module

The module ‘epyunit.SystemCalls’ provides the controlled execution of subprocesses as units.

The features are foreseen to support easy setups for unit and regression tests on complex command line options and their combinations. These are in designed to be used in conjunction with with PyUnit, either within Eclipse and PyDev, or from the command line.

The provided feature set is supported on the platforms: Linux, MacOS, BSD, Solaris, and Windows.

A quick example call is:

python -c 'from epyunit.SystemCalls import SystemCalls;x=SystemCalls(**{"proceed":"trace"});x.callit("myscript.sh xOK")'

or:

python -c 'from epyunit.SystemCalls import SystemCalls;x=SystemCalls(**{"proceed":"trace"});x.callit("myscript.py xOK")'

SECURITY REMARK: Current version supports for subprocesses the ShellMode only. For test environments in R&D this - hopefully - is perfectly OK, else eventual security issues has to be considered.

The SystemCalls class optionally uses subprocess32 when present, this is recommended. See https://pypi.python.org/pypi/subprocess32/.

Class: SystemCalls

class epyunit.SystemCalls.SystemCalls(**kargs)

Wraps system calls for the execution of subprocesses. For supported parameters refer to ‘epyunit.SystemCalls.setkargs()’.

Methods

__init__

SystemCalls.__init__(**kargs)

Prepares the interface for subprocess calls with output cache. The initial setup includes the preparation of the result cache for the response data from stdout and stderr.

Args:

**kargs:

Parameters specific for the operations, passed through to SystemCalls.setkargs [see setkargs].

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception. Success is the complete addition only, thus one failure returns False.

REMARK

When errors occur in buffered mode the output from the called process is possibly not completely cached. In that case switch to ‘dialogue’ mode in order to get an unbuffered console display.

Raises:

passed through exceptions:

callit

SystemCalls.callit(cmdcall, **kargs)

Executes a prepared ‘cmdcall’ synchronous by the a preset function pointer ‘self.myexe’. For the full scope of call parameters including multiple subprocesses use the method [see create].

Args:

cmdcall:

A prepared call, either shell-style call-string, or a list in accordance to the call convention of ‘subprocess’ package. The ‘shell’ parameter is by default set in accordance to the provided type.

**kargs:

Parameters specific for the operations, passed through to SystemCalls.setkargs [see setkargs].

The following are also evaluated instantly within the call prologue:

proceed:

Changes predefined value and dispatches to the subcall.

raw:

Suppress the split of lines for stdout and stderr.

Returns:

Result of call, the format is:

ret[0]::= exit value

ret[1]::= STDOUT as non-processed string.

ret[2]::= STDERR as non-processed string.

REMARK

When errors occur in buffered mode the output from the called process is possibly not completely cached, e.g. in case of exeptions. In that case try switching to ‘dialogue’ mode in order to get an unbuffered console display. This will deliver - hopefully - the complete output.

Raises:

SystemCallsTimeout:

Rised in case of timeout by tmax/tsig. Terminates also the outstanding subprocess.

passed through exceptions:

cancel

SystemCalls.cancel(proc=None)

Cancels a running process.

displayit

SystemCalls.displayit(ret, **kargs)

Streams result list ‘ret’ in selected format to selected outtarget.

Args:

ret: Data to be displayed.

**kargs:

outtarget: The target of display:

str: return as formatted string

stdout: print to sys.stdout

stderr: print to sys.stderr

out: Output for display, valid:

csv : CSV with sparse records

pass : pass through STDOUT and STDERR from subprocess

repr : Python ‘repr()’

str : Python ‘str()’

xml : XML

default:=self.out

Returns:

When successful returns

outtarget==’str’: returns a printable formatted string

outtarget in (‘stdout’, ‘stderr’,):

‘True’: for success

‘False’: for failure

Raises:

passed through exceptions:

get_proceed

SystemCalls.get_proceed(s)

Verifies valid proceed type.

_mode_batch

_mode_dialogue

SystemCalls._mode_dialogue(cmdcall, **kargs)

Internal call reference for processing an interactive dialogue call.

Supports shell-style parameters only.

For further information refer to ‘console’ option of constructor/setkargs.

setkargs

SystemCalls.setkargs(**kargs)

Sets provided parameters for the subprocess call context.

Applicable for the initial call of self.__init__(), and later modification. Called for each start of a subprocess in order to update optional the specific call context modification.

Args:

**kargs: Parameter specific for the operation,

bufsize:

The size of the output buffer for the called subprocess, refer to subprocess.Popen. Default value is -1.

cache:=<cache-param=list>

Defines the type and thus target caching of the output of the raw data. The output streams could be combined by defining multiple types:

cache-param-list:=cache-param[,cache-param-list]
cache-param:=(
    {type:pipe}
  | {type:fout,fpname:''}
  | {type:ferr,fpname:''}
  | {type:fouterr,fpname:''}
)

pipe:None

  the Subprocess.PIPE for stdout and stderr

fout:<fpathname>:

  reads stdout into a file

ferr:<fpathname>:

  reads stderr into a file

default := pipe

console (‘cli’,’dialogue’)

ffs (‘batch’,’ui’,’gtk’, ‘qt’)

cli:

Works in batch mode, particularly the stdin, stdout, and stderr streams are caught into a string buffer by the calling process via a pipe. The content is passed after termination of the called sub-process.

dialogue:

Works without buffered io streams. Thus allows for interaction, but not post-processing.

debug:

Sets debug for rule data flow.

emptyiserr:

Treats passed empty call strings as error. The applied ‘subprocess.Popen()’ treats them as success, which may cover errors in generated call strings, particularly in loops.

default := False

env:

Environment to be passed through to the subprocess.

Default := current

errasexcept:

Passes errors as exceptions, transforms the resuls from subprocesses into Exceptions data. Exits the process.

default := False

exectype:

Type of execution.

inproc:

Calls ‘Popen’ directly from within the process.

bythread:

Starts an intermediate thread within current process and executes ‘Popen’.

byfork

Starts an intermediate process by fork and executes ‘Popen’.

forcecmdcall:

Forces type of the command call option passed to ‘Popen’.

• shell
• list

inp:

Defines the read-in of the output of called subprocess.

communicate:

Calls Subproces.communicate, blocks until completion of subprocess.

readc:

Reads characters from subprocess, does not accurately synchronize for multiple threads onto one output stream.

Displays each character in real-time.

readl:

Reads characters from subprocess, does accurately synchronize for multiple threads onto one output stream in units of lines.

Displays each line in real-time.

default := communicate

out:

Output for display. Supported types are:

csv:

CSV seperated by ‘;’, with sparse records

pass:

Pass through STDOUT and STDERR from subprocess

repr:

Python ‘repr()’

str:

Python ‘str()’

xml:

XML

passerr:

Passes errors from subprocesses transparently through by stdout, stderr, and exit code. Exits the process.

default := False

proceed (‘print’,’trace’, ‘doit’)

print - trace only

trace - execute and trace

doit - execute

raw:

Pass through strings from STDOUT and STDERR unprocessed. Default is to split the lines into a list for post processing line scope.

rules:

Sets the rules object.

synctype:

Type of call synchronization.

async:

Executes the subprocess asynchronously, this e.g. enables in current implementation for platform independent timeouts.

sync:

Executes synchronously, thus blocks any other execution.

tsig:

Termination signal, default is KILL.

tmax:

Timeout in seconds.

useexit:

Use exit code for error detection of subprocess.

default := True

usestderr:

Use ‘sys.stderr’ output for error detection of subprocess. When set to ‘True’, the presence of a string is treated as error condition.

default := False

verbose:

Sets verbose for rule data flow.

Returns:

When successful returns ‘True’, else returns either ‘False’, or raises an exception. Success is the complete addition only, thus one failure returns False.

Raises:

passed through exceptions:

sub_get_lines

SystemCalls.sub_get_lines(fds, fde, **kargs)

Reads non-blocking from stdout and stderr into a buffer and displays each line immediately.

Args:

fds:

File descriptor for the STDOUT of the called subprocess.

fde:

File descriptor for the STDERR of the called subprocess.

**kargs:

rtd:

Returns:

Raises:

splitLines

SystemCalls.splitLines(oldres)

Converts the raw string fields including ‘ ‘ of a return value into line arrays.

Args:

oldres: The raw result of a previous call.

Returns:

Result of call, the format is:

ret[0]::=exit value

ret[1]::=list of lines from former stdout output, each as

a partial non-processed string.

ret[2]::=list of lines from former stderr output, each as

a partial non-processed string.

Raises:

passed through exceptions:

__str__

SystemCalls.__str__()

Prints preset call parameters.

__repr__

SystemCalls.__repr__()

Prints the current representation of call parameters for subprocesses.

Exceptions

exception epyunit.SystemCalls.SystemCallsException

Common error within epyunit.SystemCalls.

exception epyunit.SystemCalls.SystemCallsExceptionSubprocessError(*args, **kargs)

Error from subprocess.

exception epyunit.SystemCalls.SystemCallsTimeout

Timeout of subprocess epyunit.SystemCalls.