gc3libs.backends.shellcmd

Run applications as local processes.

class gc3libs.backends.shellcmd.ShellcmdLrms(name, architecture, max_cores, max_cores_per_job, max_memory_per_core, max_walltime, auth=None, frontend='localhost', transport='local', time_cmd=None, override='False', keyfile=None, ignore_ssh_host_keys=False, spooldir=None, resourcedir=None, **extra_args)

Execute an Application instance as a local process.

Construction of an instance of ShellcmdLrms takes the following optional parameters (in addition to any parameters taken by the base class LRMS):

Parameters:

• time_cmd (str) –

  Path to the GNU time command. Default is /usr/bin/time which is correct on all known Linux distributions.

  This backend uses many of the extended features of GNU time, so the shell-builtins or the BSD time will not work.

• spooldir (str) – Path to a filesystem location where to create temporary working directories for processes executed through this backend. The default value None means to use $TMPDIR or /tmp (see tempfile.mkftemp for details).
• resourcedir (str) – Path to a filesystem location where to create a temporary directory that will contain information on the jobs running on the machine. The default value None means to use $HOME/.gc3/shellcmd.d.
• transport (str) – Transport to use to connecet to the resource. Valid values are ssh or local.
• frontend (str) – If transport is ssh, then frontend is the hostname of the remote machine where the jobs will be executed.
• ignore_ssh_host_key (bool) – When connecting to a remote resource using ssh the server ssh public key is usually checked against a database of known hosts, and if the key is found but it does not match with the one saved in the database the connection will fail. Setting ignore_ssh_host_key to True will disable this check, thus introducing a potential security issue, but allowing connection even though the database contain old/invalid keys (the use case is when connecting to VM on a cloud, since the IP is usually reused and therefore the ssh key is recreated).
• override (bool) – ShellcmdLrms by default will try to gather information on the machine the resource is running on, including the number of cores and the available memory. These values may be different from the values stored in the configuration file. If override is True, then the values automatically discovered will be used instead of the ones in the configuration file. If override is False, instead, the values in the configuration file will be used.

cancel_job(app)

Cancel a running job. If app is associated to a queued or running remote job, tell the execution middleware to cancel it.

close()

Implement gracefully close on LRMS dependent resources e.g. transport

free(app)

Delete the temporary directory where a child process has run. The temporary directory is removed with all its content, recursively.

If the deletion is successful, the lrms_execdir attribute in app.execution is reset to None; subsequent invocations of this method on the same applications do nothing.

free_slots

Returns the number of cores free

get_resource_status()

Update the status of the resource associated with this LRMS instance in-place. Return updated Resource object.

get_results(app, download_dir, overwrite=False)

Retrieve job output files into local directory download_dir (which must already exists). Will not overwrite existing files, unless the optional argument overwrite is True.

peek(app, remote_filename, local_file, offset=0, size=None)

Download size bytes (at offset offset from the start) from remote file remote_filename and write them into local_file. If size is None (default), then snarf contents of remote file from offset unto the end.

Argument local_file is either a local path name (string), or a file-like object supporting a .write() method. If local_file is a path name, it is created if not existent, otherwise overwritten.

Argument remote_filename is the name of a file in the remote job “sandbox”.

Any exception raised by operations will be passed through.

submit_job(app)

Run an Application instance as a local process.

See :LRMS.submit_job

update_job_state(app)

Query the running status of the local process whose PID is stored into app.execution.lrms_jobid, and map the POSIX process status to GC3Libs Run.State.

validate_data(data_file_list=[])

Return False if any of the URLs in data_file_list cannot be handled by this backend.

The shellcmd backend can only handle file URLs.