API

Kliko contains various helper functions to validate Kliko files, parameter files based on a kliko definition, generate command line interfaces and django forms from Kliko definitions.

Validation

Kliko and parameter validation related functions.

kliko.validate.convert_to_parameters_schema(kliko)

Convert a kliko schema into a validator for the parameters generated with a kliko schema.

Parameters:kliko (str) – a kliko definition
Returns:A structure for a pykwalify validator
kliko.validate.validate(kliko_file=False, paramaters_file=False)

Validate the kliko and paramaters file and parse the parameters file. Should be run inside the Kliko container.

Parameters:
  • kliko_file (str) – Path to a kliko file
  • paramaters_file (str) – path to a parameters file
Returns:

The validated and parsed paramaters file
kliko.validate.validate_kliko(kliko, version=3)

validate a kliko yaml string

Parameters:kliko – a parsed kliko object
Returns:a (nested) kliko structure
Return type:dict
Raises:an exception if the string can’t be parsed or is not in the following the Kliko schema
kliko.validate.validate_opened(kliko, parameters)
kliko.validate.validate_parameters(parameters, kliko)

validate a set of parameters given a kliko definition

Parameters:
  • parameters (dict) – A structure that should follow the given kliko structure
  • kliko (dict) – A nested dict which defines the valid parameters in Kliko format
Returns:

the parsed parameters
Return type:

str
Raises:

an exception if the string can’t be parsed or is not in the defining valid parameters

Command line interface generation

Command line utilities for Kliko

kliko.cli.command_line_run(argv)
kliko.cli.directory_exists(path)

check if a directory exists

kliko.cli.file_exists(path)

check if a file exists

kliko.cli.first_parser(argv)

This is only used when script in invoked with 0 or 1 args (should be kliko image name).

kliko.cli.generate_kliko_cli_parser(kliko_data, parent_parser=None)

Generate a command line parser from a Kliko structure.

Parameters:kliko_data (dict) – A nested kliko structure
Returns:a configured argument parser
Return type:argparse.ArgumentParser
kliko.cli.second_parser(argv, kliko_data)

Used when kliko image is known, so we can extract the parameters.

Docker

Helper functions for using Kliko in combinaton with Docker

kliko.docker_util.extract_params(docker_client, image_name)
Parameters:
  • docker_client (docker.docker.Client) – a docker client object
  • image_name (str) – name of the image to use for kliko.yml extraction
Returns:

content of the param schema
Return type:

str

Luigi

class kliko.luigi_util.FileParameter(default=<object object>, is_global=False, significant=True, description=None, config_path=None, positional=True, always_in_help=False, batch_method=None)

Bases: luigi.parameter.Parameter

Parameter whose value is a path.

parse(s)
class kliko.luigi_util.KlikoTask(*args, **kwargs)

Bases: luigi.task.Task

A Luigi task that defines a Kliko task. Override the image_name method to specify which kliko container to run.

classmethod connect()
connection = None
get_instance_path(*arg)

Each Kliko image and set of paramaters has its own private folder.

classmethod get_kliko_data(*arg)
classmethod get_params()

We override this so we can populate the param list with the Kliko configuration.

classmethod image_id(*arg)
classmethod image_name()

override this and return the kliko container name you want to use as a string .

output()

We create a .finished file to mark a task finished in a Kliko work/output folder. In the case of joined IO the output folder is the same as the input folder. We add the kliko container ID to the finished file to not confuse luigi. Don’t forget to get the parent folder of the path returned by this output function.

run()
class kliko.luigi_util.Optional

Bases: str

A optional Luigiparameter.

Chaining

kliko.chaining.run_chain(steps, docker_client, kliko_dir=None)

Run a chain of kliko containers. The output of each container will be attached to the input of the successive container.

Parameters:
  • steps (list) – a list of tuples, first element of tuple container name, second parameters dict
  • docker_client (docker.Client) – a connection to the docker daemon
  • kliko_dir (str) – a path to a workfolder for storing intermediate kliko results

Django

Helper functions for using Kliko in combinaton with Django

kliko.django_form.generate_form(parsed)

Generate a django form from a parsed kliko object

Parameters:params – A parsed kliko file.
Returns:form_utils.forms.BetterForm