Client API Documentation

Contents

• Client API Documentation
  • General
  • Project
  • Folder
  • Job
  • Task
  • Layer
  • Output
  • Node
  • Proc
  • Cluster
  • Quota
  • Filtering / Matching / Actions
  • Dependencies
  • Exceptions

General

Functions

plow.client.get_plow_time()

Get the Plow server time in msec since the epoch

Returns:long - msec since epoch

plow.client.reconnect()

Re-establish the connection to the Plow server

plow.client.get_host()

Get the current host and port of the Plow server

Returns:(str host, int port)

plow.client.set_host(str host='localhost', int port=11336)

Set the host and port of the Plow server

Parameters:

• host – str = “localhost”
• port – int = 11336

plow.client.is_uuid(str identifier)

Test if a string is a valid UUID

Parameters:identifier (str) – string to test Returns:bool - True if valid UUID

Project

Classes

class plow.client.Project

A Project

Variables:

• id – str
• code – str
• title – str
• isActive – bool

get_folders(self)

Get the folders for this project

Returns:list[Folder]

get_job_board(self)

Get the folders for this project that have active jobs

Returns:list[Folder]

get_procs(self)

Get current procs

Returns:list[Proc]

refresh(self)

Refresh the attributes from the server

set_active(self, int active)

Set the active state of the Project

Parameters:active – bool

Functions

plow.client.get_project(Guid guid)

Get a Project by id

Parameters:guid – str - project id Returns:Project

plow.client.get_projects()

Get a list of all Projects

Returns:list[Project]

plow.client.get_project_by_code(string code)

Look up a Project by its code

Parameters:code – str Returns:Project

plow.client.get_active_projects()

Return a list of only active Projects

Returns:list[Project]

plow.client.create_project(string title, string code)

Create a new Project with a title and code

Parameters:

• title – str - A full project title
• code – str - A short code to indentify the project

Returns:

Project

plow.client.set_project_active(Project project, int active)

Set a project to be active

Parameters:

• project – Project
• active – bool

plow.client.get_job_board(Project project)

Get a list of folders for the given project that have active jobs.

Parameters:project – Project Returns:list[Folder]

Folder

Classes

class plow.client.Folder

Folder()

A Folder

Variables:

• id – str
• name – str
• minCores – int
• maxCores – int
• runCores – int
• runProcs – int
• order – int
• totals – TaskTotals
• jobs – list[Job]

delete(self)

Delete this Folder

get_procs(self)

Get current procs

Returns:list[Proc]

refresh(self)

Refresh the attributes from the server

set_max_cores(self, int value)

Set the maximum cores

Parameters:value – int - cores

set_min_cores(self, int value)

Set the minimum cores

Parameters:value – int - cores

Functions

plow.client.get_folder(Guid folderId)

Get a Folder by id

Parameters:folderId – str Returns:Folder

plow.client.get_folders(Project project)

Get a list of Folders by project id

Parameters:project – Project Returns:list[Folder]

plow.client.create_folder(Project project, string name)

Create a folder

Parameters:

• project – Project
• name – str - folder name

Returns:

Folder

plow.client.set_folder_min_cores(Folder folder, int value)

Set the minimum cores for a Folder

Parameters:

• folder – Folder
• value – int - cores

plow.client.set_folder_max_cores(Folder folder, int value)

Set the maximum cores for a Folder

Parameters:

• folder – Folder
• value – int - cores

plow.client.set_folder_name(Folder folder, string name)

Set the name for a Folder

Parameters:

• folder – Folder
• name – str

plow.client.delete_folder(Folder folder)

Delete a folder

Parameters:folder – Folder

Job

Constants

plow.client.JobState

Constants representing the state of a Job

plow.client.JobState.INITIALIZE

plow.client.JobState.RUNNING

plow.client.JobState.FINISHED

Classes

class plow.client.Job

Job()

A Job

Variables:

• id – str
• folderId – str
• name – str - name of the job
• username – str - name of job owner
• uid – int - uid of job owner
• state – JobState
• paused – bool
• minCores – int
• maxCores – int
• runCores – int
• runProcs – int
• startTime – long - msec since epoch
• stopTime – long - msec since epoch
• totals – TaskTotals
• stats – JobStats
• attrs – dict

eat_dead_tasks(self, callback=None)

Eat all dead tasks on a job

Parameters:callback – Optional callback function to run if tasks were eaten

get_depends(self)

Get a list of depends that others have on this job

Returns:list[Depend]

get_depends_on(self)

Get a list of depends this job has on others

Returns:list[Depend]

get_folder(self)

Get the folder that contains this Job

Returns:Folder

get_layers(self)

Get the layers for this job

Returns:list[Layer]

get_outputs(self)

Get a list of outputs

Returns:list[plowOutput]

get_procs(self)

Get current procs

Returns:list[Proc]

get_tasks(self, list states=None) → list

Get a list of tasks for this job, optionally filtered by a list of task stats

Parameters:stats – list[TaskState] = None Returns:list[Task]

kill(self, string reason)

Kill the job

Parameters:reason – str - reason for killing

kill_tasks(self, callback=None)

Kill all tasks on a job

Parameters:callback – Optional callback function to run if tasks were killed

pause(self, int paused)

Set the pause state of the job

Parameters:paused – bool

refresh(self)

Refresh the attributes from the server

retry_dead_tasks(self, callback=None)

Retry all dead tasks on a job

Parameters:callback – Optional callback function to run if tasks were retried

set_max_cores(self, int value)

Set the maximum cores the job should use

Parameters:value – int

set_min_cores(self, int value)

Set the minimum cores the job should use

Parameters:value – int

class plow.client.JobSpec

JobSpec(**kwargs)

A JobSpec specifies the parameters for launching a job

Variables:

• name – str
• project – str`.Project` code
• username – str - owner
• logPath – str - path to logfile
• paused – bool ; if True, submit in a paused state
• uid – int - uid of owner
• layers – list[LayerSpec]
• depends – list[DependSpec]
• attrs – dict
• env – dict

launch(self)

Launch this spec and return the Job

Returns:Job

logPath

logPath: ‘string’

name

name: ‘string’

paused

paused: ‘int’

project

project: ‘string’

uid

uid: ‘int’

username

username: ‘string’

class plow.client.JobStats

Data structure representing stats for a Job

Variables:

• highRam – int
• highCores – float
• highCoreTime – long
• totalCoreTime – long
• totalSuccessCoreTime – long
• totalFailCoreTime – long
• highClockTime – long

Functions

plow.client.launch_job(JobSpec spec)

Launch a job with a JobSpec.

Parameters:spec – JobSpec Returns:Job

plow.client.get_active_job(string name)

Get an active Job

Parameters:id – str name Returns:Job

plow.client.get_job(Guid id)

Get a Job

Parameters:id – str Job id Returns:Job

plow.client.kill_job(Job job, string reason)

Kill a job

Parameters:

• job – Job
• reason – str reason for killing the job

plow.client.pause_job(Job job, int paused)

Set the pause state of a job

Parameters:

• job – Job
• paused – bool pause state

plow.client.get_jobs(**kwargs)

Get a list of jobs matching a criteria.

Parameters:

• matchingOnly – bool
• regex – str regex pattern to match against job names
• project – list[str] of matching project codes
• user – list[str] of matching user names
• jobIds – list[str] of matching job ids
• name – list[str] of matching job names
• states – list[JobState]

Returns:

list[Job]

plow.client.get_job_outputs(job)

Get the outputs of all layers of a Job

Parameters:job – Job or str id Returns:list[Output]

plow.client.set_job_min_cores(Job job, int value)

Set the minimum number of cores a job should get

Parameters:

• job – Job
• value – int number of cores

plow.client.set_job_max_cores(Job job, int value)

Set the maximum number of cores a job should get

Parameters:

• job – Job
• value – int number of cores

plow.client.get_job_spec(Guid jobId)

Get a JobSpec instance from an existing job id Guid

Parameters:jobId – str Returns:JobSpec

plow.client.get_job_outputs(job)

Get the outputs of all layers of a Job

Parameters:job – Job or str id Returns:list[Output]

Task

Constants

plow.client.TaskState

Constants representing the state of a Task

plow.client.TaskState.INITIALIZE

plow.client.TaskState.WAITING

plow.client.TaskState.RUNNING

plow.client.TaskState.DEAD

plow.client.TaskState.EATEN

plow.client.TaskState.DEPEND

plow.client.TaskState.SUCCEEDED

Classes

class plow.client.Task

Represents an existing Task

Variables:

• id –
• name – str
• layerId – str
• jobId – str
• number – int
• order – int
• retries – int
• minCores – int
• minRam – int
• state – TaskState
• stats – TaskStats

eat(self)

Eats the task. This is different than a kill, indicating that the work should simply be “completed”

get_depends(self)

Get a list of depends that others have on this task

Returns:list[Depend]

get_depends_on(self)

Get a list of depends this task has on others

Returns:list[Depend]

get_job(self)

Return the Job object for this task

Returns:Job

get_layer(self)

Return the Layer object for this task

Returns:Layer

get_log_path(self, retryNum=-1)

Get a log path for the task

Optionally you may request a log from a specific retry of the task. The default is 0, which is the first run of a task. Subsequent retries will produce new log files.

If the task never started, then a log will not have been created yet, and this function will return an invalid path using a -1 as the retry number.

Parameters:retryNum – int task.retries number (default -1 == latest) Returns:str log path

get_procs(self)

Get current procs

Returns:list[Proc]

kill(self)

Kill the task

refresh(self)

Refresh the attributes from the server

retry(self)

Retry the task

class plow.client.TaskSpec

TaskSpec(**kwargs)

Defines a task specification

Variables:

• name – string name
• depends – list[DependSpec]

name

name: ‘string’

class plow.client.TaskStats

Data structure representing stats for a Task

Variables:

• cores – int
• usedCores – float
• highCores – float
• ram – int
• usedRam – int
• highRam – int
• startTime – long
• stopTime – long
• retryNum – int
• progress – int
• lastLogLine – str
• active – bool
• exitStatus – int
• exitSignal – int
• lastNode – str

class plow.client.TaskTotals

Data structure representing counts for a task

Variables:

• total – int
• succeeded – int
• running – int
• dead – int
• eaten – int
• waiting – int
• depend – int

Functions

plow.client.get_task_stats(Guid taskId)

Get a list of task stats from an existing task id

Parameters:taskId – str Returns:list[TaskStats]

plow.client.get_task(Guid taskId)

Get a task by id

Parameters:taskId – str Returns:Task

plow.client.get_tasks(**kwargs)

Get a list of tasks by providing various filtering parameters.

Parameters:

• job – Job
• layers – list of Layer
• nodes – list of Node
• taskIds – list of str Task id’s
• limit – int
• offset – int
• states – list[TaskState]
• lastUpdateTime – long msec epoch timestamp

Returns:

list[Task]

plow.client.get_task_log_path(Task task, int retryNum=-1)

Get a log path by task

Optionally you may request a log from a specific retry of the task. The default is 0, which is the first run of a task. Subsequent retries will produce new log files.

If the task never started, then a log will not have been created yet, and this function will return an invalid path using a -1 as the retry number.

Parameters:

• task – Task
• retryNum – int Task.retries number (default -1 == latest)

Returns:

str log path

plow.client.retry_tasks(**kwargs)

Retry tasks matching various keyword filter params

Parameters:

• job – Job
• layers – list of Layer
• nodes – list of Node
• taskIds – list of str Task id’s
• limit – int
• offset – int
• states – list[TaskState]
• lastUpdateTime – long msec epoch timestamp

plow.client.eat_tasks(**kwargs)

Eat tasks matching various keyword filter params

Parameters:

• job – Job
• layers – list of Layer
• nodes – list of Node
• taskIds – list of str Task id’s
• limit – int
• offset – int
• states – list[TaskState]
• lastUpdateTime – long msec epoch timestamp

plow.client.kill_tasks(**kwargs)

Kill tasks matching various filter params

Parameters:

• job – Job
• layers – list of Layer
• nodes – list of Node
• taskIds – list of str Task id’s
• limit – int
• offset – int
• states – list[TaskState]
• lastUpdateTime – long msec epoch timestamp

Layer

Classes

class plow.client.Layer

Layer()

Represents an existing layer

Variables:

• id – str
• jobId – str
• name – str
• service – str
• range – str
• chunk – int
• minCores – int
• maxCores – int
• runCores – int
• minRam – int
• maxRam – int
• maxRetries – int
• threadable – bool
• totals – list[TaskTotals]
• tags – list(str)
• stats – LayerStats

add_output(self, string path, dict attrs)

Add an output to the layer

Parameters:

• path – str
• attrs – dict

Returns:

Output

get_depends(self)

Get a list of depends that others have on this layer

Returns:list[Depend]

get_depends_on(self)

Get a list of depends this layer has on others

Returns:list[Depend]

get_job(self)

Get the Job for this layer

Returns:list[Job]

get_outputs(self)

Get the outputs for this layer

Returns:list[Output]

get_procs(self)

Get current procs

Returns:list[Proc]

get_tasks(self)

Get the tasks for this layer

Returns:list[Task]

refresh(self)

Refresh the attributes from the server

set_max_cores_per_task(self, int maxCores)

Parameters:maxCores – int

set_min_cores_per_task(self, int minCores)

Parameters:minCores – int

set_min_ram_per_task(self, int minRam)

Parameters:minRam – int

set_tags(self, vector[string] tags)

Set the tags for the layer

Parameters:tags – list(str)

set_threadable(self, int threadable)

Parameters:threadable – bool

class plow.client.LayerSpec

LayerSpec(**kwargs)

Defines a new layer

Variables:

• name – str
• range – str
• service – str
• chunk – int
• minCores – int
• maxCores – int
• minRam – int
• maxRam – int
• maxRetries – int
• threadable – bool
• command – list[str]
• depends – list[DependSpec]
• tasks – list [TaskSpec]
• tags – list(str)
• env – dict
• isPost – bool

class plow.client.LayerStats

Data structure representing stats for a Layer

Variables:

• highRam – int
• avgRam – int
• stdDevRam – float
• highCores – float
• avgCores – float
• stdDevCores – float
• highCoreTime – long
• avgCoreTime – long
• lowCoreTime – long
• stdDevCoreTime – float
• totalCoreTime – long
• totalSuccessCoreTime – long
• totalFailCoreTime – long
• highClockTime – long
• avgClockTime – long
• lowClockTime – long
• stdDevClockTime – float
• totalClockTime – long
• totalSuccessClockTime – long
• totalFailClockTime – long

class plow.client.Service

Data structure representing an existing Layer Service

Variables:

• id – str
• name – str
• tags – list[str]
• minCores – int
• maxCores – int
• minRam – int
• maxRam – int
• maxRetries – int
• threadable – bool

create(self)

Create this service, if it is a new object. Does nothing if the object already exists (already has an id)

delete(self)

Deletes this instance, if it was created (has an id)

update(self)

Commit any changes to this existing service

Functions

plow.client.get_services()

Get a list of existing Service instances

Returns:list[Service]

plow.client.create_service(Service src)

Create a new service. Updates the original service object passed in, and returns it.

Parameters:src – Service Returns:Service

plow.client.delete_service(Service src)

Delete an existing Service instance

Parameters:src – Service

plow.client.update_service(Service src)

Update an existing Service to reflect changes made to the object

Parameters:src – Service

plow.client.get_layer_by_id(Guid layerId)

Get a layer by its id

Parameters:layerId – str Returns:Layer

plow.client.get_layer(job, string name)

Get layer by its name

Parameters:

• job – Job or str job id
• name – str

Returns:

Layer

plow.client.get_layers(job)

Get layers by a job or job id

Parameters:job – Job or str job id Returns:list[Layer]

plow.client.add_layer_output(Layer layer, string path, dict attrs)

A an output to a layer

Parameters:

• layer – Layer
• path – str
• attrs – dict

Returns:

Output

plow.client.get_layer_outputs(layer)

Get the outputs for a layer

Parameters:layer – Layer or str id Returns:list[Layer]

plow.client.set_layer_tags(Layer layer, vector[string] tags)

Parameters:

• layer – Layer
• tags – list(str)

plow.client.set_layer_min_cores_per_task(Layer layer, int minCores)

Parameters:

• layer – Layer
• minCores – int

plow.client.set_layer_max_cores_per_task(Layer layer, int maxCores)

Parameters:

• layer – Layer
• maxCores – int

plow.client.set_layer_min_ram_per_task(Layer layer, int minRam)

Parameters:

• layer – Layer
• minRam – int

plow.client.set_layer_threadable(Layer layer, int threadable)

Parameters:

• layer – Layer
• threadable – bool

Output

Classes

class plow.client.Output

Output()

Represents an output of a Layer

Variables:

• outputId – str
• path – str path
• attrs – dict attributes

outputId

outputId: ‘string’

path

path: ‘string’

update(self)

Commit any changes to the attrs dict

Functions

plow.client.update_output_attrs(Guid outputId, dict attrs)

Update the dict attributes of an existing Output

Parameters:

• outputId – str Output id
• attrs – dict

plow.client.set_output_attrs(Guid outputId, dict attrs)

Replace the dict attributes of an existing Output

Parameters:

• outputId – str Output id
• attrs – dict

plow.client.get_output_attrs(Guid outputId)

Get the dict attributes of an existing Output

Parameters:outputId – str Output id Returns:dict

Node

Constants

plow.client.NodeState

Constants representing the state of a Node

plow.client.NodeState.UP

plow.client.NodeState.DOWN

plow.client.NodeState.REPAIR

plow.client.SlotMode

Constants representing the slot mode of a Node

plow.client.NodeState.DYNAMIC

plow.client.NodeState.SLOTS

plow.client.NodeState.SINGLE

Classes

class plow.client.Node

Node()

Represents an existing Node (system) that can perform tasks

Variables:

• id – str
• name – str
• clusterId – str
• clusterName – str
• ipaddr – str
• locked – bool
• createdTime – long msec epoch timestamp
• updatedTime – long msec epoch timestamp
• bootTime – long msec epoch timestamp
• totalCores – int
• idleCores – int
• slotCores – int
• totalRamMb – int
• freeRamMb – int
• slotRam – int
• tags – set(str)
• state – NodeState
• system – NodeSystem
• mode – SlotMode

get_cluster(self)

Get the cluster object to which this node is assigned

Returns:Cluster

get_procs(self)

Get current procs

Returns:list[Proc]

lock(self, int locked)

Set the lock state of the node

Parameters:locked – bool

refresh(self)

Refresh the attributes from the server

set_cluster(self, Cluster cluster)

Assign the node to a cluster

Parameters:cluster – str Cluster

set_slot_mode(self, int mode, int cores, int ram)

Set the slot mode for a node

Parameters:

• mode – SlotMode
• cores – int number of cores
• ram – int ram in MB

set_tags(self, set[string] tags)

Set the tags for the node

Parameters:tags – set(str)

class plow.client.NodeSystem

Defines properties of the hardware specs of a Node at a sampled moment in time

Variables:

• cpuModel – str
• platform – str
• physicalCores – int
• logicalCores – int
• totalRamMb – int
• freeRamMb – int
• totalSwapMb – int
• freeSwapMb – int
• load – list[int]

cpuModel

cpuModel: ‘string’

freeRamMb

freeRamMb: ‘int’

freeSwapMb

freeSwapMb: ‘int’

logicalCores

logicalCores: ‘int’

physicalCores

physicalCores: ‘int’

platform

platform: ‘string’

totalRamMb

totalRamMb: ‘int’

totalSwapMb

totalSwapMb: ‘int’

Functions

plow.client.get_node(string name)

Get a node by name

Parameters:name – str Returns:Node

plow.client.get_nodes(**kwargs)

Get nodes matching keyword filter parameters

Parameters:

• hostIds – list[str Node id]
• cluster – list[str Cluster]
• regex – str
• hostnames – list[str]
• locked – bool

Returns:

list[Node]

plow.client.set_node_locked(Node node, int locked)

Set the lock state of the node

Parameters:

• node – Node
• locked – bool

plow.client.set_node_cluster(Node node, Cluster cluster)

Assign the node to a cluster

Parameters:

• node – Node
• cluster – Cluster

plow.client.set_node_tags(Node node, set[string] tags)

Set the tags for the node

Parameters:

• node – Node
• tags – set(str)

Proc

Classes

class plow.client.Proc

Represents an existing Proc on a Node

Variables:

• id – str
• nodeId – str
• jobName – str
• layerName – str
• taskName – str
• cores – int
• usedCores – float
• highCores – float
• ram – int
• usedRam – int
• highRam – int
• ioStats – list[long]
• createdTime – long msec epoch
• updatedTime – long msec epoch
• startedTime – long msec epoch

Functions

plow.client.get_proc(Guid id)

Get a proc by id

Parameters:id – str Proc id Returns:Proc

plow.client.get_procs(**kwargs)

Get a list of procs matching a criteria.

Parameters:

• projectIds – list[str Project ids]
• folderIds – list[str Folder ids]
• jobIds – list[str Job ids]
• layerIds – list[str Layer ids]
• taskIds – list[str Task ids]
• clusterIds – list[str Cluster ids]
• quotaIds – list[str Quota ids]
• nodeIds – list[str Node ids]
• lastUpdateTime – long msec epoch
• limit – int
• offset – int

Returns:

list[Proc]

Cluster

Classes

class plow.client.Cluster

Cluster()

A Cluster

Variables:

• id – str
• name – str
• isLocked – bool
• isDefault – bool
• tags – set[str]
• total – plow.ClusterCounts

delete(self)

Delete the cluster

Returns:bool - was deleted

get_procs(self)

Get current procs

Returns:list[Proc]

lock(self, int locked)

Lock the cluster

Parameters:locked – bool - True to lock / False to unlock Returns:bool - locked

refresh(self)

Refresh the attributes from the server

set_default(self)

Set this cluster to be the default cluster

set_name(self, string name)

Set the name of the cluster

Parameters:name – str - name

set_tags(self, set[string] tags)

Set the tags for the cluster

Parameters:tags – set - a set of string tags

class plow.client.ClusterCounts

Contains various status counts about a plow.Cluster

Variables:

• nodes – int
• upNodes – int
• downNodes – int
• repairNodes – int
• lockedNodes – int
• unlockedNodes – int
• cores – int
• upCores – int
• downCores – int
• repairCores – int
• lockedCores – int
• unlockedCores – int
• runCores – int
• idleCores – int

cores

cores: ‘int’

downCores

downCores: ‘int’

downNodes

downNodes: ‘int’

idleCores

idleCores: ‘int’

lockedCores

lockedCores: ‘int’

lockedNodes

lockedNodes: ‘int’

nodes

nodes: ‘int’

repairCores

repairCores: ‘int’

repairNodes

repairNodes: ‘int’

runCores

runCores: ‘int’

unlockedCores

unlockedCores: ‘int’

unlockedNodes

unlockedNodes: ‘int’

upCores

upCores: ‘int’

upNodes

upNodes: ‘int’

Functions

plow.client.get_cluster(string name)

Return a Cluster by name

Parameters:name – str - name of a cluster Returns:plow.Cluster

plow.client.get_clusters()

Return a list of all Clusters

Returns:list of Clusters

plow.client.get_clusters_by_tag(string tag)

Return a list of Clusters matching a tag

Parameters:tag – str - tag Returns:list of Clusters

plow.client.create_cluster(str name)

Create a Cluster with a name and set of tags

Parameters:name – str - Name of Cluster Returns:Cluster

plow.client.delete_cluster(Cluster cluster)

Delete a Cluster

Parameters:cluster – Cluster Returns:bool - True if deleted

plow.client.lock_cluster(Cluster cluster, int locked)

Lock a Cluster

Parameters:

• cluster – Cluster
• locked – bool - True to lock / False to unlock

Returns:

bool - locked

plow.client.set_cluster_tags(Cluster cluster, set[string] tags)

Set the tags for a Cluster

Parameters:

• cluster – Cluster
• tags – set - A set of tags for the Cluster

plow.client.set_cluster_name(Cluster cluster, string name)

Set a name for a Cluster

Parameters:

• cluster – Cluster
• name – str - Cluster name

plow.client.set_default_cluster(Cluster cluster)

Set a given Cluster to be the default Cluster

Parameters:cluster – Cluster Returns:bool - True if deleted

Quota

Classes

class plow.client.Quota

Represents an existing Quota object, set on a project and cluster.

Variables:

• id – str
• clusterId – str Cluster id
• projectId – str Project id
• name – str
• isLocked – bool
• size – int
• burst – int
• runCores – int

get_procs(self)

Get current procs

Returns:list[Proc]

refresh(self)

Refresh the attributes from the server

set_burst(self, int burst)

Parameters:burst – int

set_locked(self, int locked)

Set the lock state

Parameters:locked – bool

set_size(self, int size)

Parameters:size – int

Functions

plow.client.get_quota(Guid id)

Get a quota by id

Parameters:id – Quota id Returns:Quota

plow.client.get_quotas(**kwargs)

Get quotas matching various keyword filter params

Parameters:

• project – list[Project]
• cluster – list[Cluster]

Returns:

list[Quota]

plow.client.create_quota(Project project, Cluster cluster, int size, int burst)

Create a quota for a project and cluster

Parameters:

• project – The target Project
• cluster – The target Cluster
• size – int
• burst – int

Returns:

Quota

plow.client.set_quota_size(Cluster cluster, int size)

Set the quota size

Parameters:

• cluster – Quota
• size – int

plow.client.set_quota_burst(Cluster cluster, int burst)

Set the quota burst

Parameters:

• cluster – Quota
• burst – int

plow.client.set_quota_locked(Cluster cluster, int locked)

Set the lock state of the quota

Parameters:

• cluster – Quota
• locked – bool

Filtering / Matching / Actions

Constants

plow.client.MatcherType

Constants definging match types

plow.client.MatcherType.CONTAINS

plow.client.MatcherType.NOT_CONTAINS

plow.client.MatcherType.IS

plow.client.MatcherType.IS_NOT

plow.client.MatcherType.BEGINS_WITH

plow.client.MatcherType.ENDS_WITH

plow.client.MatcherField

Constants field types to match against

plow.client.NodeState.JOB_NAME

plow.client.NodeState.USER

plow.client.NodeState.ATTR

plow.client.ActionType

Constants representing actions to trigger

plow.client.ActionType.SET_FOLDER

plow.client.ActionType.SET_MIN_CORES

plow.client.ActionType.SET_MAX_CORES

plow.client.ActionType.PAUSE

plow.client.ActionType.STOP_PROCESSING

Classes

class plow.client.Matcher

A matcher is a combination of a type a field and a value

Variables:

• id – str
• type – MatcherType
• field – MatcherField
• value – str

delete(self)

Delete the matcher

refresh(self)

Refresh the attributes from the server

class plow.client.Action

An Action is represents a type and a value

Variables:

• id – str
• type – ActionType
• value – str

delete(self)

Delete the action

refresh(self)

Refresh the attributes from the server

class plow.client.Filter

Filter(**kwargs)

A filter combines matcher and actions instances

Variables:

• id – str
• name – str
• order – int
• enabled – bool
• matchers – list[Matcher]
• actions – list[Action]

decrease_filter_order(self)

Decrease the order

delete(self)

Delete this filter

increase_order(self)

Increase the order

refresh(self)

Refresh the attributes from the server

set_name(self, string name)

Set the filter name

Parameters:name – str

set_order(self, int order)

Set the order

Parameters:order – int

Functions

plow.client.create_field_matcher(Filter filter, int field, int typ, string value)

Create a field Matcher

Parameters:

• filter – Filter
• field – MatcherField value
• typ – MatcherType value
• value – str value for the matcher

Returns:

Matcher

plow.client.create_attr_matcher(Filter filter, int typ, string attr, string value)

Create an attribute Matcher

Parameters:

• filter – Filter
• typ – MatcherType value
• attr – str attr for the matcher
• value – str value for the matcher

Returns:

Matcher

plow.client.get_matcher(Guid matcherId)

Get a matcher by id

Parameters:id – str Filter id Returns:Matcher

plow.client.get_matchers(Filter filter)

Get a list of Matchers by a filter

Parameters:filter – Filter Returns:list[Matcher]

plow.client.delete_matcher(Matcher matcher)

Delete a Matcher

Parameters:matcher – Matcher

plow.client.create_action(Filter filter, int typ, string value)

Create an action

Parameters:

• filter – Filter
• typ – ActionType value
• value – str

Returns:

Action

plow.client.get_action(Guid actionId)

Get an action by id

Parameters:actionId – str Action id Returns:Action

plow.client.get_actions(Filter filter)

Get a list of actions from a filter

Parameters:filter – Filter Returns:list[Action]

plow.client.delete_action(Action action)

Delete an action

Parameters:action – Action

plow.client.create_filter(Project project, string name)

Create a filter for a project

Parameters:

• project – Project
• name – str

Returns:

Filter

plow.client.get_filters(Project project)

Get a list of filters for a project

Parameters:project – Project Returns:list[Filter]

plow.client.get_filter(Guid filterId)

Get a filter by id

Parameters:filterId – str Filter.id Returns:Filter

plow.client.delete_filter(Filter filt)

Delete a filter

Parameters:filt – Filter

plow.client.set_filter_name(Filter filt, string name)

Set a filter name

Parameters:

• filt – Filter
• name – str

plow.client.set_filter_order(Filter filt, int order)

Set the filter order

Parameters:

• filt – Filter
• order – int

plow.client.increase_filter_order(Filter filt)

Increase the filter order

Parameters:filt – Filter

plow.client.decrease_filter_order(Filter filt)

Decrease the filter order

Parameters:filt – Filter

Dependencies

Constants

plow.client.DependType

Constants representing dependencies between types

plow.client.DependType.JOB_ON_JOB

plow.client.DependType.LAYER_ON_LAYER

plow.client.DependType.LAYER_ON_TASK

plow.client.DependType.TASK_ON_LAYER

plow.client.DependType.TASK_ON_TASK

plow.client.DependType.TASK_BY_TASK

Classes

class plow.client.Depend

Depend

Represents an existing dependency between two types

Variables:

• id – str
• type – DependType
• active – bool
• createdTime – long
• satisfiedTime – long
• dependentJobId – Guid
• dependOnJobId – Guid
• dependentLayerId – Guid
• dependOnLayerId – Guid
• dependentTaskId – Guid
• dependOnTaskId – Guid
• dependentJobName – str
• dependOnJobName – str
• dependentLayerName – str
• dependOnLayerName – str
• dependentTaskName – str
• dependOnTaskName – str

activate(self)

Activate the dependency

drop(self)

Drop the dependency

class plow.client.DependSpec

DependSpec(int type=0, **kwargs)

DependSpec

Specify the dependency between two types

Variables:

• type – DependType
• dependentJob – str id or Job
• dependOnJob – str id or Job
• dependentLayer – str id or Layer
• dependOnLayer – str id or Layer
• dependentTask – str id or Task
• dependOnTask – str id or Task

create(self)

Create the dependency from the current settings Return the newly created Depend instance.

Ensure that a valid DependType is set, as well as the corresponding dependent and dependsOn id.

Returns:Depend

Functions

plow.client.create_job_on_job_depend(Job job, Job onJob)

Make one job dependent on another This call is async and does not return anything.

Parameters:

• job – the Job which depends on another
• onJob – the Job which must finish first

plow.client.create_layer_on_layer_depend(Layer layer, Layer onLayer)

Make one layer dependent on another This call is async and does not return anything.

Parameters:

• layer – the Layer which depends on another
• onLayer – the Layer which must finish first

plow.client.create_layer_on_task_depend(Layer layer, Task onTask)

Make one layer dependent on another task This call is async and does not return anything.

Parameters:

• layer – the Layer which depends on a task
• onTask – the Task which must finish first

plow.client.create_task_by_task_depend(Layer layer, Layer onLayer)

Make each task of a layer dependent on the corresponding task of another layer, one by one. This call is async and does not return anything.

Parameters:

• layer – the Layer which depends on another
• onLayer – the Layer which has tasks that must finish first

plow.client.create_task_on_layer_depend(Task task, Layer onLayer)

Make one task dependent on another layer This call is async and does not return anything.

Parameters:

• task – the Task which depends on a layer
• onLayer – the Layer which must finish first

plow.client.create_task_on_task_depend(Task task, Task onTask)

Make one task dependent on another This call is async and does not return anything.

Parameters:

• task – the Task which depends on another
• onTask – the Task which must finish first

plow.client.get_depends_on_job(Job job)

Get a list of depends that others have on this job

Parameters:job – Job Returns:list[Depend]

plow.client.get_job_depends_on(Job job)

Get a list of depends that this job has on others

Parameters:job – Job Returns:list[Depend]

plow.client.get_depends_on_layer(Layer layer)

Get a list of depends that others have on this layer

Parameters:layer – Layer Returns:list[Depend]

plow.client.get_layer_depends_on(Layer layer)

Get a list of depends that this layer has on others

Parameters:layer – Layer Returns:list[Depend]

plow.client.get_depends_on_task(Task task)

Get a list of depends that others have on this task

Parameters:task – Task Returns:list[Depend]

plow.client.get_task_depends_on(Task task)

Get a list of depends that this task has on others

Parameters:task – Task Returns:list[Depend]

plow.client.drop_depend(Depend dep)

Drop the depends

Parameters:dep – Depend

plow.client.activate_depend(Depend dep)

Activate the depends

Parameters:dep – Depend

Exceptions

exception plow.client.PlowError

exception plow.client.PlowConnectionError