- Task : Task class
- Task(function, arg0, arg1, arg2,...)
TaskGenerator : functional decorator
CachedFunction : load a function result from disk, running it if needed
- barrier : function
- if you place this anywhere in a jugfile, it assures that all tasks up to this point have completed before proceeding.
- class jug.Task(f, dep0, dep1, ..., kw_arg0=kw_val0, kw_arg1=kw_val1, ...)¶
Defines a task, which is roughly equivalent to:
f(dep0, dep1,..., kw_arg0=kw_val0, kw_arg1=kw_val1, ...)
can_load() Returns whether result is available. can_run() Returns true if all the dependencies have their results available. dependencies() for dep in task.dependencies(): hash() Returns the hash for this task. invalidate() Equivalent to t.store.remove(t.hash()). is_loaded() Returns True if the task is already loaded is_locked() Note that only calling lock() and checking the result atomically checks for the lock(). load() Loads the results from the storage backend. lock() Tries to lock the task for the current process. run([force, save]) Performs the task. unload() Unload results (can be useful for saving memory). unload_recursive() Equivalent to: unlock() Releases the lock. value()
Returns whether result is available.
Returns true if all the dependencies have their results available.
- for dep in task.dependencies():
Iterates over all the first-level dependencies of task t
self : Task
deps : generator
A generator over all of self‘s dependencies
- retrieve dependencies recursively
Returns the hash for this task.
The results are cached, so the first call can be much slower than subsequent calls.
Equivalent to t.store.remove(t.hash()). Useful for interactive use (i.e., in jug shell mode).
Returns True if the task is already loaded
Note that only calling lock() and checking the result atomically checks for the lock(). This function can be much faster, though, and, therefore is sometimes useful.
is_locked : boolean
Whether the task appears to be locked.
Loads the results from the storage backend.
This function always loads from the backend even if the task is already loaded. You can use is_loaded as a check if you want to avoid this behaviour.
Returns : Nothing :
Tries to lock the task for the current process.
Returns True if the lock was acquired. The correct usage pattern is:
locked = task.lock() if locked: task.run() else: # someone else is already running this task!
Not that using can_lock() can lead to race conditions. The above is the only fully correct method.
locked : boolean
Whether the lock was obtained.
- run(force=False, save=True)¶
Performs the task.
force : boolean, optional
if true, always run the task (even if it ran before) (default: False)
save : boolean, optional
if true, save the result to the store (default: True)
Unload results (can be useful for saving memory).
for tt in recursive_dependencies(t): tt.unload()
Releases the lock.
If the lock was not held, this may remove another thread’s lock!
- class jug.Tasklet(base, f)¶
A Tasklet is a light-weight Task.
It looks like a Task, behaves like a Task, but its results are not saved in the backend.
It is useful for very simple functions and is automatically generated on subscripting a Task object:
t = Task(f, 1) tlet = t
tlet will be a Tasklet
can_load() dependencies() value()
- class jug.TaskGenerator(f)¶
@TaskGenerator def f(arg0, arg1, ...)...
Turns f from a function into a task generator.
This means that calling f(arg0, arg1) results in: Task(f, arg0, arg1)
- class jug.iteratetask(base, n)¶
- for a in iteratetask(task, n):
This creates an iterator that over the sequence task, task, ..., task[n-1].
task : Task(let)
n : integer
Loads a task object recursively. This correcly handles lists, dictonaries and eny other type handled by the tasks themselves.
obj : object
Anything that can be pickled or a Task
value : object
The result of the task obj
- jug.CachedFunction(f, *args, **kwargs)¶
is equivalent to:
task = Task(f, *args, **kwargs) if not task.can_load(): task.run() value = task.value()
That is, it calls the function if the value is available, but caches the result for the future.
f : function
Any function except unnamed (lambda) functions
value : result
Result of calling f(*args,**kwargs)
- function An alternative way to achieve similar results to CachedFunction(f) using bvalue is:: ft = Task(f) fvalue = bvalue(ft) The alternative method is more flexible, but will only be execute lazily. In particular, a jug status will not see past the bvalue call until jug execute is called to execute f, while a CachedFunction object will always execute.
In a jug file, it assures that all tasks defined up to now have been completed. If not, parsing will (temporarily) stop at that point.
This ensures that, after calling barrier() you are free to call value() to get any needed results.
- function Restricted version of this function. Often faster