Documentation for pulsar 0.7.2. For development docs, go here.
To define a job is simple, subclass from Job and implement the job callable method:
from pulsar.apps import tasks class Addition(tasks.Job): def __call__(self, consumer, a, b): "Add two numbers" return a+b
The consumer, instance of pulsar.apps.tasks.backends.TaskConsumer, is passed by the Task backend and should always be the first positional parameter in the callable method. The remaining (optional) positional and key-valued parameters are needed by your job implementation.
class Crawler(tasks.Job): def __call__(self, consumer, sample, size=10): response = yield http.request(...) content = response.content ...
This allows for cooperative task execution on each task thread workers.
However, when setting the Job.can_overlap attribute to False, a new task cannot be started unless a previous task of the same job is done.
The Job class which is used in a distributed task queue.
The unique name which defines the Job and which can be used to retrieve it from the job registry. This attribute is set to the Job class name in lower case by default, unless a name class attribute is defined.
Type of Job, one of regular and periodic.
An instance of a datetime.timedelta or None. If set, it represents the time lag after which a task which did not start expires.
Boolean indicating if this job can generate overlapping tasks. It can also be a callable which accept the same input parameters as the job callable function.
The doc string syntax.
an instance of a logger. Created at runtime.
Run a new task in the task queue.
This utility method can be used from within the job callable method and it allows tasks to act as tasks factories.
a pulsar.Deferred called back with the task id of the new job.
This method invokes the pulsar.apps.tasks.backends.TaskBackend.run_job() method with the additional from_task argument equal to the id of the task invoking the method.
Generate a task unique identifiers.
a two-elements tuple containing the unique id and an identifier for overlapping tasks if the can_overlap results in False.
Called by the TaskBackend when creating a new task.
A periodic Job implementation.
If specified it must be a datetime.datetime instance. It controls when the periodic Job is run.
Periodicity as a datetime.timedelta instance.
Returns tuple of two items (is_due, next_time_to_run), where next time to run is in seconds. e.g.
time to run is in 20 seconds.
(False, 12), means the Job should be run in 12 seconds.
You can override this to decide the interval at runtime.
Site registry for tasks.
A generator of all regular jobs.
A generator of all periodic jobs.
Register a job in the job registry.
The task will be automatically instantiated if not already an instance.
Return a generator of all tasks of a specific type.