Documentation for pulsar 0.7.2. For development docs, go here.
Pulsar ships with an asynchronous TaskQueue built on top pulsar application framework. Task queues are used as a mechanism to distribute work across threads/processes or machines. Pulsar TaskQueue is highly customizable, it can run in multi-threading or multiprocessing (default) mode and can share backends.Task across several machines. By creating models.Job classes in a similar way you do for celery, this application gives you all you need for running them with very little setup effort:
from pulsar.apps import tasks if __name__ == '__main__': tasks.TaskQueue(tasks_path=['path.to.tasks.*']).start()
Check the task queue tutorial for a running example with simple tasks.
To get started, follow the these points:
task_paths = ['myjobs','another.moduledir.*']
The * at the end of the second module indicates to collect models.Job from all submodules of another.moduledir.
The schedule_periodic flag indicates if the TaskQueue can schedule models.PeriodicJob. Usually, only one running TaskQueue application is responsible for scheduling tasks while any the other, simply consume tasks.
It can be specified in the command line via the --schedule-periodic flag.
It can be specified in the command line via the --task-backend ... option.
The concurrent_tasks parameter control the maximum number of concurrent tasks for a given task worker. This parameter is important when tasks are asynchronous, that is when they perform some sort of I/O and the job callable returns and asynchronous component.
It can be specified in the command line via the --concurrent-tasks ... option.
A pulsar.apps.Application for consuming task.Tasks and managing scheduling of tasks via a scheduler.Scheduler.
The TaskBackend for this task queue. This picklable attribute is available once the TaskQueue has started (that is when the monitor_start() method is invoked by the pulsar.apps.ApplicationMonitor running it).
Starts running the task queue in monitor.
It calles the pulsar.Application.callable (if available) and create the backend.