fireworks.queue package


fireworks.queue.queue_adapter module

class fireworks.queue.queue_adapter.Command(command)

Bases: object

Helper class - run subprocess commands in a different thread with TIMEOUT option. From Based on jcollado’s solution:


initialize the object.

command: command to run
command = None
error = u''
output = u''
process = None
run(timeout=None, **kwargs)

Run the command.

timeout (float): timeout kwargs (dict)
(status, output, error)
status = None
class fireworks.queue.queue_adapter.QScriptTemplate(template)

Bases: string.Template

delimiter = u'$$'
pattern = <_sre.SRE_Pattern object at 0x1093035d0>
class fireworks.queue.queue_adapter.QueueAdapterBase

Bases: collections.defaultdict, fireworks.utilities.fw_serializers.FWSerializable

The QueueAdapter is responsible for all interactions with a specific queue management system. This includes handling all details of queue script format as well as queue submission and


A user should extend this class with implementations that work on specific queue systems. Examples and implementations are in: fireworks/user_objects/queue_adapters.

Documentation on implementing queue adapters can be found on FireWorks home page,

defaults = {}
classmethod from_dict(m_dict)

Returns the number of jobs currently in the queue for the user.

username (str): the username of the jobs to count (default is to autodetect)
(int) number of jobs in the queue

returns a (multi-line) String representing the queue script, e.g. PBS script. Uses the template_file along with internal parameters to create the script.

launch_dir (str): The directory the job will be launched in
(str) the queue script
q_name = u'OVERRIDE_ME'
submit_cmd = u'OVERRIDE_ME'

Submits the job to the queue and returns the job id.

script_file: (str) name of the script file to use (String)
(int) job_id
template_file = u'OVERRIDE_ME'
to_dict(*args, **kwargs)

fireworks.queue.queue_launcher module

fireworks.queue.queue_launcher.launch_rocket_to_queue(launchpad, fworker, qadapter, launcher_dir=u'.', reserve=False, strm_lvl=u'INFO', create_launcher_dir=False, fill_mode=False)

Submit a single job to the queue.


launchpad (LaunchPad) fworker (FWorker) qadapter (QueueAdapterBase) launcher_dir (str): The directory where to submit the job reserve (bool): Whether to queue in reservation mode strm_lvl (str): level at which to stream log messages create_launcher_dir (bool): Whether to create a subfolder launcher+timestamp, if needed fill_mode (bool): whether to submit jobs even when there is nothing to run

(only in non-reservation mode)
fireworks.queue.queue_launcher.rapidfire(launchpad, fworker, qadapter, launch_dir=u'.', nlaunches=0, njobs_queue=0, njobs_block=500, sleep_time=None, reserve=False, strm_lvl=u'INFO', timeout=None, fill_mode=False)

Submit many jobs to the queue.


launchpad (LaunchPad) fworker (FWorker) qadapter (QueueAdapterBase) launch_dir (str): directory where we want to write the blocks nlaunches (int): total number of launches desired; “infinite” for loop, 0 for one round njobs_queue (int): stops submitting jobs when njobs_queue jobs are in the queue, 0 for no limit njobs_block (int): automatically write a new block when njobs_block jobs are in a single block sleep_time (int): secs to sleep between rapidfire loop iterations reserve (bool): Whether to queue in reservation mode strm_lvl (str): level at which to stream log messages timeout (int): # of seconds after which to stop the rapidfire process fill_mode (bool): whether to submit jobs even when there is nothing to run (only in

non-reservation mode)
fireworks.queue.queue_launcher.setup_offline_job(launchpad, fw, launch_id)

Module contents

Table Of Contents

This Page