simpler_with_shared_logging – How to share data across jobs

See here for a run through of this example.

The code for this example is in the test subdirectory of the ruffus module:

ruffus/test/simpler_with_shared_logging.py

The shared data:

--log_file_name, -L

shared log file location

Programme options:

--help, -h

show help message

To specify tasks:

--target_tasks TARGET_TASK, -t TARGET_TASK

Target task(s) of pipeline. TARGET_TASK can be

• task1
• task2
• task3
• task4

For example:

complicated_example.py -t task1 -t task4

--forced_tasks FORCED_TASK, -f FORCED_TASK

Pipeline task(s) which will be run even if they are up to date.

See above for a list of pipelined tasks

--jobs N, -j N

N specifies number of concurrent process running jobs in parallel

--minimal_rebuild_mode, -M

Rebuild a minimum of tasks necessary for the target. Ignore upstream out of date tasks if intervening tasks are up to date.

To print flowchart:

--dependency FILE, -d file

Print a dependency graph of the pipeline that would be executed to FILE, but do not execute it.

--dependency_graph_format FORMAT, -F FORMAT

Format of dependency graph file.

Can be:

* 'ps'
    (PostScript)

* 'svg'

* 'svgz'
    (Structured Vector Graphics),

* 'png'

* 'gif'
     (bitmap  graphics)

--just_print, -n

Print a description of the jobs that would be executed, but do not execute them.

--no_key_legend_in_graph, -K

Do not print out legend and key for dependency graph.

--draw_graph_horizontally, -H

Draw horizontal (left to right) dependency graph.

Code: