Reference: dispel4py modulesΒΆ

Core modules

dispel4py.core The core module for dispel4py.
dispel4py.base Base PEs implementing typical processing patterns.
dispel4py.utils Collection of dispel4py utilities.
dispel4py.visualisation The IPython module for visualising a dispel4py graph using Graphviz dot.
dispel4py.workflow_graph The dispel4py workflow graph.

Enactment platforms

dispel4py.new.simple_process Simple sequential processor mapping for dispel4py graphs.
dispel4py.new.multi_process Enactment of dispel4py graphs using multiprocessing.
dispel4py.new.mpi_process Enactment of dispel4py graphs with MPI.
dispel4py.new.processor Enactment of dispel4py graphs.
dispel4py.storm.storm_submission Submits a dispel4py graph for processing to Storm.
dispel4py.storm.topology Creates a Storm topology from a dispel4py graph.

Examples and tests

dispel4py.examples.graph_testing.pipeline_test This is a dispel4py graph which produces a pipeline workflow with one producer node (prod) and 5 consumer nodes.
dispel4py.examples.graph_testing.split_merge This is a dispel4py graph which produces a workflow that splits the data and sends it to two nodes (cons1 and cons2) and the output of those two nodes is merged by another node (last).
dispel4py.examples.graph_testing.teecopy This is a dispel4py graph which produces a workflow that copies the data (from node prod) to two nodes (cons1 and cons2).
dispel4py.examples.graph_testing.group_by This is a dispel4py graph that shows the group-by data pattern to count words.
dispel4py.examples.graph_testing.grouping_alltoone This is a dispel4py graph which produces a workflow with a pipeline in which the producer node prod sends data to the consumer node cons1 which then sends data to node cons2.
dispel4py.examples.graph_testing.grouping_onetoall This is a dispel4py graph which produces a workflow which copies the data (from node prod) to one node (cons).
dispel4py.examples.graph_testing.grouping_split_merge This is a dispel4py graph which produces a workflow that sends copies of the output data from the producer node (words) to two nodes (filter1 and filter2), and the outputs of those two filters are merged in the last node (count).
dispel4py.examples.graph_testing.parallel_pipeline This is a dispel4py graph where each MPI process computes a partition of the workflow instead of a PE instance.
dispel4py.examples.graph_testing.partition_parallel_pipeline This graph is a modification of the parallel_pipeline example, showing how the user can specify how the graph is going to be partitioned into MPI processes.
dispel4py.examples.graph_testing.pipeline_composite This is a dispel4py graph which produces a pipeline workflow with one producer node and a chain of functions that process the data.
dispel4py.examples.graph_testing.word_count Counts words produced by a WordProducer.
dispel4py.examples.graph_testing.word_count_filter Counts words produced by RandomWordProducer and filtered by RandomFilter.
dispel4py.examples.graph_testing.testing_PEs Example PEs for test workflows, implementing various patterns.
dispel4py.test.simple_process_test Tests for simple sequential processing engine.
dispel4py.new.aggregate Processing elements that implement aggregation functions (AVG, SUM, COUNT, MIN, MAX).
dispel4py.new.aggregate_test Tests for aggregation processing elements.