Get number of CPUs available.
Fetch the number of CPUs available based on the current priority and limit configuration.
| Returns: | An integer with the number of CPUs that are available. |
|---|---|
| Return type: | int |
Fetch all running DummyMP system process IDs.
Looking through all of the DummyMP processes, determine if any are running, and if any, retrieve their system process IDs.
| Returns: | An list with the system process IDs of current running spawned DummyMP processes. |
|---|---|
| Return type: | list |
Get the total number of CPUs on the system.
Detects and returns the total number of CPUs on the system.
| Returns: | An integer with the number of CPUs that are on the system. |
|---|---|
| Return type: | int |
Check if the CPU availability needs to be updated.
Utilizing configuration and checking the time elapsed from the previous check, determine whether the CPU availability function, getCPUAvail(), needs to be run or not to update CPU availability.
| Returns: | Boolean indicating whether the CPU availability needs to be updated or not. |
|---|---|
| Return type: | bool |
Poll for processes and return information about them.
Polls all processes and returns a dictionary with information about all of the processes.
| Parameters: | interval (int) – Delay before fetching process status. This is akin to top -d ##, where a delay is specified. |
|---|---|
| Returns: | Tuple with the first element being a list of Process
objects, each with a .dict attribute. Each .dict attribute
contains information about the process, stored in the form of a
dict. It includes:
The second element in the tuple contains a summary dictionary with the total number of running, stopped, and zombie processes, with keys ‘running’, ‘stopped’, and ‘zombie’, respectively. (
[Process(
pid=30159, name='services.exe',
dict={'username':USERNAME, ...}
), ...],
{'running': ##, 'stopped': ##, ...}
)
|
| Return type: | tuple |
Get maximum processors for DummyMP to use.
Fetch and return the maximum number of processors/CPUs for DummyMP to use.
| Returns: | Integer specifying the current maximum number of processors (or CPUs) for DummyMP to use. |
|---|---|
| Return type: | int |
Get the priority mode that DummyMP is using.
Fetch and return the priority mode integer constant that DummyMP is using.
| Returns: | Integer constant specifying the priority mode integer constant that DummyMP is using. |
|---|---|
| Return type: | int |
Get function returns from completed function runs.
Fetch and return a dictionary of function returns from completed function runs. The dictionary is indexed by call order, zero indexed.
| Returns: | Dictionary of function returns, indexed by call order, and zero indexed. |
|---|---|
| Return type: | dict |
Get DummyMP’s version
Fetch and return the DummyMP library version.
| Returns: | String specifying the DummyMP library version. |
|---|---|
| Return type: | str |
Kill all currently running processes and remove them from queue.
Kill all of the currently running processes and remove them from the internal running queue.
Reset DummyMP state and kill all currently running processes.
This resets the DummyMP state, clearing any configuration or state data set. Before resetting, this will kill all of the currently running processes and remove them from the internal running queue. (This calls killall() before resetting.)
Run a function with multiprocessing.
Run a function with multiprocessing. This actually queues the function and its arguments for running - you need to run process_process() or process_until_done() in order to actually execute the function.
| Parameters: |
|---|
Set whether to deepcopy arguments or not.
Change whether DummyMP will deepcopy the arguments or not. If it is disabled, it is up to the user to ensure that a copy of the arguments is preserved for further use.
| Parameters: | tf (bool) – Boolean indicating whether to enable argument deepcopy or not. |
|---|
Set the process completion callback for DummyMP.
Set the callback for DummyMP to call when a process has completed.
The callback is called with the following arguments: (config.total_completed, config.total_running, config.total_procs)
| Parameters: | callback (function) – Function callback to call when a process has completed. |
|---|
Set whether to deepcopy keyword arguments or not.
Change whether DummyMP will deepcopy the keyword arguments or not. If it is disabled, it is up to the user to ensure that a copy of the keyword arguments is preserved for further use.
| Parameters: | tf (bool) – Boolean indicating whether to enable keyword argument deepcopy or not. |
|---|
Set maximum processors for DummyMP to use.
Set the maximum number of processors/CPUs for DummyMP to use. Regardless of priority mode, the number of processors/CPUs it takes advantage of will be capped at the number you specify.
| Parameters: | max_proc (int) – Integer specifying the maximum number of processors (or CPUs) for DummyMP to use. |
|---|
Set the priority mode for DummyMP.
Set the priority mode for DummyMP. The priority mode controls how conscious DummyMP is of other running processes on the system.
Available modes:
- DUMMYMP_GENEROUS - Generous mode.
- Very conservative about using any CPU, and ensures that no one else is disrupted. Note that this is VERY GENEROUS - if all CPUs are taken, DummyMP will wait until there are available CPUs! (All other modes will run a single process, regardless of CPU usage.) This is the slowest mode!
- DUMMYMP_NORMAL - Normal mode.
- Careful not to take up too much resources on the CPU, but it will try to get things done. This is faster than GENEROUS, but it isn’t the fastest. This mode is the default and is recommended for most conditions.
- DUMMYMP_AGGRESSIVE - Aggressive mode.
- This mode considers other users, but it may spawn processes anyway depending on how other processes behave. This is faster than NORMAL, and is recommended for semi-important conditions.
- DUMMYMP_EXTREME - Extreme mode.
- This mode somewhat considers other users, but unless the other processes are using a significant portion of the CPU, it will spawn processes anyway. This is faster than AGGRESSIVE, and is recommended for important conditions.
- DUMMYMP_NUCLEAR - Nuclear mode.
- This mode does NOT consider other users, and just runs as many processes as it can allow (total number of cores). This is much faster than EXTREME, and is recommended for really important conditions. Note that this may earn you very angry co-workers knocking down your door with pitchforks, so use sparingly!
Regardless of priority mode, if the number of CPUs to use is specified, the number of processors/CPUs it takes advantage of will always be capped at the maximum number specified.
| Parameters: | mode (int) – Integer constant specifying the mode for DummyMP to use. |
|---|
Set the process starting callback for DummyMP.
Set the callback for DummyMP to call when a process is initially started.
The callback is called with the following arguments: (config.total_completed, config.total_running, config.total_procs)
| Parameters: | callback (function) – Function callback to call when a process starts. |
|---|
Bases: logging.Handler
DummyMP logging handler to allow multiprocess logging.
This class is a custom logging handler to allow spawned processes (from multiprocessing) to log without any issues. This works by intercepting emitted log records, and sending them via queue to the master process. The master process will process each record and call logging.Logger.handle() to emit the logging record at the master process level.
Note that this class can be used as a general multiprocess logging handler simply by removing the int_pid attribute.
The Queue object to forward logging records to.
int
The internal PID used to reference the process.
Multiprocess function wrapper for running a function given args.
This function wraps an existing function with its args, and allows for additional fields related to multiprocessing. In particular, the internal process ID and the multiprocessing.Queue object are added to facilitate communication between the subprocess and the master process.
| Parameters: |
|
|---|---|
| Returns: | Nothing... but the function will periodically send messages to the Queue in this format:
|
Process the execution queue and inter-process messages.
Process the execution queue by starting processes in said queue, handle processes that have completed, and process inter-process messages via process_queue().
(In plain English: start the queued processes, check processes to see if they are done running, and grab any inter-process messages sent from the spawned process.)
| Returns: | A boolean indicating whether the execution queue has completed or not. Returns True if it has completed, False if it has not. This return value can be used in a while loop to block until processes have completed. (This is somewhat similar to multiprocessing’s multiprocessing.Process.join() if used in a loop.) |
|---|---|
| Return type: | bool |
Process inter-process messages.
Process the inter-process multiprocessing.Queue objects, which receive messages from the spawned process for logging events and function returns.
Note
Warnings are emitted to log if an invalid message is received.
Process the execution queue until all have been completed.
Process the execution queue until it has indicated that all processes in the queue have been completed. (This is somewhat similar to multiprocessing’s multiprocessing.Process.join().)