taylorism.examples

Basic examples of how to use the module.

A more advanced example of use can be found in epygram’s epy_conv.py tool.

Functions

taylorism.examples.logarithmer_example(verbose=True)

Example: how to use a numpy array, shared among workers.

taylorism.examples.matrixproduct_example(A_shape=(6, 7), B_shape=(7, 8), iblocks=2, jblocks=4, verbose=True)

Example: how to use several numpy array shared among workers.

Parameters:

• A_shape – size of the A matrix

• B_shape – size of the B matrix

• iblocks – number of blocks in direction i of result matrix; must divide A_shape[0]

• jblocks – number of blocks in direction j of result matrix; must divide B_shape[1]

taylorism.examples.sleepers_example_bindedmemory(verbose=True)

Example: scheduling is driven by memory consumption and processes are binded.

taylorism.examples.sleepers_example_bindedthreads(verbose=True)

Example: scheduling is driven by number of threads and processes are binded.

taylorism.examples.sleepers_example_laxist(verbose=True)

Example: assuming no selection of strategy for scheduling.

taylorism.examples.sleepers_example_memory(verbose=True)

Example: scheduling is driven by memory consumption.

taylorism.examples.sleepers_example_threads(verbose=True)

Example: scheduling is driven by number of threads.

taylorism.examples.sleepers_generic_program(verbose=True, scheduler=None)

Generic example: how to run and control the Boss.

taylorism.examples.summer_example(verbose=True, use_lock=True)

Example: how to use a process-safe numpy array, shared among workers.

Classes

class taylorism.examples.Logarithmer(*args, **kwargs)

Bases: Worker

Computes the logarithm of an array.

OK, this would be much more efficient with numpy.log(array), but it is just an example of using shared numpy arrays among workers.

array

The shared-memory array on which to work on.

expected_time

How long the worker is expected to run (in s).

memory

Memory that should be used by the worker (in MiB).

name

Name of the worker.

row

Index of the row of the array on which the Worker is supposed to work.

scheduler_hooks

List of callbacks before starting effective task work.

scheduler_ticket

The slot number given by the scheduler (optional).

class taylorism.examples.MatrixProducter(*args, **kwargs)

Bases: Worker

Computes a Matrix Product C = A x B by hand, but parallel by blocks, using 3 SharedNumpyArray, each Worker being responsible for a [i1:i2, j1:j2] block of the resulting C matrix.

Again, numpy matrix products may probably be more efficient…

A

The A shared-memory array matrix.

B

The B shared-memory array matrix.

C

The C shared-memory array matrix.

expected_time

How long the worker is expected to run (in s).

i1

The first index i of the the Worker is responsible for.

i2

The last index i of the the Worker is responsible for.

j1

The first index j of the the Worker is responsible for.

j2

The first index j of the the Worker is responsible for.

memory

Memory that should be used by the worker (in MiB).

name

Name of the worker.

scheduler_hooks

List of callbacks before starting effective task work.

scheduler_ticket

The slot number given by the scheduler (optional).

class taylorism.examples.Sleeper(*args, **kwargs)

Bases: Worker

Sample worker for tutorial or debugging purpose, that sleeps a given time.

The over-loading of __init__() is not mandatory, but a possibility.

expected_time

How long the worker is expected to run (in s).

memory

Memory that should be used by the worker (in MiB).

name

Name of the worker.

scheduler_hooks

List of callbacks before starting effective task work.

scheduler_ticket

The slot number given by the scheduler (optional).

sleeping_time

Sleeping time in s.

wakeup_sentence

What to say after sleep.

class taylorism.examples.Summer(*args, **kwargs)

Bases: Worker

Computes a Sum, each Worker adding its value to a SharedNumpyArray.

This needs and illustrates process-safetiness.

expected_time

How long the worker is expected to run (in s).

memory

Memory that should be used by the worker (in MiB).

name

Name of the worker.

scheduler_hooks

List of callbacks before starting effective task work.

scheduler_ticket

The slot number given by the scheduler (optional).

shared_sum

The shared-memory array on which to sum.

use_lock

Whether to use the lock (thread-safe) or not (may lead to a wrong result !).

value

Value to be added by the worker.