taylorism.examples

Basic examples of how to use the module.

A more advanced example of use can be found in epygram’s epy_conv.py tool.

Functions

taylorism.examples.logarithmer_example(verbose=True)[source]

Example: how to use a numpy array, shared among workers.

taylorism.examples.matrixproduct_example(A_shape=(6, 7), B_shape=(7, 8), iblocks=2, jblocks=4, verbose=True)[source]

Example: how to use several numpy array shared among workers.

Parameters:

  • A_shape – size of the A matrix

  • B_shape – size of the B matrix

  • iblocks – number of blocks in direction i of result matrix; must divide A_shape[0]

  • jblocks – number of blocks in direction j of result matrix; must divide B_shape[1]

taylorism.examples.sleepers_example_bindedmemory(verbose=True)[source]

Example: scheduling is driven by memory consumption and processes are binded.

taylorism.examples.sleepers_example_bindedthreads(verbose=True)[source]

Example: scheduling is driven by number of threads and processes are binded.

taylorism.examples.sleepers_example_laxist(verbose=True)[source]

Example: assuming no selection of strategy for scheduling.

taylorism.examples.sleepers_example_memory(verbose=True)[source]

Example: scheduling is driven by memory consumption.

taylorism.examples.sleepers_example_threads(verbose=True)[source]

Example: scheduling is driven by number of threads.

taylorism.examples.sleepers_generic_program(verbose=True, scheduler=None)[source]

Generic example: how to run and control the Boss.

taylorism.examples.summer_example(verbose=True, use_lock=True)[source]

Example: how to use a process-safe numpy array, shared among workers.

Classes

class taylorism.examples.Logarithmer(*args, **kwargs)[source]

Bases: Worker

Computes the logarithm of an array.

OK, this would be much more efficient with numpy.log(array), but it is just an example of using shared numpy arrays among workers.

Note

This class is managed by footprint.

  • info: Not documented

  • priority: PriorityLevel::DEFAULT (rank=1)

Automatic parameters from the footprint:

  • array (taylorism.util.SharedNumpyArray) - rwx - The shared-memory array on which to work on.

  • expected_time (builtins.float) - rxx - How long the worker is expected to run (in s).

    • Optional. Default is 0.0.

  • memory (builtins.float) - rxx - Memory that should be used by the worker (in MiB).

    • Optional. Default is 0.0.

  • name (builtins.str) - rwx - Name of the worker.

    • Optional. Default is None.

  • row (builtins.int) - rxx - Index of the row of the array on which the Worker is supposed to work.

  • scheduler_hooks (footprints.stdtypes.FPList) - rxx - List of callbacks before starting effective task work.

    • Optional. Default is FPList::<<as_list:: []>>.

  • scheduler_ticket (builtins.int) - rxx - The slot number given by the scheduler (optional).

    • Optional. Default is None.

array

The shared-memory array on which to work on (see the documentation above for more details).

expected_time

How long the worker is expected to run (in s) (see the documentation above for more details).

memory

Memory that should be used by the worker (in MiB) (see the documentation above for more details).

name

Name of the worker (see the documentation above for more details).

row

Index of the row of the array on which the Worker is supposed to work (see the documentation above for more details).

scheduler_hooks

List of callbacks before starting effective task work (see the documentation above for more details).

scheduler_ticket

The slot number given by the scheduler (optional) (see the documentation above for more details).

class taylorism.examples.MatrixProducter(*args, **kwargs)[source]

Bases: Worker

Computes a Matrix Product C = A x B by hand, but parallel by blocks, using 3 SharedNumpyArray, each Worker being responsible for a [i1:i2, j1:j2] block of the resulting C matrix.

Again, numpy matrix products may probably be more efficient…

Note

This class is managed by footprint.

  • info: Not documented

  • priority: PriorityLevel::DEFAULT (rank=1)

Automatic parameters from the footprint:

  • A (taylorism.util.SharedNumpyArray) - rwx - The A shared-memory array matrix.

  • B (taylorism.util.SharedNumpyArray) - rwx - The B shared-memory array matrix.

  • C (taylorism.util.SharedNumpyArray) - rwx - The C shared-memory array matrix.

  • expected_time (builtins.float) - rxx - How long the worker is expected to run (in s).

    • Optional. Default is 0.0.

  • i1 (builtins.int) - rxx - The first index i of the the Worker is responsible for.

  • i2 (builtins.int) - rxx - The last index i of the the Worker is responsible for.

  • j1 (builtins.int) - rxx - The first index j of the the Worker is responsible for.

  • j2 (builtins.int) - rxx - The first index j of the the Worker is responsible for.

  • memory (builtins.float) - rxx - Memory that should be used by the worker (in MiB).

    • Optional. Default is 0.0.

  • name (builtins.str) - rwx - Name of the worker.

    • Optional. Default is None.

  • scheduler_hooks (footprints.stdtypes.FPList) - rxx - List of callbacks before starting effective task work.

    • Optional. Default is FPList::<<as_list:: []>>.

  • scheduler_ticket (builtins.int) - rxx - The slot number given by the scheduler (optional).

    • Optional. Default is None.

A

The A shared-memory array matrix (see the documentation above for more details).

B

The B shared-memory array matrix (see the documentation above for more details).

C

The C shared-memory array matrix (see the documentation above for more details).

expected_time

How long the worker is expected to run (in s) (see the documentation above for more details).

i1

The first index i of the the Worker is responsible for (see the documentation above for more details).

i2

The last index i of the the Worker is responsible for (see the documentation above for more details).

j1

The first index j of the the Worker is responsible for (see the documentation above for more details).

j2

The first index j of the the Worker is responsible for (see the documentation above for more details).

memory

Memory that should be used by the worker (in MiB) (see the documentation above for more details).

name

Name of the worker (see the documentation above for more details).

scheduler_hooks

List of callbacks before starting effective task work (see the documentation above for more details).

scheduler_ticket

The slot number given by the scheduler (optional) (see the documentation above for more details).

class taylorism.examples.Sleeper(*args, **kwargs)[source]

Bases: Worker

Sample worker for tutorial or debugging purpose, that sleeps a given time.

The over-loading of __init__() is not mandatory, but a possibility.

Note

This class is managed by footprint.

  • info: Sleeps.

  • priority: PriorityLevel::DEFAULT (rank=1)

Automatic parameters from the footprint:

  • expected_time (builtins.float) - rxx - How long the worker is expected to run (in s).

    • Optional. Default is 0.0.

  • memory (builtins.float) - rxx - Memory that should be used by the worker (in MiB).

    • Optional. Default is 0.0.

  • name (builtins.str) - rwx - Name of the worker.

    • Optional. Default is None.

  • scheduler_hooks (footprints.stdtypes.FPList) - rxx - List of callbacks before starting effective task work.

    • Optional. Default is FPList::<<as_list:: []>>.

  • scheduler_ticket (builtins.int) - rxx - The slot number given by the scheduler (optional).

    • Optional. Default is None.

  • sleeping_time (builtins.float) - rxx - Sleeping time in s.

    • Values: set([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 15.0, 20.0, 25.0, 30.0, 35.0, 40.0, 45.0, 50.0, 55.0, 60.0, 0.1, 0.01, 0.001])

  • wakeup_sentence (builtins.str) - rwx - What to say after sleep.

    • Optional. Default is ‘Hello !’.

expected_time

How long the worker is expected to run (in s) (see the documentation above for more details).

memory

Memory that should be used by the worker (in MiB) (see the documentation above for more details).

name

Name of the worker (see the documentation above for more details).

scheduler_hooks

List of callbacks before starting effective task work (see the documentation above for more details).

scheduler_ticket

The slot number given by the scheduler (optional) (see the documentation above for more details).

sleeping_time

Sleeping time in s (see the documentation above for more details).

wakeup_sentence

What to say after sleep (see the documentation above for more details).

class taylorism.examples.Summer(*args, **kwargs)[source]

Bases: Worker

Computes a Sum, each Worker adding its value to a SharedNumpyArray.

This needs and illustrates process-safetiness.

Note

This class is managed by footprint.

  • info: Not documented

  • priority: PriorityLevel::DEFAULT (rank=1)

Automatic parameters from the footprint:

  • expected_time (builtins.float) - rxx - How long the worker is expected to run (in s).

    • Optional. Default is 0.0.

  • memory (builtins.float) - rxx - Memory that should be used by the worker (in MiB).

    • Optional. Default is 0.0.

  • name (builtins.str) - rwx - Name of the worker.

    • Optional. Default is None.

  • scheduler_hooks (footprints.stdtypes.FPList) - rxx - List of callbacks before starting effective task work.

    • Optional. Default is FPList::<<as_list:: []>>.

  • scheduler_ticket (builtins.int) - rxx - The slot number given by the scheduler (optional).

    • Optional. Default is None.

  • shared_sum (taylorism.util.SharedNumpyArray) - rwx - The shared-memory array on which to sum.

  • use_lock (builtins.bool) - rxx - Whether to use the lock (thread-safe) or not (may lead to a wrong result !).

  • value (builtins.int) - rxx - Value to be added by the worker.

expected_time

How long the worker is expected to run (in s) (see the documentation above for more details).

memory

Memory that should be used by the worker (in MiB) (see the documentation above for more details).

name

Name of the worker (see the documentation above for more details).

scheduler_hooks

List of callbacks before starting effective task work (see the documentation above for more details).

scheduler_ticket

The slot number given by the scheduler (optional) (see the documentation above for more details).

shared_sum

The shared-memory array on which to sum (see the documentation above for more details).

use_lock

Whether to use the lock (thread-safe) or not (may lead to a wrong result !) (see the documentation above for more details).

value

Value to be added by the worker (see the documentation above for more details).