Skip to content

Termination strategies

This module is concerned with the termination of the workers of a scheduler.

Most Executors on shutdown(wait=False) will not terminate their workers but just cancel pending futures. This means that currently running tasks will continue to run until they finish and have their result discarded as the program will have moved on.

We provide some custom strategies for known executors.

Note

There is no known way in basic Python to forcibully terminate a thread that does not account for early terminiation explicitly.

def termination_strategy(executor) #

Return a termination strategy for the given executor.

PARAMETER DESCRIPTION
executor

The executor to get a termination strategy for.

TYPE: _Executor

RETURNS DESCRIPTION
Callable[[_Executor], None] | None

A termination strategy for the given executor, or None if no

Callable[[_Executor], None] | None

termination strategy is available.

Source code in src/amltk/scheduling/termination_strategies.py
def termination_strategy(executor: _Executor) -> Callable[[_Executor], None] | None:
    """Return a termination strategy for the given executor.

    Args:
        executor: The executor to get a termination strategy for.

    Returns:
        A termination strategy for the given executor, or None if no
        termination strategy is available.
    """
    if isinstance(executor, ThreadPoolExecutor):
        return None

    if isinstance(executor, ProcessPoolExecutor):
        return _terminate_with_psutil  # type: ignore

    # Dask process based things seem pretty happy to close nicely and need
    # no special treatment.

    return None