Retrying: RetryExecutor

RetryExecutor produces futures which will implicitly retry on error.

class more_executors.RetryExecutor(delegate, retry_policy=None, logger=None, name='default', **kwargs)

An executor which delegates to another executor while adding implicit retry behavior.

  • Callables are submitted to the delegate executor, and may be submitted more than once if retries are required.

  • The callables may be submitted to that executor from a different thread than the calling thread.

  • Cancelling is supported if the delegate executor allows it.

  • Cancelling between retries is always supported.

  • Attempting to cancel a future prevents any more retries, regardless of whether the cancel succeeds.

  • The returned futures from this executor are only resolved or failed once the callable either succeeded, or all retries were exhausted. This includes activation of the done callback.

  • This executor is thread-safe.

Parameters

  • delegate (Executor) – executor to which callables will be submitted

  • retry_policy (RetryPolicy) – policy used to determine when futures shall be retried; if omitted, an ExceptionRetryPolicy is used, supplied with keyword arguments to this constructor

  • logger (Logger) – a logger used for messages from this executor

  • name (str) – a name for this executor

Changed in version 2.7.0: Introduced name.

submit_retry(retry_policy, fn, *args, **kwargs)

Submit a callable with a specific retry policy.

Parameters

retry_policy (RetryPolicy) – a policy which is used for this call only

classmethod Executors.with_retry(executor, retry_policy=None, logger=None)
Returns

a new executor which will retry callables on failure

Return type

RetryExecutor

class more_executors.RetryPolicy

Instances of this class may be supplied to RetryExecutor to customize the retry behavior.

This base class will never retry. See ExceptionRetryPolicy for a general-purpose implementation.

should_retry(attempt, future)
Parameters

  • attempt (int) – number of times the future has been attempted; starts counting at 1

  • future (Future) – a completed future

Returns

True if and only if a future should be retried.

Return type

bool

sleep_time(attempt, future)
Parameters

  • attempt (int) – number of times the future has been attempted; starts counting at 1

  • future (Future) – a completed future

Returns

The amount of time (in seconds) to delay before the next attempt at running a future.

Return type

float

class more_executors.ExceptionRetryPolicy(max_attempts, exponent, sleep, max_sleep, exception_base)

Retries on any exceptions under the given base class(es), up to a fixed number of attempts, with an exponential backoff between each attempt.

Parameters

  • max_attempts (int) – maximum number of times a callable should be attempted

  • exponent (float) – exponent used for backoff, e.g. 2.0 will result in a delay which doubles between each attempt

  • sleep (float) – base value for delay between attempts in seconds, e.g. 1.0 will delay by one second between first two attempts

  • max_sleep (float) – maximum delay between attempts, in seconds

  • exception_base (type, list(type)) – future will be retried if (and only if) it raises an exception inheriting from one of these classes

All parameters are optional; reasonable defaults apply when omitted.