shell-retry
Wrapper for call any utilities with retries until they succeed
Install
pip install shell-retryExamples
Let's start from --help:
$ shell-retry --help
usage: shell-retry [-h] [--backoff BACKOFF] [--retry-count RETRY_COUNT]
[--interval INTERVAL] [--verbose]
cmd [cmd ...]
positional arguments:
cmd
optional arguments:
-h, --help show this help message and exit
--backoff BACKOFF backoff factor (sleep(--interval *= --backoff)
--retry-count RETRY_COUNT
How many time re-run cmd if it fails
--interval INTERVAL Initial interval between retries
--interval-max INTERVAL_MAX
upper limit for interval
--interval-min INTERVAL_MIN
lower limit for interval
--verbose Be verbose, write how many retries left and how long
will we wait- To debug something use
--verboseflag. -
--retry-countspecifies retry (not a try) count. -
--intervalsets initial interval between retries, interval multiplies with backoff before next retry.
$ shell-retry --verbose --retry-count=5 --backoff=1.3 false
2018-02-22 18:23:06,682 INFO: Namespace(backoff=1.3, cmd=['false'], interval=1, retry_count=5, verbose=True)
2018-02-22 18:23:06,683 INFO: run ['false']
2018-02-22 18:23:06,687 INFO: command returned 1
2018-02-22 18:23:06,687 INFO: waiting 1.000000 seconds, 5 retries left
2018-02-22 18:23:07,687 INFO: run ['false']
2018-02-22 18:23:07,692 INFO: command returned 1
2018-02-22 18:23:07,692 INFO: waiting 1.300000 seconds, 4 retries left
2018-02-22 18:23:08,995 INFO: run ['false']
2018-02-22 18:23:08,999 INFO: command returned 1
2018-02-22 18:23:08,999 INFO: waiting 1.690000 seconds, 3 retries left
2018-02-22 18:23:10,690 INFO: run ['false']
2018-02-22 18:23:10,696 INFO: command returned 1
2018-02-22 18:23:10,697 INFO: waiting 2.197000 seconds, 2 retries left
2018-02-22 18:23:12,896 INFO: run ['false']
2018-02-22 18:23:12,902 INFO: command returned 1
2018-02-22 18:23:12,903 INFO: waiting 2.856100 seconds, 1 retries left
2018-02-22 18:23:15,764 INFO: run ['false']
2018-02-22 18:23:15,769 INFO: command returned 1To use some flags in cmd use -- before cmd.
$ shell-retry --retry-count=5 --backoff=1.2 -- curl -m 1 --connect-time 1 http://10.30.33.32
curl: (28) Connection timed out after 1000 milliseconds
curl: (28) Connection timed out after 1004 milliseconds
curl: (28) Connection timed out after 1003 milliseconds
curl: (28) Connection timed out after 1002 milliseconds
curl: (28) Connection timed out after 1000 milliseconds
curl: (28) Connection timed out after 1000 millisecondsTo limit interval between retries you can use options --interval-max and --interval-min:
$ shell-retry --verbose --retry-count=3 --backoff=1.2 --interval-max=1.2 -- curl -m 1 --connect-time 1 http://10.30.33.32
2018-02-22 19:21:59,170 INFO: Namespace(backoff=1.2, cmd=['curl', '-m', '1', '--connect-time', '1', 'http://10.30.33.32'], interval=1, interval_max=1.2, interval_min=None, retry_count=3, verbose=True)
2018-02-22 19:21:59,170 INFO: run ['curl', '-m', '1', '--connect-time', '1', 'http://10.30.33.32']
curl: (28) Connection timed out after 1000 milliseconds
2018-02-22 19:22:00,184 INFO: command returned 28
2018-02-22 19:22:00,185 INFO: waiting 1.000000 seconds, 3 retries left
2018-02-22 19:22:01,187 INFO: run ['curl', '-m', '1', '--connect-time', '1', 'http://10.30.33.32']
curl: (28) Connection timed out after 1005 milliseconds
2018-02-22 19:22:02,209 INFO: command returned 28
2018-02-22 19:22:02,210 INFO: waiting 1.200000 seconds, 2 retries left
2018-02-22 19:22:03,414 INFO: run ['curl', '-m', '1', '--connect-time', '1', 'http://10.30.33.32']
curl: (28) Connection timed out after 1001 milliseconds
2018-02-22 19:22:04,432 INFO: command returned 28
2018-02-22 19:22:04,432 INFO: waiting 1.200000 seconds, 1 retries left
2018-02-22 19:22:05,638 INFO: run ['curl', '-m', '1', '--connect-time', '1', 'http://10.30.33.32']
curl: (28) Connection timed out after 1006 milliseconds
2018-02-22 19:22:06,662 INFO: command returned 28
