我有一个任务,运行大约两分钟,做一些数据库操作。我启用了延迟确认,但我发现在超过软时间限制后,该任务从未重试。
所以我添加了autoretry_for和retry_backoof,如我正在使用的芹菜版本的文档中所述。对延迟确认的任务进行重试是否是一种好的做法?
如果我没有设置autoretry_for
则任务不会从终止的位置运行。
tasks.py
@shared_task(soft_time_limit=20, autoretry_for=(SoftTimeLimitExceeded, ), retry_backoff=True)
def test_func_celery(current_time):
fake = Faker()
logger = logging.getLogger()
time1 = time.perf_counter()
main_time_end = current_time + 25
logger.info('Task started here n n')
while time.time() < main_time_end:
logger.info('Before doing main operations')
time.sleep(10)
logger.info(f'Inside Main operation ===>>> {time.time()}')
while True:
name = fake.first_name()
logger.info(f'Inside inner operation for {name}')
logger.info(f'inner operation before starts for {name}')
time.sleep(12)
logger.info(f'Inside inner {name}===>>> {time.time()}')
logger.info(f'inner operation after finishes for {name}')
break
logger.info('Completed Main operation')
break
if time.time() >= main_time_end:
logger.info('Out from main operations')
time2 = time.perf_counter()
logger.info(f'Total time =======>>>>>>> {time2 - time1}')
return None
views.py
def home(request):
current_time = time.time()
test_func_celery.apply_async(args=(current_time, ), acks_late=True)
return HttpResponse("Celery being tested here")
芹菜.log
[2020-01-21 15:28:30,708: INFO/ForkPoolWorker-7] Task started here
[2020-01-21 15:28:30,708: INFO/ForkPoolWorker-7] Before doing main operations
[2020-01-21 15:28:40,715: INFO/ForkPoolWorker-7] Inside Main operation ===>>> 1579599820.7153742
[2020-01-21 15:28:40,716: INFO/ForkPoolWorker-7] Inside inner operation for Joe
[2020-01-21 15:28:40,716: INFO/ForkPoolWorker-7] inner operation before starts for Joe
[2020-01-21 15:28:50,680: WARNING/MainProcess] Soft time limit (20s) exceeded for home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4]
[2020-01-21 15:28:50,726: INFO/MainProcess] Received task: home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] ETA:[2020-01-21 15:28:51.683861+05:45]
[2020-01-21 15:28:50,728: INFO/ForkPoolWorker-7] Task home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] retry: Retry in 1s: SoftTimeLimitExceeded()
[2020-01-21 15:28:51,938: INFO/ForkPoolWorker-7] Task started here
[2020-01-21 15:28:51,938: INFO/ForkPoolWorker-7] Before doing main operations
[2020-01-21 15:29:01,948: INFO/ForkPoolWorker-7] Inside Main operation ===>>> 1579599841.948412
[2020-01-21 15:29:01,949: INFO/ForkPoolWorker-7] Inside inner operation for Lisa
[2020-01-21 15:29:01,949: INFO/ForkPoolWorker-7] inner operation before starts for Lisa
[2020-01-21 15:29:11,915: WARNING/MainProcess] Soft time limit (20s) exceeded for home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4]
[2020-01-21 15:29:11,965: INFO/MainProcess] Received task: home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] ETA:[2020-01-21 15:29:12.917995+05:45]
[2020-01-21 15:29:11,967: INFO/ForkPoolWorker-7] Task home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] retry: Retry in 1s: SoftTimeLimitExceeded()
[2020-01-21 15:29:13,136: INFO/ForkPoolWorker-7] Task started here
[2020-01-21 15:29:13,136: INFO/ForkPoolWorker-7] Out from main operations
[2020-01-21 15:29:13,136: INFO/ForkPoolWorker-7] Total time =======>>>>>>> 0.00026752900157589465
[2020-01-21 15:29:13,138: INFO/ForkPoolWorker-7] Task home.tasks.test_func_celery[4234d05e-672e-4565-bdb6-cd3d234c82c4] succeeded in 0.05194566200225381s: None
有没有更好的解决方案来描述上面解释的场景?这将是一个很大的帮助。谢谢。
将retry_backoff设置为 False 会导致任务在异常后立即运行。当任务依赖于外部服务器或网络时,建议将 retry_backoff 设置为 True 以等待任何网络问题得到解决,然后再重试任务。如果任务没有外部依赖项,则无需将 retry_backoff 设置为 True。