Celery任务获取SoftTimeLimitExceeded调用API

Question

我有两个任务，需要调用两个不同的API来发送短信和电子邮件。

视图.py

def ...:
...
send_payment_failed_sms.delay(payment_log.id, 1)
task_send_booking_failed_email(payment_log.id, 1)  

但我总是在发送短信的任务上出现以下错误，但发送电子邮件的任务运行良好：

[2018-07-12 20:47:03,519: INFO/MainProcess] Received task: payments.tasks.send_payment_failed_sms[ac4f1c2a-f09d-4e79-89c9-dfe1ab26b25d]  
[2018-07-12 20:47:18,548: WARNING/MainProcess] Soft time limit (15s) exceeded for payments.tasks.send_payment_failed_sms[ac4f1c2a-f09d-4e79-89c9-dfe1ab26b25d]
[2018-07-12 20:47:18,563: WARNING/ForkPoolWorker-1] on_failure
[2018-07-12 20:47:18,563: WARNING/ForkPoolWorker-1] ac4f1c2a-f09d-4e79-89c9-dfe1ab26b25d
[2018-07-12 20:47:18,564: WARNING/ForkPoolWorker-1] 'ac4f1c2a-f09d-4e79-89c9-dfe1ab26b25d' failed: SoftTimeLimitExceeded()
[2018-07-12 20:47:18,564: ERROR/ForkPoolWorker-1] Task payments.tasks.send_payment_failed_sms[ac4f1c2a-f09d-4e79-89c9-dfe1ab26b25d] raised unexpected: SoftTimeLimitExceeded()
Traceback (most recent call last):
...
billiard.exceptions.SoftTimeLimitExceeded: SoftTimeLimitExceeded()

tasks.py

@shared_task(bind=True, base=MyBaseClassForTask, max_retries=5, default_retry_delay=1 * 60, soft_time_limit=15,
time_limit=30)
def send_payment_failed_sms(self, payment_id, reason):
try:
try:
payload = ""
headers = {'content-type': 'application/x-www-form-urlencoded'}
url = 'https://sms-site.com/apikey=' + api_key + '&to=%s&var1=%s' % (
mobile, payment_log.id
)
response = requests.request("GET", url, data=payload, headers=headers)
response = response.json()
if response['Status'] == 'Success':
print('Sent message'), response['Details']
else:
print('Error:', response['Details'])
except ConnectionError as exc:
print('connection error @ failed sms')
raise self.retry(exc=exc)
except PaymentLog.DoesNotExist:
...

此外，它不会重试max_retrys，即5次。你能告诉我我做错了什么吗？或者在调用API时有更好的方法来定义任务吗？

Answer 1

你有soft_time_limit=15 seconds，在这段时间之后，芹菜提高SoftTimeLimitExceeded这个选项允许你在任务中进行清理

from celery.exceptions import SoftTimeLimitExceeded
@celery.task(soft_time_limit=15, time_limit=20)
def mytask():
try:
return do_work()
except SoftTimeLimitExceeded:
cleanup_in_a_hurry()