我正在使用Eventlet类和并发设置为8的标准芹菜工人。这些在发生这种情况时非常忙(但是可能在不忙碌时发生,很难告诉)。
我知道我的任务没有任何泄漏,并且在每个孩子设置的最大任务中运行它,以防万一。
不过,我遇到了这些错误,我不知道为什么。据我所知,这是随机的。它在我的环境中吗?已知的问题?除非我误会,否则由于任务本身,这似乎并不是错误。我对此错了吗?任何想法都会很棒!
另外,当作为预叉运行时,我没有得到这些错误。
错误1:
2015-05-02 06:13:37,452: WARNING/MainProcess] /opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/app/trace.py:364:
RuntimeWarning: Exception raised outside body: SystemError('error return without exception set',):
Traceback (most recent call last):
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/app/trace.py", line 283, in trace_task
uuid, retval, SUCCESS, request=task_request,
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/backends/base.py", line 248, in store_result
request=request, **kwargs)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/backends/base.py", line 481, in _store_result
self.set(self.get_key_for_task(task_id), self.encode(meta))
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/backends/cache.py", line 126, in set
return self.client.set(key, value, self.expires)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/memcache.py", line 584, in set
return self._set("set", key, val, time, min_compress_len)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/memcache.py", line 835, in _set
return _unsafe_set()
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/memcache.py", line 827, in _unsafe_set
return(server.expect("STORED", raise_exception=True)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/memcache.py", line 1196, in expect
line = self.readline(raise_exception)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/memcache.py", line 1182, in readline
data = recv(4096)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/eventlet-0.16.0.dev-py2.7.egg/eventlet/greenio.py", line 325, in recv
timeout_exc=socket.timeout("timed out"))
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/eventlet-0.16.0.dev-py2.7.egg/eventlet/greenio.py", line 200, in _trampoline
mark_as_closed=self._mark_as_closed)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/eventlet-0.16.0.dev-py2.7.egg/eventlet/hubs/__init__.py", line 159, in trampoline
return hub.switch()
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/eventlet-0.16.0.dev-py2.7.egg/eventlet/hubs/hub.py", line 293, in switch
return self.greenlet.switch()
SystemError: error return without exception set
错误2:
Traceback (most recent call last):
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/worker/__init__.py", line 227, in _process_task
req.execute_using_pool(self.pool)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/worker/job.py", line 263, in execute_using_pool
correlation_id=uuid,
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/concurrency/base.py", line 156, in apply_async
**options)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/concurrency/eventlet.py", line 144, in on_apply
self.getpid)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/eventlet-0.16.0.dev-py2.7.egg/eventlet/greenpool.py", line 106, in spawn_n
self.sem.acquire()
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/eventlet-0.16.0.dev-py2.7.egg/eventlet/semaphore.py", line 96, in acquire
hubs.get_hub().switch()
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/eventlet-0.16.0.dev-py2.7.egg/eventlet/hubs/hub.py", line 293, in switch
return self.greenlet.switch()
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/eventlet-0.16.0.dev-py2.7.egg/eventlet/greenpool.py", line 93, in _spawn_n_impl
self._spawn_done(coro)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/eventlet-0.16.0.dev-py2.7.egg/eventlet/greenpool.py", line 125, in _spawn_done
self.coroutines_running.remove(coro)
KeyError: <greenlet.greenlet object at 0xc759e0>
错误3:
[2015-05-02 08:41:48,786: WARNING/MainProcess] /opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/app/trace.py:364:
RuntimeWarning: Exception raised outside body: TypeError('sequence item 1: expected string, NoneType found',):
Traceback (most recent call last):
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/app/trace.py", line 253, in trace_task
I, R, state, retval = on_error(task_request, exc, uuid)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/app/trace.py", line 201, in on_error
R = I.handle_error_state(task, eager=eager)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/app/trace.py", line 85, in handle_error_state
}[self.state](task, store_errors=store_errors)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/app/trace.py", line 118, in handle_failure
req.id, exc, einfo.traceback, request=req,
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/backends/base.py", line 121, in mark_as_failure
traceback=traceback, request=request)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/backends/base.py", line 248, in store_result
request=request, **kwargs)
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/backends/base.py", line 481, in _store_result
self.set(self.get_key_for_task(task_id), self.encode(meta))
File "/opt/app/thisapp/software/python/lib/python2.7/site-packages/celery-3.1.16-py2.7.egg/celery/backends/base.py", line 406, in get_key_for_task
self.task_keyprefix, key_t(task_id), key_t(key),
TypeError: sequence item 1: expected string, NoneType found
尝试降低芹菜并发。
我们没有Solaris的良好并发实施(HUB)。除了真正没有支持的Pyevent。因此,它将使用Select或Poll,并且仅限于观看1024个文件描述符。一项任务可能使用零或更多文件描述符,因此我提议比现在更低,没有特定的数字。
升级到最新的Eventlet版本也是一个好主意。今天2015-05-04是0.17.3
。