Redis在与Celery组或链抛出一起使用时失去连接ERROR/MainProcess]与Redis的连接丢失:现在重

Question

我的问题是，当我使用芹菜任务启动我的工人时，redis在终端中显示以下错误。

我很幸运，在芹菜文档中使用了chord的修改版本来重现错误。问题似乎发生在和弦或我有太多组并行运行的情况下，即chord(add.s(i, i) for i in range(1, num))(list_add.s())或group(add.s(i, i) for i in range(1, num))()

下面是我的代码示例

@task
def add(x, y):
    return [x, y, x + y, "Next"]
@task
def list_add(nums):
    numbers = []
    count = 1
    for i in nums:
        print("{}). {}".format(count, i))
        numbers.extend(i)
        count += 1
    print(numbers)
    return numbers
@task
def foo(num):
    return chord(add.s(i, i) for i in range(1, num))(list_add.s())

下面是我的带有回溯的终端输出的一部分

    [2015-11-04 20:36:14,912: INFO/MainProcess] Received task: b2b.tasks.add[b87fdc44-e759-4224-bce4-11f9468d12b3]
[2015-11-04 20:36:14,913: INFO/MainProcess] Received task: b2b.tasks.add[120f5bf2-b962-4424-894b-d6f0ca56102b]
[2015-11-04 20:36:14,914: INFO/MainProcess] Task b2b.tasks.bar[9dc93c75-6404-4db3-a685-ff91460e1adb] succeeded in 1.00830382s: <AsyncResult: c891df3e-aa5c-4c9f-ad3f-30abe3b3ccc1>
    [2015-11-04 19:44:34,922: ERROR/MainProcess] Connection to Redis lost: Retry (0/20) now.
    [2015-11-04 19:44:34,922: ERROR/MainProcess] Connection to Redis lost: Retry (1/20) in 1.00 second.
    ...
    /Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/app/trace.py:365: RuntimeWarning: Exception raised outside body: ConnectionError('Error 8 connecting to localhost:6379. nodename nor servname provided, or not known.',):
    Traceback (most recent call last):
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/app/trace.py", line 235, in trace_task
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/base.py", line 256, in store_result
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/base.py", line 490, in _store_result
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/redis.py", line 160, in set
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/redis.py", line 149, in ensure
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/kombu/utils/__init__.py", line 243, in retry_over_time
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/celery/backends/redis.py", line 169, in _set
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/redis/client.py", line 2593, in execute
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/redis/client.py", line 2447, in _execute_transaction
      File "/Users/Me/.virtualenvs/djangoscrape/lib/python2.7/site-packages/redis/connection.py", line 532, in send_packed_command
      File "/Users/Michael/.virtualenvs/djangoscrape/lib/python2.7/site-packages/redis/connection.py", line 436, in connect
    ConnectionError: Error 8 connecting to localhost:6379. nodename nor servname provided, or not known.

这是我用来在终端中启动工作程序的命令芹菜-一个刮刀工人-P eventlet-c 1000-l信息

当foo（）被传递20时，它工作得很好，但当出现错误。

>>> a = foo.delay(20)     # works 
>>> a = foo.delay(1000)   # fails

如果你有想法，请建议如何解决这个问题，并提前表示感谢。

Answer 1

在IRC和redis用户组的Greg的帮助下，我终于找到了答案

通过将我的Mac OSX Yosemite上打开的文件限制更改为65536，问题得到了解决。

我希望这有一天能帮助到别人。