dask分布式未能用numpy.Arrays和Sparse.Matrices进行挑选



我沿图(执行之间的更改(在不同任务上多次收到以下错误。当某些任务返回numpy.array S和scipy.sparse矩阵时,可能是

distributed.protocol.pickle - INFO - Failed to deserialize b'x80x04'
Traceback (most recent call last):
  File "/home/user/venv/lib/python3.5/site-packages/distributed/protocol/pickle.py", line 59, in loads
    return pickle.loads(x)
EOFError: Ran out of input
distributed.protocol.core - CRITICAL - Failed to deserialize
Traceback (most recent call last):
  File "/home/user/venv/lib/python3.5/site-packages/distributed/protocol/core.py", line 119, in loads
    value = _deserialize(head, fs)
  File "/home/user/venv/lib/python3.5/site-packages/distributed/protocol/serialize.py", line 158, in deserialize
    return f(header, frames)
  File "/home/user/venv/lib/python3.5/site-packages/distributed/protocol/serialize.py", line 20, in <lambda>
    deserializers = {None: lambda header, frames: pickle.loads(b''.join(frames))}
  File "/home/user/venv/lib/python3.5/site-packages/distributed/protocol/pickle.py", line 59, in loads
    return pickle.loads(x)
EOFError: Ran out of input
distributed.comm.utils - ERROR - truncated data stream (485 bytes): [b'', b"x92x83xa6reportxc2xa4keysx91xd9P('_avro_body-read-block-bag-from-delayed-67c7a9690149de9743ed970f873fa1d6', 283)xa2opxabdelete-datax86xa8priorityx93x00x01xccxcexa6nbytesx81xd9:('bag-from-delayed-67c7a9690149de9743ed970f873fa1d6', 283)xcex00 x86pxa8durationxcb@x18x16mx88xXx00xa7who_hasx81xd9:('bag-from-delayed-67c7a9690149de9743ed970f873fa1d6', 283)x91xb5tcp://127.0.0.1:38623xa2opxaccompute-taskxa3keyxd9K('pluck-map-process_features_sparse-d94d304dc59efb780c39bfb0ca4df37f', 283)", b'x83xabbytestringsx90xa7headersx81x92x01xa4taskx83xabcompressionx91xc0xa5countx01xa7lengthsx91x02xa4keysx91x92x01xa4task', b'x80x04']
distributed.worker - INFO - Connection to scheduler broken. Reregistering
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -         Registered to:       tcp://127.0.0.1:8786
distributed.worker - INFO - -------------------------------------------------

这始终是一个EOFError: Ran out of input错误,大小不同(有时与几个字节一样小(,整个群集都在一台计算机上运行。

理想情况下,我希望解决实际问题,但是研究问题并理解可能出错的方法也将受到赞赏。现在,我有点不知道如何解决该问题。

运行 client.get_versions(check=True)毫无疑问完成,并且在更新所有软件包后(即numpy,scipy,dask,dask-distribed,cloudpickle(持续存在。

最近对cloudpickle项目(DASK使用(进行了修补,以解决可能导致此错误的问题。

在此评论中解释了一些细节:https://github.com/ray-project/ray/ray/issues/2685#issuecomment-423182347

...以及更多详细信息可以在cloudpickle github repo中的相关问题/PR中找到。

fwiw,我今天遇到了此错误(包括b'x80x04'部分(,将cloudpickle更新为0.8.0似乎已修复。

最新更新