我沿图(执行之间的更改(在不同任务上多次收到以下错误。当某些任务返回numpy.array
S和scipy.sparse
矩阵时,可能是
distributed.protocol.pickle - INFO - Failed to deserialize b'x80x04'
Traceback (most recent call last):
File "/home/user/venv/lib/python3.5/site-packages/distributed/protocol/pickle.py", line 59, in loads
return pickle.loads(x)
EOFError: Ran out of input
distributed.protocol.core - CRITICAL - Failed to deserialize
Traceback (most recent call last):
File "/home/user/venv/lib/python3.5/site-packages/distributed/protocol/core.py", line 119, in loads
value = _deserialize(head, fs)
File "/home/user/venv/lib/python3.5/site-packages/distributed/protocol/serialize.py", line 158, in deserialize
return f(header, frames)
File "/home/user/venv/lib/python3.5/site-packages/distributed/protocol/serialize.py", line 20, in <lambda>
deserializers = {None: lambda header, frames: pickle.loads(b''.join(frames))}
File "/home/user/venv/lib/python3.5/site-packages/distributed/protocol/pickle.py", line 59, in loads
return pickle.loads(x)
EOFError: Ran out of input
distributed.comm.utils - ERROR - truncated data stream (485 bytes): [b'', b"x92x83xa6reportxc2xa4keysx91xd9P('_avro_body-read-block-bag-from-delayed-67c7a9690149de9743ed970f873fa1d6', 283)xa2opxabdelete-datax86xa8priorityx93x00x01xccxcexa6nbytesx81xd9:('bag-from-delayed-67c7a9690149de9743ed970f873fa1d6', 283)xcex00 x86pxa8durationxcb@x18x16mx88xXx00xa7who_hasx81xd9:('bag-from-delayed-67c7a9690149de9743ed970f873fa1d6', 283)x91xb5tcp://127.0.0.1:38623xa2opxaccompute-taskxa3keyxd9K('pluck-map-process_features_sparse-d94d304dc59efb780c39bfb0ca4df37f', 283)", b'x83xabbytestringsx90xa7headersx81x92x01xa4taskx83xabcompressionx91xc0xa5countx01xa7lengthsx91x02xa4keysx91x92x01xa4task', b'x80x04']
distributed.worker - INFO - Connection to scheduler broken. Reregistering
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO - Registered to: tcp://127.0.0.1:8786
distributed.worker - INFO - -------------------------------------------------
这始终是一个EOFError: Ran out of input
错误,大小不同(有时与几个字节一样小(,整个群集都在一台计算机上运行。
理想情况下,我希望解决实际问题,但是研究问题并理解可能出错的方法也将受到赞赏。现在,我有点不知道如何解决该问题。
运行 client.get_versions(check=True)
毫无疑问完成,并且在更新所有软件包后(即numpy,scipy,dask,dask-distribed,cloudpickle(持续存在。
最近对cloudpickle
项目(DASK使用(进行了修补,以解决可能导致此错误的问题。
在此评论中解释了一些细节:https://github.com/ray-project/ray/ray/issues/2685#issuecomment-423182347
...以及更多详细信息可以在cloudpickle
github repo中的相关问题/PR中找到。
fwiw,我今天遇到了此错误(包括b'x80x04'
部分(,将cloudpickle
更新为0.8.0
似乎已修复。