我正在尝试将一些JSON对象和两个整数传递给池。
for i in range(0, multiprocessing.cpu_count()-1):
fromindex = i * chunklen
toindex = (i+1) * chunklen
chunkedData.append([data['features'][fromindex:toindex], weekdaytopredict, hourtopredict])
chunkedData.append([data['features'][toindex:], weekdaytopredict, hourtopredict])
parallelstart = time.time()
result = (pool.map(parallelUpdateWithDT, chunkedData))
data
是一个包含一些多边形的地理 JSON 文件。我想分发这些多边形以进行并行处理。我将n/cpu_count()
多边形传递给 parallelUpdateWithDT
函数,该函数应进一步处理它们。我的问题是类型错误:即使print(chunkedData)
返回<class 'list'>
,我也会遇到以下错误:TypeError: a bytes-like object is required, not 'str'
。我在哪里搞砸了?全栈跟踪:
---------------------------------------------------------------------------
RemoteTraceback Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.5/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "<ipython-input-114-bf56cacb90b9>", line 34, in parallelUpdateWithDT
if('rain' in result):
TypeError: a bytes-like object is required, not 'str'
"""
The above exception was the direct cause of the following exception:
TypeError Traceback (most recent call last)
<ipython-input-115-031a5e24ee66> in <module>()
----> 1 decisionTreePrediciton(3, 5)
<ipython-input-114-bf56cacb90b9> in decisionTreePrediciton(weekdaytopredict, hourtopredict)
15 print (type(chunkedData))
16
---> 17 result = (pool.map(parallelUpdateWithDT, chunkedData))
18 parallelend = time.time()
19
/usr/lib/python3.5/multiprocessing/pool.py in map(self, func, iterable, chunksize)
258 in a list that is returned.
259 '''
--> 260 return self._map_async(func, iterable, mapstar, chunksize).get()
261
262 def starmap(self, func, iterable, chunksize=None):
/usr/lib/python3.5/multiprocessing/pool.py in get(self, timeout)
606 return self._value
607 else:
--> 608 raise self._value
609
610 def _set(self, i, obj):
chunkedData
示例:
[[[{'geometry': {'coordinates': [[[10.914622377957983, 45.682007076150505], [10.927456267537572, 45.68179119797432], [10.927147329501077, 45.672795442796335], [10.914315493899755, 45.67301125363092], [10.914622377957983, 45.682007076150505]]], 'type': 'Polygon'}, ///////////////////////etc, waaay too big////////////, 'id': 6574, 'properties': {'cellId': 11454}}], 3, 5]
这怎么是str
?我不明白。感谢您的任何帮助!
从您发布的代码中无法分辨出来,但我怀疑您正在尝试检查str
是否in
bytes
。 例如:
>>> bytes_obj = b'result'
>>> 'res' in bytes_obj
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: a bytes-like object is required, not 'str'
这意味着代码中的result
是 bytes
类型。 这里有两个决议。 第一种是将'rain'
也变成字节对象:
if b'rain' in result:
...
二是把result
变成str
:
result = result.decode(whatever_codec_it_should_be)
如果你要采取第二种方法,你应该尽早将结果转换为str
,以避免各种str
vs。 bytes
头痛。 通常,如果您不知道需要不同的编解码器,那么这些天大多数事情都可以utf-8
,因此您可以尝试一下......