Python3，多处理模块在不同环境下的行为

Question

我有以下代码，并尝试在Linux和Windows上运行。在Linux上，代码运行良好，但在Windows上出现运行时错误。我也试着在在线编译器上运行代码，代码运行得非常好。

这个问题在某种程度上与self关键字有关。

我没有任何苹果产品，如果有人能在Mac操作系统上运行它会很有帮助。

非常感谢。

环境：

• Windows 10，带有Python 3.8.0，带有命令"python3.\GeneralUse.py">

• Ubuntu 18.01，带有Python 3.6.8，带有命令"python3 GeneralUse.py">

• https://rextester.com/l/python3_online_compiler，带有Python 3.5.2

• https://repl.it/languages/python3，使用Python 3.7.4

class A:
def __init__(self):
# ==== case 1 =====
# Ubuntu: ok, Windows: ok
a = multiprocessing.Process(target=self.t, args=())
a.start()
a = multiprocessing.Process(target=self.t, args=())
a.start()
# ================
# ==== case 2 ====
# Ubuntu: ok, Windows: ok
self.b = multiprocessing.Process(target=self.t, args=())
self.b.start()
self.b = multiprocessing.Process(target=self.t, args=())
self.b.start()
# ================
# ==== case 3 ====
# Ubuntu: ok, Windows: Runtime Error
c = multiprocessing.Process(target=self.t, args=())
c.start()
self.d = multiprocessing.Process(target=self.t, args=())
self.d.start()
# ================
# ==== case 4 ====
# Ubuntu: ok, Windows: ok
self.e = multiprocessing.Process(target=self.t, args=())
self.e.start()
f = multiprocessing.Process(target=self.t, args=())
f.start()
# ================
# ==== case 5 ====
# Ubuntu: ok, Windows: Runtime Error
self.g = [multiprocessing.Process(target=self.t, args=()) for _ in range(2)]
for proc in self.g:
proc.start()
# ================
# ==== case 6 ====
# Ubuntu: ok, Windows: ok
h = [multiprocessing.Process(target=self.t, args=()) for _ in range(2)]
for proc in h:
proc.start()
# ================
time.sleep(1) 
def t(self):
print("Hi")

if __name__ == "__main__":
A()

Python在Windows上单独运行case 5时给出的错误。

Traceback (most recent call last):
File ".GeneralUse.py", line 52, in <module>
A()
File ".GeneralUse.py", line 42, in __init__
proc.start()
File "C:Program FilesPython3libmultiprocessingprocess.py", line 121, in start
self._popen = self._Popen(self)
File "C:Program FilesPython3libmultiprocessingcontext.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:Program FilesPython3libmultiprocessingcontext.py", line 326, in _Popen
return Popen(process_obj)
File "C:Program FilesPython3libmultiprocessingpopen_spawn_win32.py", line 93, in __init__
reduction.dump(process_obj, to_child)
File "C:Program FilesPython3libmultiprocessingreduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle 'weakref' object
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:Program FilesPython3libmultiprocessingspawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:Program FilesPython3libmultiprocessingspawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

Answer 1

这与操作系统无关。问题出在Python版本上。我使用Python 3.6.6在Windows上运行您的代码，它运行良好，但在Python 3.9(适用于Windows(上失败。你也有3.6为Ubuntu，但3.8为Windows，因此表明这一点。

当您启动另一个进程并将以前启动的进程馈送到此进程时，就会出现问题。这对于传递未启动的进程来说不是问题。在您的情况下，这是通过self完成的：self在过程开始时被腌制。

我在这里回答了同样的问题：https://stackoverflow.com/a/65749012/13696660

简而言之，我认为最干净的解决方案是从酸洗物体中去除有问题的过程：

class A:
def __init__(self):
...
def __getstate__(self):
# capture what is normally pickled
state = self.__dict__.copy()
# remove unpicklable/problematic variables
# (multiprocessing.Process in this case)
state['b'] = None
state['d'] = None
state['e'] = None
state['g'] = None
return state
...

现在它在我的3.9版Windows上运行。