如何在windows10中修复python中的多处理问题



我尝试使用本教程来训练我自己的汽车模型识别模型:https://github.com/Helias/Car-Model-Recognition.我想使用coda和我的gpu性能来提高训练速度(预处理步骤已经完成,没有任何错误(。但当我试图训练我的模型时,我遇到了以下错误:

######### ERROR #######
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.

######### batch #######
Traceback (most recent call last):
File "D:Car-Model-Recognitionmain.py", line 78, in train_model

######### ERROR #######
[Errno 32] Broken pipe
for i, batch in enumerate(loaders[mode]):

######### batch #######  File "C:Program FilesPython37libsite-packagestorchutilsdatadataloader.py", line 279, in __iter__
return _MultiProcessingDataLoaderIter(self)
Traceback (most recent call last):
File "C:Program FilesPython37libsite-packagestorchutilsdatadataloader.py", line 719, in __init__
File "main.py", line 78, in train_model
w.start()
File "C:Program FilesPython37libmultiprocessingprocess.py", line 112, in start
for i, batch in enumerate(loaders[mode]):
File "C:Program FilesPython37libsite-packagestorchutilsdatadataloader.py", line 279, in __iter__
self._popen = self._Popen(self)
File "C:Program FilesPython37libmultiprocessingcontext.py", line 223, in _Popen
return _MultiProcessingDataLoaderIter(self)
File "C:Program FilesPython37libsite-packagestorchutilsdatadataloader.py", line 719, in __init__
return _default_context.get_context().Process._Popen(process_obj)
File "C:Program FilesPython37libmultiprocessingcontext.py", line 322, in _Popen
w.start()
return Popen(process_obj)
File "C:Program FilesPython37libmultiprocessingpopen_spawn_win32.py", line 46, in __init__
File "C:Program FilesPython37libmultiprocessingprocess.py", line 112, in start
prep_data = spawn.get_preparation_data(process_obj._name)
File "C:Program FilesPython37libmultiprocessingspawn.py", line 143, in get_preparation_data
self._popen = self._Popen(self)
File "C:Program FilesPython37libmultiprocessingcontext.py", line 223, in _Popen
_check_not_importing_main()
File "C:Program FilesPython37libmultiprocessingspawn.py", line 136, in _check_not_importing_main
return _default_context.get_context().Process._Popen(process_obj)
File "C:Program FilesPython37libmultiprocessingcontext.py", line 322, in _Popen
is not going to be frozen to produce an executable.''')
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
return Popen(process_obj)

我已经使用了给定链接中的确切代码,如果我使用wsl启动代码,一切都可以,但我不能使用wsl中的gpu。我应该在哪里插入这个name=='main检查以防止出现这样的错误,或者我如何禁用这个多处理

查看main.py,您可以在模块级别运行大量代码。在Windows上,python的multiprocessing模块将启动一个新的python解释器,导入模块,取消拾取父上下文的快照,然后调用辅助函数。问题是,所有的模块级代码都只是通过导入来执行,而实际上是运行程序的新副本,而不是为工作人员构建上下文。

解决方案有两个方面。首先,将所有模块级代码移动到函数中。你想成为一个能够导入你的模块没有副作用。其次,调用从条件启动程序的函数

def main():
the stuff you were doing a module level
if __name__ == "__main__":
main()

这样做的原因在于模块名称。当您运行python的顶级脚本(例如python main.py(时,它是一个名为"__main__"的脚本,而不是一个模块。如果另一个程序导入main,它就是一个名为"main"的模块(或者您为脚本命名的任何模块(。如果主代码是由其他python代码(如多处理模块(导入的,则"if"会阻止主代码执行。

在模块级别上有一些可执行代码是可以的,尤其是在设置默认值之类的情况下。但是,如果其他代码导入您的脚本,请不要在模块级别执行任何您不希望执行的操作。

最新更新