同时运行多个Web驱动程序

Question

我正在运行代码blow。我正在尝试同时运行多个网页，这些网页将转到mail.google.com。例如，下面的代码应该在多个不同的窗口上同时打开mail.gooble.com网站。

from selenium import webdriver
from multiprocessing import Process, Pipe, Pool
from webdriver_manager.firefox import GeckoDriverManager
def create_browser(num):
# Defines Browser
browser = webdriver.Firefox(executable_path=GeckoDriverManager().install())
browser.get('https://mail.google.com/')
return browser

pool = Pool(processes=10)  # Maximum number of browsers opened at same time
for i in range(0, 2):  # Five browsers will be created
async_result = pool.apply_async(create_browser, args=(i))
pool.close()
pool.join()

但是浏览器永远不会打开。我经常在下面得到这个错误：

Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingspawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingspawn.py", line 125, in _main
prepare(preparation_data)
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingspawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingspawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0librunpy.py", line 265, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0librunpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0librunpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:UsersAdminDesktopPycharmProjectssnkrsNike_test.py", line 14, in <module>
pool = Pool(processes=10)  # Maximum number of browsers opened at same time
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingcontext.py", line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingpool.py", line 212, in __init__
self._repopulate_pool()
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingpool.py", line 303, in _repopulate_pool
return self._repopulate_pool_static(self._ctx, self.Process,
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingpool.py", line 326, in _repopulate_pool_static
w.start()
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingprocess.py", line 121, in start
self._popen = self._Popen(self)
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingcontext.py", line 327, in _Popen
return Popen(process_obj)
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingpopen_spawn_win32.py", line 45, in __init__
prep_data = spawn.get_preparation_data(process_obj._name)
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingspawn.py", line 154, in get_preparation_data
_check_not_importing_main()
File "C:Program FilesWindowsAppsPythonSoftwareFoundation.Python.3.8_3.8.1729.0_x64__qbz5n2kfra8p0libmultiprocessingspawn.py", line 134, in _check_not_importing_main
raise RuntimeError('''
RuntimeError: 
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.

Answer 1

问题是，您的每个子进程都在递归、无休止的循环中重新执行创建进程池及其子进程的代码。试着运行您的进程，在一个特殊的块中创建代码，如下所示：

if __name__ == '__main__':
pool = Pool(processes=10)  # Maximum number of browsers opened at same time
for i in range(0, 2):  # Five browsers will be created
#async_result = pool.apply_async(create_browser, args=(i))
# args must be an iterable. (i) is not a tuple but (i,) is:
async_result = pool.apply_async(create_browser, args=(i,))

pool.close()
pool.join()

处理结果：

if __name__ == '__main__':
with Pool(2) as pool: # don't create more processes than you need
# submit tasks:
result_objects = [pool.apply_async(create_browser, args=(i,) for i in range(2)]
# wait for tasks to complete and get reurn values
for result_object in result_objects:
return_value = result_object.get()
print(return_value)
# no need to do close or join

或者更简单：

if __name__ == '__main__':
with Pool(2) as pool: # don't create more processes than you need
return_values = pool.map(create_browser, range(2))
for return_value in return_values:
print(return_value)