我通过使用chromedriver的解析器收集评论,我在服务器上的芹菜任务中运行解析,我面临以下错误
(The process started from chrome location /home/ubuntu/.../chromedriver
is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
这是ChromeDriverManager返回的路径。
我需要在服务器上安装和配置谷歌浏览器吗?这里是我的chrome选项:
chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument("--disable-extensions")
chrome_options.add_argument("--headless")
# Adding argument to disable the AutomationControlled flag
chrome_options.add_argument("--remote-debugging-port=9222")
chrome_options.add_argument("--disable-dev-shm-usage")
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--disable-blink-features=AutomationControlled")
chrome_options.add_argument("start-maximized")
# Exclude the collection of enable-automation switches
chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])
chrome_options.add_argument("window-size=1280,800")
# Turn-off userAutomationExtension
chrome_options.add_experimental_option("useAutomationExtension", False)
path = ChromeDriverManager().install()
chrome_options.binary_location = path #"/home/ubuntu/chromedriver"
service = Service(path)
driver = webdriver.Chrome(service=service, options=chrome_options)
如果有人遇到同样的问题,请尝试下载chromium-browser并更改binary_path=/usr/bin/chromium-browser,请记住,您必须设置相同的版本。