如何使用chromedriver和chrome模拟phantomjs行为，以通过Python和contextlib使用S

Question

我遇到的一些网页有从javascript代码生成的链接，我只能按照下面的代码使用phantomjs访问它们。

 dcap = dict(DesiredCapabilities.PHANTOMJS)
        dcap["phantomjs.page.settings.userAgent"] = "Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; Microsoft; Lumia 640 XL LTE) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Mobile Safari/537.36 Edge/12.10166"
        driverpjs = webdriver.PhantomJS("/Users/xx/Downloads/phantomjs-2.1.1-macosx/bin/phantomjs",desired_capabilities=dcap)
with contextlib.closing(driverpjs) as browser:
                browser.get(link)
                links = browser.find_elements_by_xpath('.//a')

我如何使用铬执行此操作？现在我正在尝试以下方法：

options = webdriver.ChromeOptions()
        options.add_argument("headless")
    options.add_argument('--user-agent="Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; Microsoft; Lumia 640 XL LTE) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Mobile Safari/537.36 Edge/12.10166"')
        driver =  webdriver.Chrome(executable_path="/usr/local/bin/chromedriver", chrome_options=options)
with contextlib.closing(driver) as browser:
                browser.get(link)
                # GET ALL LINKS                                                                                                                                                                                                           
                #links = browser.find_elements_by_css_selector("a")                                                                                                                                                                       
                links = browser.find_elements_by_xpath('.//a')

Answer 1

要使用 contextlib 获取页面上模拟 PhantomJS 与 Chrome 类似功能的所有链接，您可以使用以下解决方案：

• 代码块：

  from contextlib import closing
  from selenium import webdriver
  options = webdriver.ChromeOptions()
  options.add_argument("headless")
  options.add_argument('--user-agent="Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; Microsoft; Lumia 640 XL LTE) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Mobile Safari/537.36 Edge/12.10166"')
  driver =  webdriver.Chrome(executable_path=r'C:WebDriverschromedriver.exe', chrome_options=options)
  with closing(driver) as browser:
      browser.get("https://www.google.com/")
      # get all the elements with name as q                                                                                                                                                                                                          
      print(browser.find_elements_by_name('q'))
  

• 控制台输出：

  [<selenium.webdriver.remote.webelement.WebElement (session="ab581b3b679b521ffa5bf2220f801fcf", element="0.39081088826075705-1")>]